Deploying a model into production
von Deepak Nagaraj
1. Deployment
1.1. Use Docker for reproducible analyses if possible
2. Performance
2.1. Use load test
2.1.1. Test with >= 2X the max requests you expect to see
3. Documentation
3.1. Write API doc before you write code
3.2. Endpoint URLs
3.3. Request format
3.4. Response format, content
3.5. System requirements, how to install
3.6. Why the model exists and the basic methods behind it
4. Monitoring
4.1. Throughput: requests/hr
4.2. Assessment metric: is it accurate?
4.3. Any errors?
5. Interfacing
5.1. Take the time to design the API well
5.2. As important as the model itself
5.3. REST is the norm because any technology can connect
5.4. R: Plumber, Python: Flask
5.5. R Web apps: use Shiny
6. Robustness
6.1. Code should not blow up on bad inputs
6.2. What happens on the margins is really important in production systems
6.3. Try to have 100% coverage
7. Maintainability
7.1. Can run for years!
7.2. Follow good software engg practices
7.3. Follow standards of other models
7.4. Don't use a long-dead programming language
8. Explainability
8.1. Use the simplest acceptable model
8.1.1. Netflix had to retire its prize-winning algorithm
8.1.2. The extra complexity wasn't worth the 10% gain