Developing ML solutions
Machine Learning Development Lifecycle
The ML lifecycle is the end-to-end process of developing, deploying, and maintaining models. Its phases, in order:
- Business goal identification: stakeholders set the value, budget, and success criteria (KPIs).
- ML problem framing: turn the business problem into an ML problem.
- Data processing: collection, preprocessing, and feature engineering.
- Model development: training, tuning, and evaluation.
- Deployment: inference and prediction.
- Monitoring
- Retraining
It needs collaboration between product managers, developers, data scientists, and engineers.
Worked example: Amazon call center routing
| Phase | What Amazon did |
|---|---|
| Business goal | Reduce call transfers caused by a menu that routed customers to the wrong agent |
| Problem framing | Predict the agent skill a call needs: multiclass classification using supervised learning on historical calls |
| Data collection | Features such as recent orders, Kindle ownership, and Prime membership |
| Preprocessing and visualization | Cleaned the data, merged similar labels (all Kindle skills into one), and explored label distributions |
| Training | Split labeled data into training, validation, and test sets, typically 80/10/10 or 70/15/15. Never evaluate on training data |
| Tuning | Hyperparameter optimization, such as learning rate (too fast never converges, too slow takes too long), plus more feature engineering |
| Evaluation and deployment | Transfers dropped and customer experience improved |
Feature engineering creates and transforms the input variables the model learns from. The model only learns from what you show it.