Machine Learning Development Lifecycle
The seven phases
The story: Building a house goes in order. The family decides what they need and their budget. An architect turns "we need room for three kids" into a floor plan. You buy and prepare materials. Builders put it up and inspectors check it. The family moves in. You keep an eye on cracks and leaks. Every few years you renovate. And it takes the family, architect, builders, and inspectors working together.
In AI/AWS terms: The ML lifecycle is the end-to-end process of developing, deploying, and maintaining models. Its phases, in order:
- Business goal identification: stakeholders set the value, budget, and success criteria (KPIs).
- ML problem framing: turn the business problem into an ML problem (the floor plan).
- Data processing: collection, preprocessing, and feature engineering (materials).
- Model development: training, tuning, and evaluation (building and inspecting).
- Deployment: inference and prediction (moving in).
- Monitoring (watching for cracks).
- Retraining (renovating).
It needs collaboration between product managers, developers, data scientists, and engineers.
For the exam: Know the order. It starts with a business goal and KPIs, not with data or a model.
Worked example: Amazon call center routing
The story: You call a helpline, press "2 for orders", and still get sent to the wrong person, who transfers you again. Amazon had this problem and fixed it with ML.
In AI/AWS terms:
| Phase | What Amazon did |
|---|---|
| Business goal | Reduce call transfers caused by a menu that routed customers to the wrong agent |
| Problem framing | Predict the agent skill a call needs: multiclass classification using supervised learning on historical calls |
| Data collection | Features such as recent orders, Kindle ownership, and Prime membership |
| Preprocessing and visualization | Cleaned the data, merged similar labels (all Kindle skills into one), and explored label distributions |
| Training | Split labeled data into training, validation, and test sets, typically 80/10/10 or 70/15/15. Never evaluate on training data |
| Tuning | Hyperparameter optimization, such as learning rate, plus more feature engineering |
| Evaluation and deployment | Transfers dropped and customer experience improved |
For the exam: Picking one of several categories from labeled history is multiclass classification with supervised learning.
Splitting the data
The story: A teacher has 100 practice questions. They use 80 for lessons, keep 10 for weekly quizzes to see if students really understand, and lock away 10 for the final exam. If the final exam reused lesson questions, the grades would be meaningless.
In AI/AWS terms: Split labeled data into training, validation, and test sets, typically 80/10/10 or 70/15/15. Never evaluate on training data.
For the exam: Typical splits are 80/10/10 or 70/15/15. Never evaluate on the training data.
Hyperparameters and learning rate
The story: Adjusting a shower's temperature. Turn the knob a huge amount each time and you swing between freezing and scalding, never landing on warm. Turn it a tiny bit each time and you'll be standing there for ages.
In AI/AWS terms: Hyperparameters are settings you choose before training, like the learning rate. Too high and training never converges. Too low and it takes too long. Hyperparameter optimization searches for good settings.
For the exam: Learning rate too high = never converges. Too low = too slow.
Feature engineering
The story: A doctor diagnosing you doesn't just read your raw weight and height. They compute your BMI, because that single number tells them more. And if they never ask about your family history, they can't use it.
In AI/AWS terms: Feature engineering creates and transforms the input variables the model learns from. The model only learns from what you show it.
For the exam: Feature engineering = creating and transforming input variables so the model can learn from them.