About ten hours a week. Follow the rows in order, including the baseline and the README.
| Week | Focus | Do this | Done when |
|---|---|---|---|
| 6 | Problem + split | Recheck the target and reserved split (random / time / group). Justify it before fitting | README records the split and earlier exploration |
| 7 | Baseline + leakage | Prior-probability baseline (DummyClassifier). Leak preprocessing on purpose, then fix it with Pipeline | Before/after scores in README |
| 8 | Logistic regression | ISL or scikit-learn MOOC. Pick a metric tied to cost | Beats the baseline, or explains why to keep the baseline |
| 9 | Trees | Random forest + one boosted tree. SHAP on one wrong prediction | Quality, calibration, runtime/memory and latency compared |
| 10 | Calibration + threshold | Reliability plot on held-out data. Tune the decision on validation costs | Can explain probability and threshold separately |
| 11 | API | FastAPI /predict with Pydantic validation. Test a valid and an invalid request | Served predictions match saved bundle. Bad input returns 422 |
| 12 | Polish | Confidence intervals, a "what failed" section and a load-test note | README is the deliverable |
| 13 | Buffer | Rebuild one model from an empty file. Practise your leakage story for interviews | Can rebuild without a tutorial |
The table uses binary classification as its first-project example. For a numeric target, use
DummyRegressor, regularised linear regression, RandomForestRegressor and a boosted regressor.
Swap the threshold exercise for error/interval analysis and a cost-based decision rule. The same
split, holdout and API requirements apply. A well-explained negative result completes the
comparison. Avoid searching for a misleading split just to beat the baseline.
For a first multiclass project, compare a class-prior baseline. Report a confusion matrix and per-class precision/recall alongside the aggregate you chose. A binary cost threshold does not carry over to several classes. Define the cost of each action/outcome pair and choose the lowest expected cost using suitable probabilities. Multilabel tasks also need their own label-level decision and evaluation design. Leave them for later if the basic workflow is new.
In your uv project, start with uv add scikit-learn. In Week 9, add the boosted-tree and SHAP
packages you chose, following their official install guides, and commit the updated lock. Keep one
boosted candidate. Installing three libraries is not a model comparison.