← Back to the full chapter
Lesson

What to build

Build a baseline, protect the holdout, and connect a prediction to the decision it informs.

3 min read80 hours practice planWeeks 6–13Updated
Read source ↗

A model behind an API. Take your Chapter 2 dataset and complete the loop:

  1. A split you chose deliberately and justified in writing
  2. All preprocessing inside a Pipeline
  3. A simple baseline, then a linear/logistic model, a random forest and a boosted tree for the target type
  4. A metric tied to a stated cost, reported with confidence intervals
  5. A classification threshold or regression decision rule taken from the stated costs and constraints
  6. SHAP explanations, including a misclassified case or a large-error regression case
  7. A FastAPI endpoint with schema validation, and tests for both code and data
  8. A reliability plot or regression error analysis, an untouched test comparison, and documented label maturity, outcome coverage and calibration/decision-selection splits
  9. A versioned model/preprocessor/calibrator bundle with dependency and data provenance, plus a fresh-process API check

The README is the deliverable. Include a table of every model and what it bought you, a section on what did not work, and a section on known failures. Those last two are unusual in a junior portfolio and they do a lot of quiet good.

Return a model version with predictions, validate feature names/types/units, and define behaviour for missing or unknown categories. Serve the preprocessing/model/calibration bundle you tested, and keep retraining and scaler fitting out of the request. For Week 11, add the API and testing dependencies:

uv add "fastapi[standard]"
uv add --dev pytest httpx
uv run --locked pytest

Follow the official request-body and testing guides. In a fresh process, test a valid request against the saved bundle's prediction, including the probability or numeric prediction, model version and chosen decision rule. Also test a malformed request and your chosen handling of missing/unknown inputs. Load a Python model artifact with the training environment's dependency versions. Loading across scikit-learn versions is unsupported, as the persistence guide explains. Document measured latency, memory and CPU fallback. Chapter 6 covers production operations.

The Week 7 leakage exercise: fit preprocessing on the full dataset on purpose and record the score. Then move the fitting inside a Pipeline and repeat on the same split. Report the real difference, even if it is small, since this leak does not always change the score much. Availability-time and group errors need their own data or split fixes. A pipeline fixes this kind of leakage, not every kind.

Handoff to Chapter 4. Commit the split definition, feature pipeline and every model's results. Chapter 4 can train a small network on these same features and compare it with these numbers on the same rows. Keep the split fixed and the test untouched.

Something unclear or out of date? Suggest a correction ↗

Find your next idea

Loading local search…

Search is built from this edition. No account needed.