← Back to the full chapter
Lesson

What to build

Turn messy records into a reproducible analysis, with a clear account of what you know.

2 min read50 hours practice planWeeks 1–5Updated
Read source ↗

A published data product. A public repository containing:

  • A README stating the question in one sentence, the answer in one sentence, and the limitation in one sentence
  • pyproject.toml, uv.lock, Python/platform details and a command tested after uv sync --locked
  • A src/ package with your cleaning functions, not just a notebook
  • At least eight meaningful checks spanning cleaning functions, unique keys, joins, timestamps and missingness
  • Your cleaning decision log
  • Only charts that answer named questions, each with a denominator, units, source and limitation. Cover the relevant distributions, missingness/coverage and time or group variation. Show uncertainty when making an estimate. Use readable labels and a palette that works without colour.
  • A data contract covering row keys, both timestamps, feature windows, label maturity, actual outcome availability/coverage and permitted use
  • A source manifest with raw-input checksums, extraction cutoff and documented exclusions

Give the package an analysis entry point and document its exact uv run --locked ... command. It must read the documented inputs and rebuild the cleaned output, summary results and chosen charts without notebook state. Run uv run --locked pytest separately. Passing tests alone do not reproduce the analysis.

Choose a dataset that you found yourself: a city open-data portal, a government export, a niche marketplace, or your own data exported from a service you use. Before Week 4, pick an outcome you could later predict, such as a delay, a price or a category, and the inputs available at prediction time. Chapter 3 will ask you to model it. In the handoff, state the prediction time, row/group key, reserved holdout and a workable split. If the dataset cannot support a supervised question, change the dataset and write down why. Avoid inventing labels or using the outcome as an input.

Something unclear or out of date? Suggest a correction ↗

Find your next idea

Loading local search…

Search is built from this edition. No account needed.