← Back to the full chapter
Lesson

Your first AI product: a two-week plan

Use real examples and independent evaluations to build a useful language-model application.

4 min read100 hours practice planWeeks 23–32Updated
Read source ↗

One user, one job, one screen, one main quality metric, plus cost, latency and access checks. Spread about 20 hours across these sessions, reusing weeks 23–26 work. Cut down any task needing complex integration or a custom authentication system before this sprint.

Write the non-goals on day one. No payments, autonomous writes, custom vector store, mobile app or model training. Use public/non-sensitive data, or platform authentication with verified per-user access for a private application. Optional agents and tuning experiments come later, in weeks 29–31.

DayWhat you doWhat exists at the end
1Name the person and the task. Write the README first: problem, user, success metric, non-goalsA scoped problem
2Review the 30–50 cases already collected; split independent familiesDevelopment cases + reserved release cases; grading assets isolated
3The roughest possible end-to-end path: hardcoded input, one call, printed outputYou have hit every part of the pipeline once
4Assertions on development cases; record versions and settingsQuality and cost baseline
5If sources are needed, ingest/index with source IDs and access metadataSmall retrieval path; otherwise improve input validation
6Compare two retrieval/context variants on labelled queriesSource coverage, quality and latency comparison
7Generation: structured output, citations to source spans, a path for "I do not know"Grounded answers
—Weekend check: a pass-rate number exists, or reduce scope nowAn honest decision point
8A thin interface; public data or platform authentication with per-user access checksSomething usable with its intended data permissions
9Redacted request logs, restricted access and the crudest possible trace viewerYou can inspect failures without exposing private inputs
10Read development failures; group them; add development casesError taxonomy; release holdout untouched
11Fix one common failure and compare on developmentMeasured change, including no improvement if that is the result
12Minimal read-only deploy; server-side secrets if needed; request/spending bounds; safe fallbackA usable URL or documented local demo if hosting is unavailable
13Validate a judge if needed; run regression command; freeze candidate and release criteriaReproducible local checks and declared decision rules
14Run reserved release cases; save demo, costs and known failuresRelease or defer decision supported by results

Use deterministic checks for objective tasks. A model judge is optional. Run some repeated trials for stochastic or tool-heavy cases within budget, and disclose the count. If a release run fails, investigate on development data rather than rerunning it until it happens to pass. If your changes were tuned to the old holdout, collect fresh release cases.

The Chapter 5 minimum is explicit: a read-only, narrowly scoped interface, a repeatable local regression command, independent quality evidence, known failures and limits on any paid requests. Use synthetic/public inputs, or verified platform authentication for private ones. A static demo with recorded or deterministic responses is a valid first deployment if it is labelled accurately and does not claim to call a live model. If you add a paid API, keep its credential on the server, validate requests, cap input and output, enforce request and spending limits, and fail safely. These controls apply now, before the later hardening chapter.

Complete in Chapter 5Extend in Chapter 6
Repeatable local regression commandRun the command in a least-privilege CI workflow
Minimal platform deploy or documented local demoContainer build, readiness and measured serving load
Request bounds, safe fallback and a measured cost tableStructured monitoring, alerts and a rehearsed rollback
Independent release casesVersioned release process and outcome-feedback loop

Chapter 6 takes this exact application as its starting point. Its container and CI work is not needed to complete Chapter 5. Check current hosting eligibility and charges before choosing a route. Hugging Face Spaces still offers free Static Spaces. Creating Gradio or Docker Spaces has account-plan requirements, with some ZeroGPU exceptions. Free CPU hardware and permission to create a Space are separate questions. Modal pricing separates compute credits from Shared Endpoints, whose per-token charges those credits do not cover. Neither route guarantees free model API calls or a free running demo.

Completion means a usable URL or documented local-demo fallback, a candid independent quality report, a repeatable local regression command, the access and spending controls that deployment needs, known failures, a cost table and feedback from at least one other user. A measured lack of improvement is a valid result. Release claims need independent evidence.

After release, model/SDK upgrades, prompt changes and corpus refreshes are separate versioned changes. Recheck retrieval coverage, access filtering, task quality and cost before rollout. Keep the previous compatible configuration and index for rollback, subject to current permissions and deletions. A rollback must never restore revoked access or withdrawn sources. Review sampled real failures and user corrections under your trace-retention policy. Chapter 6 develops this operating loop. If the sprint's release results shape later changes, collect fresh release cases for week 32 and keep the earlier ones for regression testing.

Something unclear or out of date? Suggest a correction ↗

Find your next idea

Loading local search…

Search is built from this edition. No account needed.