Applied track after Chapters 3 and 6 · about 40 hours for one case, 90 for all three.
Book index · Chapter resources
The short version. Build a system that chooses a useful business action and measures its effect. Offer recommendation, price recommendation and churn share data and operations, but ask different statistical questions. Start with a transparent baseline, test a stronger model on a realistic split, and ask for proof before turning predictions into interventions.
The examples use retail and subscriptions. Adapt the outcome, allowed actions, customer policies and economics to your domain. This is an engineering recommendation based on the research below, not a promise that one model wins in every company.
In this chapter: Plan · Decision trees · Model choices · Architecture · Build · Completion
Your applied plan
Read the short causal primer and the shared delivery plan, then choose a guide. Use one case for the Chapter 7 capstone, or finish all three as one connected business project. The hours are for a small learning prototype on public or synthetic data. Collecting real data over time and running a powered business experiment take extra calendar time.
These are scaffolded practice budgets: you reuse the earlier environment, a small prepared dataset and the service boundary. Starting from a blank repository or collecting your own historical data needs extra setup time. The reference lab gives a synthetic starting point, so it cannot show a real business effect. On the first pass, finish the baseline, constraints and evidence report before opening the optional research sections.
| Block | Hours | Work | Deliverable |
|---|---|---|---|
| Shared setup | 5 | Decision brief, economics, data/label contract, experiment assumptions | Written specification and snapshot checks |
| Offer recommendation | 25 | Response baseline, identified uplift, constrained offer selection | Policy comparison and campaign delivery contract |
| Price recommendation | 25 | Demand/elasticity baseline, challenger, approved-price optimiser | Demand backtest and price experiment proposal |
| Churn | 25 | Risk baseline, calibrated ranking, survival/retention extension | Cohort evaluation and retention decision memo |
| Shared delivery | 10 | Feedback join, shadow run, budget/freshness checks, monitoring and rollback | Reproducible package and prototype launch-readiness review |
One guide plus the two shared blocks takes about 40 hours. Every row takes 90. Finish one baseline before adding a second model family. The starting models do not need a GPU. The shared delivery plan lists the artifacts and the completion gates.
The questions are different
| Use case | Prediction question | Decision question | Main outcome |
|---|---|---|---|
| Offers | Will this customer respond to this offer? | Which eligible action improves margin relative to no offer? | Incremental net contribution per eligible customer |
| Prices | How much demand is expected for this item and context? | Which supported, permitted price improves contribution under demand and inventory constraints? | Net contribution per market/item opportunity over a fixed horizon |
| Churn | Who will leave, and when? | Whose retention improves enough under an action to justify its cost? | Incremental retained contribution after contact and incentive costs |
Prediction estimates outcomes in observed data. Intervention estimates what would change under an action. A model can predict well from patterns that break once you act, as scikit-learn shows in its causal interpretation example. Randomisation is the preferred start for actionable effects. Observational designs need stated identification assumptions and sensitivity analysis, and a more complex learner cannot make up for missing identification.
Choose your ML approach
Start with the business question and the evidence you have. Follow the decision tree in the right guide, then read the matching recommendation: which model to start with, why it fits, what data and evidence it needs, and what to validate before adding complexity.
| Use case | Model-selection decision tree | The distinction to resolve first |
|---|---|---|
| Offer recommendation | Choose an offer approach | Product relevance, response prediction, or the incremental effect of an incentive |
| Price recommendation | Choose a pricing approach | Forecast demand at observed prices, or estimate demand under a price change |
| Churn and retention | Choose a churn approach | Predict fixed-horizon risk, estimate event timing, or learn a retention action's effect |
You can combine branches. A recommender can retrieve candidates and estimate offer effects. A churn system can estimate both risk and retention effects. The final business policy still applies costs, eligibility, budget, capacity and the approved fallback.
Each model branch starts with a baseline and has an evidence gate for promotion. If timestamps, outcomes, costs or action support are missing, do the data-collection step first. Recent research challengers stay in the detailed model sections, with their reproduction and evaluation conditions.
Suggested models and approach
These are starting candidates. The final choice comes from your business metric, fair evaluation over time, inference cost, calibration and evidence about treatment effects. Only a model that improves the decision enters the production path.
| Case and data regime | Start with | Strong practical candidate | Add complexity when | Avoid |
|---|---|---|---|---|
| Small offer set, tabular customer data | Eligibility rules, no-offer control, logistic response baseline | CatBoost or LightGBM response model; randomised T/S-learner uplift and incremental-profit policy | Compare X/DR learners or causal forests when arm sizes, heterogeneity and overlap support them | Ranking by response probability and calling it incrementality |
| Large product/offer catalog, interaction history | Popularity/content candidates plus eligibility | Two-stage retrieval and ranking; LambdaMART/boosted ranker | Two-tower retrieval when interaction volume and catalog scale justify learned embeddings | Treating logged non-clicks as unbiased preference labels |
| Demand/pricing with adequate exposure data | Seasonal/current-price policy; log-price Poisson/negative-binomial GLM | CatBoost or LightGBM demand challenger plus an approved-grid profit optimiser | Flexible elasticity or DML/causal forest after identification; limited bandit exploration after logging and policy approval | Regressing historical selling price and presenting it as an optimal price |
| Fixed-horizon churn with mature labels | Recency rule and logistic regression | Calibrated CatBoost or LightGBM, evaluated at contact capacity | Better features or another learner only after realistic error analysis | Using accuracy or a universal 0.5 cutoff |
| Churn timing with censored follow-up | Kaplan–Meier cohort summary and Cox baseline | Random survival forest or survival boosting | Non-proportional hazards/time-varying approaches when diagnostics require them | Labelling customers with unfinished follow-up as non-churners |
| Retention intervention with randomised actions | No-contact control and average treatment effect | Uplift/DR model plus expected incremental retained contribution | Multi-arm treatment policy when there is enough support per action | Assuming highest churn risk means highest save probability |
CatBoost's categorical-feature support makes it convenient for many customer tables. LightGBM is another candidate when its training, memory and serving behaviour suit the data. Neither wins by default, so compare on the same splits and cost budget. Current tabular results depend on model version and evaluation protocol. Compare BeyondArena’s June 2026 benchmark with the September TabPFN 3.5 report. The second reports aggregate gains but does not lead everywhere across full grouped, temporal and large-data slices, and its group-feature protocol differs. Chapter 3 explains how to make the comparison match deployment.
The guides explain the specific conditions behind each row. Read them before choosing a library.
How to use the newest research
The research review includes sources available through 9 October 2026. The guides include optional challengers: causal, generative recommendation, inventory-learning and survival foundation models. Being new does not make them the best starting models. Some are recent preprints or workshop results. Industrial engagement results also answer a different question from incremental business contribution.
Promote a candidate only after it clears these comparisons:
- Task and evidence: match the prediction, treatment, censoring and deployment setting to yours. Write the identifying assumptions before fitting.
- Reproduction: check the release, licence, preprocessing, tuning budget and evaluation split. Compare on the same data, action support, hardware and allowed candidate count.
- Decision value: measure calibrated risks or effect sizes, supported policy value and uncertainty, as well as ranking. Include no action and a simple nonpersonalised policy.
- Operation: test cold start, missing inputs, drift, inference cost, latency, constraints and fallback. A research benchmark says little about reliability in production.
The practical starting point remains a transparent baseline and a calibrated boosted-tree challenger for tabular prediction, identified causal estimation for interventions, and an explicit constrained decision policy. Optional research comparisons sit outside the prototype time budget.
One architecture, three decision paths
Use the same lifecycle and execution contract for all three cases. Only the model output changes: offer response or uplift, price-dependent demand, or churn risk, survival and retention effects.
Diagram source
%%{init: {"theme":"base","themeVariables":{"fontFamily":"Arial, sans-serif","fontSize":"16px","textColor":"#17324D","primaryColor":"#F5F7FA","primaryTextColor":"#17324D","primaryBorderColor":"#8796A8","lineColor":"#718096","secondaryColor":"#EFF6FF","tertiaryColor":"#E9F6F2","clusterBkg":"#F8FAFC","clusterBorder":"#D6DEE8","edgeLabelBackground":"#FFFFFF"},"flowchart":{"htmlLabels":true,"nodeSpacing":18,"rankSpacing":28,"curve":"linear","padding":10,"subGraphTitleMargin":{"top":8,"bottom":12}}}}%%
flowchart TB
accTitle: Shared business ML architecture
accDescr: Prepare point-in-time snapshots, validate the case model, choose a permitted action, persist the assignment before execution, and join mature outcomes for monitoring and the next validated release.
S["Point-in-time snapshots<br/>Verified training targets"]
M["Validate case model<br/>Offers · prices · churn"]
D["Constrained policy<br/>Include approved default"]
A["Record before delivery<br/>Reserve + execute once"]
F["Join mature outcomes<br/>Costs + contribution"]
S --> M --> D --> A --> F
F -.->|Evaluate + refresh| S
classDef data fill:#EFF6FF,stroke:#3973AC,color:#17324D,stroke-width:1.5px
classDef model fill:#EFF2FD,stroke:#596FC2,color:#17324D,stroke-width:1.5px
classDef decision fill:#FFF5DF,stroke:#A8751A,color:#17324D,stroke-width:1.5px
classDef action fill:#E9F6F2,stroke:#23866E,color:#17324D,stroke-width:1.5px
classDef neutral fill:#F5F7FA,stroke:#8796A8,color:#17324D,stroke-width:1.5px
class S,F data
class M model
class D decision
class A actionBlue: data and outcomes. Indigo: model validation. Amber: policy decisions. Teal: delivery. Labels identify each role as well as colour.
Training targets need mature fixed-horizon outcomes or properly recorded event and censoring follow-up, as set out in the data contract. Save the original decision, assignment and relevant logging probabilities before delivery. Record delivery events separately, including nondelivery and overrides. The delivery lifecycle shows the order in detail.
The coordinator is application logic or an optimisation step. An LLM can help explain approved results or extract inputs from documents. Price, eligibility, causal effects and budget limits come from your data and rules, not from the LLM.
Start with one pipeline, a SQL snapshot job, one fitted model artifact and one batch job or API. Add a feature store, registry platform, embedding service or bandit when a concrete need justifies its running cost. The feedback loop is the architecture that matters.
Make the three systems cooperate
Offers change the price the customer actually pays. Retention offers use the same customer attention and often the same subsidy budget. If each system maximises its own score alone, a customer can get competing discounts, a campaign can contaminate a pricing test, and the same revenue can be counted twice.
Log list price, recommended price, final paid price, subsidy source and every concurrent campaign assignment. Start with mutually exclusive experiments or one shared assignment plan. Give one coordinator ownership of the final action, including no change/no contact. Cap total spend, contact frequency and allowed price movement. Treat customer-level policy effects and inventory spillovers as part of experiment design.
Optimise combined actions only after you have data supporting them. Separate offer and price models cannot identify the effect of an untested discount-plus-price change. Use a factorial design (testing the combinations) where it fits, or test the combined policy directly. Report total net contribution once, with component metrics to explain it.
For logged policy evaluation, record the coordinator's joint assignment probability, or its stage-by-stage rule, with pre-decision budget, stock and contact state. Separate marginal probabilities do not usually describe a combined action. A new policy can also change future inventory and eligibility, so a one-decision evaluation cannot show the whole system's long-term value. The shared policy evaluation protocol sets out these limits and the controlled-test path.
What to build
A business decision system with an evidence report. Choose one guide and follow the delivery plan. Publish the decision brief, data contract, model comparison, policy comparison, experiment protocol, batch/API contract, monitoring and rollback artifacts. For private business data, publish code and synthetic examples instead of customer records.
Keep three evidence levels separate in the report:
| Evidence | What you may claim |
|---|---|
| Synthetic simulator with known effects | The pipeline and policy work under the simulator's assumptions |
| Retrospective public or business data | Predictive performance on that held-out population; causal effects only when the design supports identification |
| Valid randomised policy experiment | An estimated effect of that policy in the tested population, period and operating conditions |
A finished prototype can have an experiment proposal and a rehearsed rollout. A measured live profit lift needs executed assignments and observed outcomes. Say this plainly.
Before you move on
- You can state the decision, outcome horizon, baseline and unit of assignment in one minute.
- The feature query excludes events not yet available. Fixed-horizon labels are mature, and survival evaluation uses recorded censoring and supported follow-up.
- You can defend your model choice against a simple baseline using the same data and budget.
- You know which effects are identified, and where the policy must abstain or fall back.
- You can trace one chosen action through execution, cost, outcome and experiment assignment.
- You can disable the policy and reproduce its last release from its recorded inputs.
Resources
The annotated source list separates official implementation documentation, research papers and practice datasets. The three guides cite their supporting sources directly.
Previous: Finding the work · Delivery plan · Back to the index