Resources

Resources · Business machine learning

Annotated reading and implementation references for Chapter 9.

10 min readApplied extensionUpdated
Read source ↗

Chapter  ·  Offers  ·  Prices  ·  Churn

Sources checked for the October 2026 revision. The recommendations combine these sources. Each paper or document shows a capability or result under stated assumptions, not a universally best model. Read each source alongside its task instead of working through the list. Each guide explains the assumptions behind the methods it recommends.

Browse: Start here  ·  Offer recommendation and retention effects  ·  Pricing, elasticity and inventory  ·  Churn timing and evaluation  ·  Policy evaluation and operation  ·  Practice datasets: what each can prove

Start here

ResourceTypeUse it forKeep in mind
CatBoost categorical featuresOfficial docsMixed categorical customer/offer baselinesIts feature handling does not repair temporal leakage or confounding
LightGBM advanced topicsOfficial docsCategorical handling, LambdaRank and position biasA ranker needs meaningful query groups and labels
Probability calibrationOfficial docsReliability plots and independent calibrationProper scoring losses also reflect discrimination; refitting changes the mapping
Feast point-in-time joinsOfficial docsReplaying historical featuresVerify availability timestamps and historical corrections in your implementation
Microsoft pre-experiment practicesPractitioner researchHypothesis, assignment, metrics and power before launchYour assignment unit and outcome horizon determine the design

Offer recommendation and retention effects

Künzel et al. — Meta-learners for estimating heterogeneous treatment effects

Research paper · published in PNAS, 2019 · preprint linked. Read the S/T/X learner setup and the assumptions behind each learner's advantages. It supports comparing simple treatment-effect learners first. It names no universal winner and does not validate treatment effects in unrandomised CRM data.

EconML — Doubly robust learning

Official implementation documentation. Use it for categorical or multi-arm treatment estimation, nuisance models and cross-fitting. You still need identification and overlap. Choose an estimator and inference method suited to the treatment and data. Use known randomisation probabilities where supported.

Ascarza — Retention Futility: Targeting High-Risk Customers Might Be Ineffective

Journal of Marketing Research, 2018. The field experiments separate churn risk from retention response. The tested campaigns and populations are specific. The lesson is to measure who responds, instead of assuming their treatment effects transfer to your company.

TensorFlow Recommenders — Basic retrieval

Official tutorial. A practical two-tower candidate-retrieval example. Use when catalogue scale and interaction history justify embeddings. Retrieving movie preferences is a different goal from deciding whether a paid incentive creates incremental profit.

The retrieval task API documents accidental-hit removal and negative-sampling correction. The side-feature tutorial feeds metadata into the trained tower representations. Keep training-sampler probabilities separate from campaign assignment probabilities, and evaluate new-item retrieval explicitly.

Google — Deep Neural Networks for YouTube Recommendations

Covington, Adams and Sargin, 2016. Note how it separates candidate generation from ranking. The industrial architecture is a reference for scale, not a required starting stack for a small customer-offer table.

Recent research to test after the baseline

These are optional comparisons, outside the 40-hour single-case project. Read the methods, how the splits were built and the limitations before reproducing a leaderboard result.

PaperStatus and dateUseful questionBoundary
Athey and Wager — Policy Learning with Observational DataEconometrica, 2021; revised preprint linkedCan a constrained binary decision policy improve held-out value?Requires identification and nuisance estimates; not a multi-arm budget solver
CausalPFNNeurIPS 2025; arXiv revised 27 October 2025Does a pretrained causal prior help a supported binary effect-estimation task?Released implementation is binary; check unconfoundedness, overlap and prior fit
UpliftBenchAugust 2026 preprintDo ranking and cost-based policy selection agree?Dataset-specific evidence; some threshold comparisons reuse evaluation data
UNIQUESeptember 2026 preprintDoes unified generative retrieval/ranking justify its serving complexity?Feed-engagement results do not validate paid-incentive profit

CIPS/CDR — Ranking evaluation from deterministic slates

ICLR 2026 conference paper · March 2026 arXiv version · optional ranking research. See the conference publication. Study click-wise support and slate-independent expected post-click reward before using stochastic clicks to evaluate a different ranking. Estimated click ratios determine the bias. The paper's CDR does not get the usual doubly robust guarantee from an accurate reward model. These assumptions do not identify incentive treatment effects.

Pricing, elasticity and inventory

Fisher, Gallino and Li — Competition-Based Dynamic Pricing in Online Retailing

Management Science, 2018 · online publication 2017. The method and field experiments deal with historical-price endogeneity and randomised price variation. Read it for the design needed to link response estimation to a price decision, instead of copying another retailer's elasticity coefficient.

statsmodels — Generalized linear models

Official docs. Build an inspectable demand or count baseline with a log link, a suitable family and exposure offsets. Log price is a predictor, and the count outcome can include zeros. A fitted price coefficient is causal only when the research design identifies it.

scikit-learn — Poisson regression and non-normal loss

Official worked example. Learn count objectives, exposure weighting and diagnostics. The example is an insurance application, so adapt the opportunity definition and economics for retail demand.

Conlon and Mortimer — Demand Estimation Under Incomplete Product Availability

Research working paper. Shows why availability and substitution matter in demand estimation. Sales recorded during stockouts are not automatically unconstrained demand. Whether you can recover lost demand depends on the data and identification assumptions.

EconML — Double machine learning and instrumental variables

Official method documentation. Optional depth after you write the identification design. DML needs suitable observed confounders and model assumptions. IV needs a relevant instrument, exclusion and its other identifying assumptions. Neither makes up for missing price exposure or an unexplained pricing intervention.

Bojinov, Simchi-Levi and Zhao — Design and Analysis of Switchback Experiments

Research paper. Optional depth for when users share inventory or a market, so randomising individuals creates interference. Switchback periods, carryover and inference need an explicit design. Alternating a price each day is not enough on its own.

Causal Foundation Models with Continuous Treatments — CCPFN

May 2026 preprint, revised July. Optional continuous-treatment challenger. Compare supported price-response estimates with identified classical baselines. A synthetic prior does not remove confounding or justify extrapolating to unobserved prices.

Zhang, Wang and Luo — Optimal Nonparametric Dynamic Pricing with Censored Demand and Adversarial Inventory

September 2026 preprint. Optional inventory-aware online-learning depth. Study the stationary demand assumptions, demand-tail sharing and simulated comparisons. The learner observes inventory but does not jointly optimise replenishment. Its revenue guarantees need care if the reward becomes contribution.

Zheng and Jin — Prediction Sets for Counterfactual Decisions

Preprint submitted 2 July 2026. Optional uncertainty-and-decision depth. Coverage for each action separately does not automatically cover the outcome after a policy selects the action. Study the identified, supported potential-outcome setting and the separate training, policy-learning and calibration splits. The experiments include a held-out analysis of a randomised email dataset, not a new live pricing trial. It is a reason to check uncertainty for the final policy, not a required replacement for the guide's baselines.

Churn timing and evaluation

scikit-survival — Introduction

Official docs. Learn event indicators, follow-up and right censoring before turning a customer history into a survival dataset. Avoid turning tenure into a made-up time-to-churn label.

scikit-survival — Evaluating survival models

Official docs. Use IPCW concordance, time-dependent AUC and Brier/IBS at supported horizons. Censoring assumptions and enough reference follow-up matter. Risk scores and survival probabilities suit different metrics.

Competing risks and survival calibration

Official documentation. Keep mutually exclusive event types and estimate cause-specific cumulative incidence. Censoring another cause and using 1 − Kaplan–Meier answers a different question. Aalen–Johansen gives a nonparametric baseline. For fitted lifelines regression models, inspect out-of-sample censor-aware horizon calibration. That diagnostic does not accept every survival estimator.

scikit-survival — Random survival forests and survival boosting

Official docs. Compare model families after a Cox baseline. Cox-loss boosting can fit nonlinear feature effects while keeping a proportional-hazards structure. Name the loss and assumptions. Calling something a tree does not remove survival assumptions.

SurvPFN — Towards Foundation Models for Survival Predictions

June 2026 paper · accepted to the ICML Foundation Models for Structured Data workshop. Optional small-data censored-survival comparison. The authors report Cox as best and no significant pairwise model differences. Test future-cohort calibration and censoring assumptions. This is not a longitudinal retention-policy experiment.

Policy evaluation and operation

Dudík, Langford and Li — Doubly Robust Policy Evaluation and Learning

Research paper, 2011. Read it for evaluating policies from logged actions and propensities. It motivates IPS and DR estimators. It cannot make unsupported actions evaluable or remove hidden confounding from arbitrary business histories.

Sequential and adaptive policy evaluation

Optional method depth. Jiang and Li (ICML 2016) separate sequential from one-step estimators. Record histories and transitions, and check trajectory support. Hadad et al. (PNAS, 2021; author preprint) address inference under adaptive assignment. Logging action probabilities does not by itself justify ordinary independent-sample intervals.

Lee et al. — When Offline Evaluation Misleads

August 2026 paper · arXiv reports acceptance at a KDD customer-journey workshop. Optional diagnostic depth for delayed rewards. Keep alignment with business action effects apart from how easy a policy is to learn. Individual-level proxy correlation answers a different question. The illustrative replay uses one path, delay is budgeted instead of modelled, and later deployment has a null overall result with exploratory subgroup findings. Check cold start, logging support and mature outcomes. This is not a universal inference recipe or evidence of profit gains for you.

Vowpal Wabbit — Contextual bandits

Official tutorial. Optional, once a fixed policy, action-probability logs, a reward definition and an experiment control work. Delayed business outcomes and adaptive evaluation need more than the tutorial's immediate-reward example.

Lancewicki et al. (ICML, 2021) separate reward-independent from reward-dependent delays. Their theoretical setting helps you check update assumptions, but is not evidence of settled retail-profit gains. The pricing guide starts with scheduled updates on mature cohorts.

Microsoft — Diagnosing sample-ratio mismatch

Practitioner research. Use when logged experiment counts differ from the intended design. Diagnose assignment, execution, filtering and join issues before interpreting the outcome.

MLflow — Model registry workflows

Official docs. Reference for versions, aliases and release management once a local artifact manifest is not enough. Version the feature contract, calibration and action policy as well as the fitted model.

Practice datasets: what each can prove

SourceGood exerciseLimit
Criteo corrected uplift datasetBinary randomised-treatment/uplift methodsUse the corrected release and declare sampling assumptions; no original propensity logs, coupon costs or multi-arm profit evidence
GroupLens MovieLensRetrieval, preference ranking and cold-start evaluationRatings are not randomised incentive assignments
UCI Online RetailTransaction cleaning, demand summaries and inactivity snapshotsPurchase rows omit nonbuyer exposure, full availability and campaign assignments; no causal price/offer effect by default
IBM Telco sample documentationClassification and leakage audit on a fictional-company sampleSnapshot variants cannot establish a longitudinal backtest; exclude supplied churn scores/reasons and hindsight fields
UCI Iranian ChurnSmall customer-table baseline comparisonDo not invent scoring timestamps, event dates or retention treatment labels
Explicit synthetic generatorDecision-time replay, known causal effects, uncertainty and failure drillsValidates the specified simulator, not a real business effect

Keep dataset version, checksum, licence, dictionary and limitations with your experiment. The guides list the extra fields and evidence an operational business project needs.


Back to the chapter  ·  Back to the book

Something unclear or out of date? Suggest a correction ↗

Find your next idea

Loading local search…

Search is built from this edition. No account needed.