Build a retention decision service with an evidence report. You create these deliverables in your own project. This roadmap does not supply them:
decision-contract.md: population, episode rules, event, horizon, label maturity, capacity, comparator, costs and action owner.- A versioned snapshot query and data dictionary, with a saved example showing that a late event cannot change features available to an earlier decision.
- A reproducible training and chronological backtest command, source hashes, dependency versions and a table comparing rules, logistic regression and the chosen challenger.
- A calibration report and capacity curve with future-cohort results, segment counts and dependence-aware uncertainty. Use survival diagnostics too if that is your target formulation.
- A batch CRM export or validated API, idempotent decision writes, eligibility rechecks and action logs linked to decision IDs.
- A randomised pilot plan, and an uplift and margin report only when valid intervention data exist. Label a simulated campaign as a simulation and a historical association as an association.
- Monitoring and rollback instructions, plus checks for feature availability, label maturity, duplicate decisions, exhausted budgets and contact cooldowns.
| Gate | Required evidence | Honest completion claim |
|---|---|---|
| Offline prototype | Reproducible snapshots, mature horizon labels or valid event/censoring records, future-cohort comparison, appropriate calibration and capacity analysis | "The model prioritises observed future risk under the documented data and policy" |
| Operational readiness | Successful shadow run, execution-time eligibility, idempotency, logging, fallback, ownership and rollback | "The workflow can deliver and audit decisions" |
| Retention effect | A completed controlled trial with matured outcomes, uncertainty, incremental net contribution and guardrails | "Estimated the tested policy’s effect in this population and period"; state its direction and uncertainty |
| Sustained operation | Repeated outcome review, control comparison, model and policy governance, documented maintenance | "The service has evidence and an owner for continuing use" |
A learner can finish the first gate and show the second with a local integration. The last two need more than a public benchmark. If the data do not support a gate, name the missing evidence and give a concrete plan to collect it.