7. Risk is not the expected effect of an offer

This is the most important distinction in the use case. Two field experiments in Eva Ascarza's Retention Futility found that the customers at highest risk were not necessarily the best targets for retention. It supports testing who responds instead of treating risk ranking as a retention policy. It does not promise the same treatment effects in your company. Ascarza, 2018.

For a churn outcome Y = 1, define the action effect with a helpful sign:

tau_a(x) = P(churn by H | do(no incremental contact), x)
           - P(churn by H | do(action a), x)

Positive tau_a means the action reduces churn. It is a conditional average treatment effect among customers with similar observed information, not proof that one named person will be saved. Historical response rates among contacted customers cannot identify it, because the earlier team may have chosen whom to contact using risk, value or information missing from the dataset.

Design a randomised pilot among the eligible population. Include no incremental retention contact, the current policy and the candidate actions you can actually deliver. Randomise by customer or account, or by household or another cluster when actions spill over. Avoid giving the same customer conflicting treatments in simultaneous campaigns. Record assignment probabilities, assignment, delivery, cost, redemption and matured outcomes separately. Evaluate the main trial by assigned treatment, including failed deliveries, using intention to treat. Put the churn horizon, follow-up margin, minimum worthwhile effect, stopping rules and sample size assumptions in the plan before reading results. Measure complaints, unsubscribes, discount cost and longer-term retention as well as the first renewal.

With randomised assignment as treatment, outcome learners estimate effects of assignment with that delivery process, including nondelivery, not effects among recipients only. A change in channel, delivery success, overrides or capacity can change those effects. Evaluate the revised workflow instead of carrying old numbers over. Follow the shared experiment protocol for missing outcomes and dependent assignment.

My suggested modelling sequence once a valid pilot exists:

1. Estimate average action effects with uncertainty and compare them with no contact. A null average can still hide useful heterogeneity, but a subgroup or targeting-policy claim needs its own prespecified or independent evaluation. If neither level shows credible benefit, investigate the intervention and delivery before scaling personalised targeting.
2. Fit a simple T-learner: one regularised logistic or boosted-tree outcome model per arm. Subtract the predicted churn probabilities in the sign convention above. Compare it with an S-learner and segment-level effects, since a tiny treatment arm cannot support elaborate heterogeneity. The T/S/X learner framework offers different bias and variance tradeoffs, not a guaranteed winner. Künzel et al., 2019.
3. When sample size and overlap support it, compare a cross-fitted doubly robust learner or a suitable causal forest. Use the known randomised propensities where supported, keep nuisance fitting and policy evaluation separate, and check effect stability. Doubly robust methods still need identification and overlap. They cannot fix unrecorded confounding in a historical campaign. EconML doubly robust learning.
4. On an independent randomised evaluation cohort, compare the frozen targeting policy with the current policy, equal-capacity risk targeting and no incremental contact. Report uplift or Qini curves with their sign and normalisation. The decision metric is incremental contribution at the actual budget, with a confidence interval. For unequal assignment probabilities, use a suitable propensity-weighted or doubly robust policy-value estimator, and inspect overlap and weight variance. Repeat in a prospective controlled rollout.

A margin example, using hypothetical known effects. Assume avoiding a churn event creates ₹1,500 of incremental contribution over the chosen follow-up, and an assigned action costs ₹100 in total, whatever the response. Check this simplifying value assumption too.

At a one-contact capacity, B gives the higher expected value despite its lower churn risk. Include discounts given to customers who would have stayed anyway. If the discount is paid only on redemption, use expected redemption cost instead of a fixed assigned cost. If an offer only delays churn, a short-horizon rescue probability times a long-horizon value can mislead. Evaluate realised incremental contribution over a suitable follow-up directly.

Try it yourself. In the margin example, how large must segment A's absolute churn reduction be for the ₹100 action to break even? If the action cost falls to ₹50, which segments have positive expected value, and which comes first at a one-contact capacity?

For value forecasts, use revenue minus fulfilment, service, incentive and relevant contact costs, with a stated horizon and discounting assumption. Build those forecasts from information available at as_of, and handle incomplete future observation with care. A total-spend or CLTV column computed after departure is hindsight. Current revenue times a risk score is not incremental lifetime value. Keep causal benefit estimation separate from model explanations. SHAP can describe a classifier's score, but cannot show that fixing a feature will keep the customer.