← Back to the full chapter
Lesson

When assignment was not random

What changes because of an action, and what the data can show.

2 min readApplied extensionUpdated
Read source ↗

A confounder affects both who gets the action and the outcome. For example, an account manager may target unhappy customers, who were already more likely to leave. A treated-versus-untreated comparison then mixes that selection with the offer's effect.

A mediator is changed by the action and sits on a path to the outcome, such as post-offer engagement. Adjusting for it usually changes the causal question and can bias a total-effect analysis. A collider is a shared result of other variables, and conditioning on it can create a false association. A diagram helps state these choices, but cannot prove the assumptions.

Observational adjustment usually needs a believable case that measured pre-action variables block the relevant confounding. It also needs overlap: each compared action has a nonzero chance in the contexts where the policy would choose it. If an action was never tried in a segment, a flexible model can extrapolate a number without identifying that effect. Restrict the policy or collect data.

The propensity is the probability that the assignment policy selects an action, given recorded pre-action context. It differs from the predicted chance of a purchase. Estimated propensities and outcome models are often called nuisance models, because the main target is the effect or policy value, not their predictions.

Cross-fitting trains a nuisance model on other folds before scoring a fold. It can reduce some overfitting, but cannot repair unmeasured confounding, unsupported actions or a wrong time or group split. “Doubly robust” is a property under stated assumptions, so the design still matters. The linked guides give each estimator's requirements.

Something unclear or out of date? Suggest a correction ↗

Find your next idea

Loading local search…

Search is built from this edition. No account needed.