8. Connect the decision to a working retention workflow

Most churn decisions suit batch scoring. Start with a daily or weekly job that builds point-in-time features, scores eligible accounts and writes a versioned decision table. Add an API only when a live workflow needs fresher information. An endpoint does not make the model more actionable.

%%{init: {"theme":"base","themeVariables":{"fontFamily":"Arial, sans-serif","fontSize":"16px","textColor":"#17324D","primaryColor":"#F5F7FA","primaryTextColor":"#17324D","primaryBorderColor":"#8796A8","lineColor":"#718096","secondaryColor":"#EFF6FF","tertiaryColor":"#E9F6F2","clusterBkg":"#F8FAFC","clusterBorder":"#D6DEE8","edgeLabelBackground":"#FFFFFF"},"flowchart":{"htmlLabels":true,"nodeSpacing":18,"rankSpacing":28,"curve":"linear","padding":10,"subGraphTitleMargin":{"top":8,"bottom":12}}}}%%
flowchart TB
    accTitle: Churn evidence and retention workflow
    accDescr: Historical snapshots predict risk, randomised pilots estimate action effects, and a constrained policy delivers actions for mature-outcome review.
    subgraph Risk["Risk prediction"]
        direction TB
        D["Eligible<br/>snapshots"] --> M["Risk or<br/>survival"]
    end
    subgraph Effects["Action evidence"]
        direction TB
        T["Randomised<br/>pilot"] --> A["Action<br/>effects"]
    end
    subgraph Delivery["Retention workflow"]
        direction TB
        P["Value +<br/>constraints"] --> C["CRM queue<br/>or API"]
        C --> O["Mature outcome<br/>review"]
    end
    Risk --> Delivery
    Effects --> Delivery

    classDef data fill:#EFF6FF,stroke:#3973AC,color:#17324D,stroke-width:1.5px;
    classDef model fill:#EFF2FD,stroke:#596FC2,color:#17324D,stroke-width:1.5px;
    classDef decision fill:#FFF5DF,stroke:#A8751A,color:#17324D,stroke-width:1.5px;
    classDef execution fill:#E9F6F2,stroke:#23866E,color:#17324D,stroke-width:1.5px;
    classDef neutral fill:#F5F7FA,stroke:#8796A8,color:#17324D,stroke-width:1.5px;
    class D,T data;
    class M,A model;
    class P decision;
    class C execution;
    class O neutral;

Retention workflow. Risk scores and intervention evidence serve different purposes. The pilot includes a randomised no-contact comparator. The policy compares incremental value and feasible actions before delivery. Assignment, delivery and cost logs join to matured outcomes for governed model and policy review, including the option of no incremental contact.

Export decision_id, account, as_of, horizon, expiry, calibrated risk, model and feature versions, assigned action, policy version and a workflow reason. If an effect estimate exists, store its sign convention, action and expected incremental value too. Join the eventual outcome back to the original decision. Make the write idempotent so a retry cannot create duplicate contacts or spend the same budget twice.

Include the experiment or assignment key, actual assignment probability and relevant coordinator state from the shared decision record. Keep the assigned action after delivery overrides, and log execution separately.

Recheck eligibility at execution. The account may have cancelled since scoring, lost contact eligibility or already received an offer. Enforce suppression lists, cooldowns, budget, agent capacity and mutually exclusive campaigns. Log the action actually delivered separately from the proposed action. Label explanation text as a model explanation, so agents do not read it as a verified cause of churn. If an LLM drafts message copy, keep it downstream of the validated decision and evaluate it as its own feature.

For stale or missing required data, use a documented fallback such as the existing service workflow or no incremental campaign. Route payment issues to payment recovery instead of automatically subsidising every high-risk account. Keep a manual escalation path for account questions the score cannot resolve.

Monitor at three speeds:

Document data and calibration triggers for retraining, the available-label cutoff, model owner, rollback artefact and campaign stop owner. Shadow a challenger before it allocates contacts. Review policy changes as well as model changes, because a new discount or altered renewal terms changes the outcome distribution. Keep a randomised control group when measuring continuing impact. Without it, the model can look worse simply because successful interventions reduced churn among its high-risk customers. A drifting input distribution is a reason to investigate, not proof that a replacement model is better.