The primary key is (customer_id, episode_id, as_of). This is a landmark snapshot: what you
knew about an eligible customer at one decision time. The same customer can have many rows. They
are correlated observations, not many independent customers.
Use two clocks when the source allows it. event_time records when an event happened.
available_at records when the scoring system could first use it. A feature needs both to be at or
before as_of. A payment dated Monday but loaded on Thursday was not available to Tuesday's score.
Historical dimensions must also use the version available then, not today's plan or account status
copied backwards.
Keep these fields alongside the features:
| Field | Purpose |
|---|---|
customer_id, episode_id, as_of | Join, deduplicate and replay the decision; keep identifiers out of the default model |
eligible, eligibility_version | Reconstruct the population the policy could actually contact |
feature_version, feature_max_available_at | Audit feature computation and late arrivals |
horizon_days, label_end | Make the prediction question explicit |
label_observed_through, label_available_at, label_version | Distinguish a complete outcome from an immature or revised one |
churn_label or event_observed, duration | Fixed-horizon classification or survival target |
prior_campaign_id, prior_action, prior_action_time | Describe earlier interventions without importing a future action into features |
Build it in this order. Reconstruct eligible accounts at each scoring time. Join only the historical events and dimensions available then, and compute features. Join future outcomes into a separate label table and mark incomplete labels as unknown. Split by decision time, and fit all learned transformations inside the training folds. Save the query, extraction date, source versions and counts of excluded rows.
For fixed-horizon classification, use a cohort whose labels have all matured. Avoid mixing early known positives with immature, presumed negatives. If the outcome feed is complete through day 100, the latest fully observed 30-day snapshot is day 70. A snapshot on day 85 with no cancellation yet has an unknown label. Completeness means what had arrived when you trained, not the calendar date. Suppose a day-99 cancellation for the day-70 snapshot only arrived on day 103. The feed was then not complete through day 100 for a model retrained on day 100, so that label was unavailable to it. Use the completeness watermark recorded at training time. An ingestion allowance moves the eligible cutoff further back to cover such delays.
Good first features are simple aggregates whose source you can explain.
| Family | Examples | Availability check |
|---|---|---|
| Usage | Active days, sessions, minutes, recency, change between successive windows | Exclude future sessions and distinguish missing telemetry from zero usage |
| Relationship | Tenure, plan, renewal date known then, acquisition channel | Read the historical dimension; do not substitute a later plan |
| Billing | Failed payments already recorded, arrears, price changes already communicated | Do not use final debt or a future chargeback |
| Experience | Complaints, unresolved tickets, service outages, latency | Ticket resolution and survey responses after scoring are future information |
| Purchases | Recency, frequency, net historical margin, returns already known | Apply return and cancellation rules; avoid treating invoice lines as independent orders |
| Trajectory | 7-day versus 30-day activity, rolling variance, trend | Use fixed past windows with stable handling of low exposure |
| Earlier actions | Prior contact, earlier discount, time since campaign | Only earlier actions; keep the forthcoming treatment out of the risk feature table |
Run feature ablations and inspect suspiciously strong predictors. Drop post-churn status, churn reason, a supplied churn score, future redemption and hindsight lifetime value. A supplied "customer value" field needs an as-of definition before use. Avoid giving plan names an arbitrary order. CatBoost supports categorical inputs and advises against one-hot encoding them yourself. Its encoding still cannot repair a wrong time split or a column derived from the future. CatBoost categorical-feature documentation.
Earlier campaigns also change what the target means. A classifier trained on customers who received a mix of past campaigns estimates risk under that historical mix, not untreated risk. Keep the campaign records and report this limit. When the decision needs a no-campaign comparator, use a randomised control group.