Compare the business-as-usual rule, a simple statistical baseline and one challenger. Use your guide's split and keep all tuning and preprocessing out of the final test. When rows depend on each other, report uncertainty at the customer, market or time-block level that matches the decision. Evaluate both the model and the policy built from it.
Save a release manifest like this, with real values instead of placeholders:
release_id: example-2026-10-08-001
code_commit: <commit>
data_snapshot: <immutable-source-version>
feature_query_version: <version>
training_cutoff_utc: <timestamp>
label_contract: <version>
feature_schema: <version>
model_artifact: <path-and-checksum>
calibrator_artifact: <path-or-null>
policy_config: <version>
experiment_config: <version-or-null>
dependency_lock: <lockfile-checksum>
evaluation_report: <path>
approved_actions: <versioned-action-set>
fallback_policy: <version>This manifest is enough for a first project. MLflow's model registry adds lifecycle management when a team needs it. A deployment must load the matching feature schema, model, calibrator and policy together. Changing a threshold or budget counts as a policy release.