Preregistered metrics for AI systems

If you pick the metric after you see the outcome, you are doing storytelling. Preregistered metrics are a contract: what good looks like, frozen before the run that will be judged.

This is standard in serious science. AI product teams skip it because demos reward flexibility. Enterprise buyers punish that flexibility when the board asks for the number that was promised.

Primary outcome metric. Guardrail metrics (latency, cost, safety fails). Eval set identity and version. Decision rule (ship / hold / rollback). Owner of the score.

Publish the contract where the team can see it. Score on a schedule. When the growth analyst, cron job, or CI eval runs, it scores the contract, not a new narrative.

Renovly's admin surface watches live journeys, funnels, and CAC on the same truth as the product. The AI growth analyst every few hours is only honest if the activation and acquisition metrics were named in advance.

Same rule for agent quality: define tool-success and harmful-action rates before you tune prompts to flatter a private notebook.

Attestation answers what ran. Preregistered metrics answer whether what ran counted as success. Together they stop both silent drift and post-hoc victory laps.