​

Quality score and release decisions

Use the local gate for your first release check. A quality score summarizes measured components; it never grants release permission. Failed cases, missing required measurements, provider failures and untrustworthy judges can block independently of the blended number. The CI mode table explains the different evidence used by gate, ci and hosted check.

​

Components

The descriptive score is the rounded weighted sum on a 0–100 scale. Organization policy may change weights. Custom measured metrics may contribute with explicit weights. Always inspect components, provenance and the captured policy before comparing runs. Scores from different versions or policies are not comparable without an explicit recomputation. Missing or empty specification hashes do not establish a match; those baseline comparisons remain inconclusive.

​

Missing means not measured

In v2, an unmeasured component is omitted from breakdown and carries a MISSING_SAFETY, MISSING_JUDGE, MISSING_SCHEMA, MISSING_LATENCY or MISSING_COST flag. It earns zero contribution in the weighted total; its weight is not redistributed. This does not label the underlying behavior a measured failure. A run with only perfect pass/fail data earns 50 points at default weights, with the other five dimensions unmeasured. Safety, judge and schema never borrow pass rate. Absent latency never becomes perfect latency. Functional assertions and the absence of error keywords do not measure safety. Partial safety coverage leaves the run’s safety component unknown. Cost requires both attributable cost and a comparison budget; missing cost or budget is unknown. A measured zero-cost result is valid. With a zero budget, zero cost earns full contribution and positive cost earns zero. Non-finite measurements and invalid case counts cannot become strong evidence. Historical v1 used pass-rate fallbacks and positive missing latency/cost contributions. Those results are historical evidence with known limitations; their number must not certify an untested dimension.

​

Completeness is not evaluator credibility

Sample thresholds are configurable. Ten cases plus a judge is not a safety qualification. Use independent labeled anchors, protected slices and credible-judge requirements when your release depends on model judgments. A weak score does not excuse missing required evidence.

​

Release contract

A release needs its reviewed cases and every configured requirement to pass. A high composite cannot override a failing case. A configured latency, cost, baseline-cost-delta or structured-safety requirement fails closed when its measurement is missing. Provider availability and judge credibility remain separate decisions with distinct failure reasons. Qualification is not permission: approving a baseline, promoting a case and deploying each require their own authority. Cost checks compare evidence already produced. They are not hard spending limits. Result budgets and provider estimates describes the separate source-candidate admission controls and their required deployment boundary. BYOK charges remain separate; arbitrary customer-owned provider code is outside that managed dispatch boundary. Keep the first loop deterministic.

​

Reproducibility and API

Each persisted score retains scoringVersion, exact inputsJson, scoringSpecJson, inputsHash, scoringSpecHash and the source commit when available. New computation uses v2; old rows must not be relabeled.

• GET /api/quality/spec: current descriptive specification and hash.
• GET /api/quality?evaluationId=<id>&action=latest: authenticated latest evidence.
• GET /api/quality?evaluationId=<id>&action=trend: authenticated historical scores.
• POST /api/quality with { "runId": <id> }: explicitly recompute supported run evidence.

These legacy evaluation-score routes are distinct from Planned portable quality at /api/portable-quality/*. A packaged client is not proof that portable-quality APIs or the /quality application are deployed.