> ## Documentation Index > Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Create quality quality > Requires scopes: runs:write ## OpenAPI ````yaml /api-reference/openapi.json post /api/quality openapi: 3.1.2 info: title: EvalGate API version: 18.13.0 description: EvalGate API. See docs/api-contract.md for stability commitments. servers: - url: https://evalgate.com description: Production - url: http://localhost:3000 description: Local security: [] paths: /api/quality: post: tags: - quality summary: Create quality quality description: 'Requires scopes: runs:write' operationId: post_quality requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostApiQualityRequest' responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/PostApiQualityResponse200ApplicationJson' '400': $ref: '#/components/responses/ValidationError' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/Conflict' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' '503': $ref: '#/components/responses/ServiceUnavailable' security: - bearerAuth: [] components: schemas: PostApiQualityRequest: type: object properties: runId: type: number baseline: oneOf: - type: string enum: - production - type: string enum: - previous - type: string enum: - published - type: string enum: - auto minScore: type: number maxDrop: type: number warnDrop: type: number gateCheck: oneOf: - type: boolean enum: - false - type: boolean enum: - true persistCheckReport: oneOf: - type: boolean enum: - false - type: boolean enum: - true measurementGate: type: object properties: mode: oneOf: - type: string enum: - block - type: string enum: - warn - type: string enum: - observe minimumSampleSize: type: number allowedMeasurementStatuses: type: array items: type: string requireConclusiveInterval: oneOf: - type: boolean enum: - false - type: boolean enum: - true maximumEce: type: number minimumAgreement: type: number maximumPositionFlipRate: type: number requireCoverage: oneOf: - type: boolean enum: - false - type: boolean enum: - true allowedPredictiveValidityStatuses: type: array items: type: string maximumPredictiveValidityAgeDays: type: number additionalProperties: false required: - mode redTeamGate: type: object properties: campaignRunId: type: string expectedTarget: type: object properties: artifactId: type: number versionId: type: number contentHash: type: string additionalProperties: false required: - artifactId - versionId - contentHash expectedSnapshotHash: type: string policyProfile: oneOf: - type: string enum: - custom - type: string enum: - strict - type: string enum: - pre_release_strict - type: string enum: - post_remediation_strict maxEvidenceAgeMs: type: number maxCriticalOpen: type: number maxHighOpen: type: number maxMediumOpen: type: number requireAttackResistance: oneOf: - type: boolean enum: - false - type: boolean enum: - true maxAttackSuccessRate: type: number requireBenignUtility: oneOf: - type: boolean enum: - false - type: boolean enum: - true minBenignUtilityPassRate: type: number requireCompleteCampaign: oneOf: - type: boolean enum: - false - type: boolean enum: - true requireCostAttribution: oneOf: - type: boolean enum: - false - type: boolean enum: - true requireRemediationRerun: oneOf: - type: boolean enum: - false - type: boolean enum: - true requireNoNewFingerprints: oneOf: - type: boolean enum: - false - type: boolean enum: - true requireActiveGoldenCoverage: oneOf: - type: boolean enum: - false - type: boolean enum: - true requireSignedReport: oneOf: - type: boolean enum: - false - type: boolean enum: - true requireAttestation: oneOf: - type: boolean enum: - false - type: boolean enum: - true allowDeferred: oneOf: - type: boolean enum: - false - type: boolean enum: - true additionalProperties: false required: - campaignRunId - expectedTarget additionalProperties: false required: - runId PostApiQualityResponse200ApplicationJson: type: object properties: success: type: boolean enum: - true runId: type: number integrity: oneOf: - type: 'null' - type: object properties: summary: type: object properties: byBehavior: type: object properties: classification: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings extraction: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings multi_step_reasoning: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings retrieval: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings tool_use: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings additionalProperties: false required: - classification - extraction - multi_step_reasoning - retrieval - tool_use byDifficulty: type: object properties: easy: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings flaky: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings hard: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings medium: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings additionalProperties: false required: - easy - flaky - hard - medium trajectory: type: object properties: efficiency: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings toolUsage: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings additionalProperties: false required: - efficiency - toolUsage workflow: type: object properties: axes: type: object properties: autonomyFit: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings escalationQuality: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings approvalSafety: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings hookCompliance: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings memoryCorrectness: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings recoveryQuality: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings handoffQuality: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings debuggability: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings additionalProperties: false required: - autonomyFit - escalationQuality - approvalSafety - hookCompliance - memoryCorrectness - recoveryQuality - handoffQuality - debuggability slices: type: object properties: byAutonomy: type: object properties: autopilot: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings bypass_approvals: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings default_approvals: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings manual: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings additionalProperties: false required: - autopilot - bypass_approvals - default_approvals - manual byWorkflowClass: type: object properties: approval_gated_mutation: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings background_execution: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings handoff_heavy: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings memory_sensitive: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings parallel_tool_use: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings policy_conflict: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings recovery_required: type: object properties: mean: type: number stdDev: type: number 'n': type: number confidenceInterval: oneOf: - type: 'null' - type: object properties: lower: type: number upper: type: number method: oneOf: - type: string enum: - normal - type: string enum: - wilson additionalProperties: false required: - lower - upper - method composition: type: object properties: percentageOfDataset: type: number additionalProperties: false required: - percentageOfDataset status: oneOf: - type: string enum: - invalid - type: string enum: - unstable - type: string enum: - valid warnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - mean - stdDev - 'n' - confidenceInterval - composition - status - warnings additionalProperties: false required: - approval_gated_mutation - background_execution - handoff_heavy - memory_sensitive - parallel_tool_use - policy_conflict - recovery_required additionalProperties: false required: - byAutonomy - byWorkflowClass additionalProperties: false required: - axes - slices metadata: type: object properties: totalCases: type: number evaluatedAt: type: string evaluationRunId: oneOf: - type: 'null' - type: number uncategorizedBehaviorCount: type: number uncategorizedDifficultyCount: type: number additionalProperties: false required: - totalCases - evaluatedAt - evaluationRunId - uncategorizedBehaviorCount - uncategorizedDifficultyCount additionalProperties: false required: - byBehavior - byDifficulty - metadata datasetIntegrity: type: object properties: behaviorDistribution: type: object properties: classification: type: number extraction: type: number multi_step_reasoning: type: number retrieval: type: number tool_use: type: number additionalProperties: false required: - classification - extraction - multi_step_reasoning - retrieval - tool_use difficultyDistribution: type: object properties: easy: type: number flaky: type: number hard: type: number medium: type: number additionalProperties: false required: - easy - flaky - hard - medium failureOriginShare: type: number syntheticOnly: type: boolean coverageWarnings: type: array items: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed additionalProperties: false required: - behaviorDistribution - difficultyDistribution - failureOriginShare - syntheticOnly - coverageWarnings metricValidity: type: object properties: historicalSnapshotsChecked: type: number driftSignals: type: array items: type: object properties: metric: type: string currentMean: type: number previousMean: type: number delta: type: number triggered: type: boolean additionalProperties: false required: - metric - currentMean - previousMean - delta - triggered improvementWithoutFailureReduction: type: boolean additionalProperties: false required: - historicalSnapshotsChecked - driftSignals - improvementWithoutFailureReduction antiGaming: type: object properties: improvedEasyBucketOnly: type: boolean failureDistributionStagnant: type: boolean redundancyIncreaseWithScoreGain: type: boolean detected: type: boolean additionalProperties: false required: - improvedEasyBucketOnly - failureDistributionStagnant - redundancyIncreaseWithScoreGain - detected coverageIntegrity: type: object properties: coverageByBehavior: type: object properties: classification: type: number extraction: type: number multi_step_reasoning: type: number retrieval: type: number tool_use: type: number additionalProperties: false required: - classification - extraction - multi_step_reasoning - retrieval - tool_use dominantFailureMode: type: object properties: mode: oneOf: - type: 'null' - type: string share: type: number additionalProperties: false required: - mode - share lowFailureDiversity: type: boolean additionalProperties: false required: - coverageByBehavior - dominantFailureMode - lowFailureDiversity lifecycle: type: object properties: counts: type: object properties: active: type: number active_gate: type: number archived: type: number candidate: type: number deprecated: type: number superseded: type: number validated: type: number additionalProperties: false required: - active - active_gate - archived - candidate - deprecated - superseded - validated recommendedTransitions: type: array items: type: object properties: testCaseId: type: number currentState: oneOf: - type: string enum: - active - type: string enum: - active_gate - type: string enum: - archived - type: string enum: - candidate - type: string enum: - deprecated - type: string enum: - superseded - type: string enum: - validated nextState: oneOf: - type: string enum: - active - type: string enum: - active_gate - type: string enum: - archived - type: string enum: - candidate - type: string enum: - deprecated - type: string enum: - superseded - type: string enum: - validated reason: oneOf: - type: string enum: - activated_for_ci - type: string enum: - consistent_failures - type: string enum: - low_signal - type: string enum: - manual_approval - type: string enum: - no_transition - type: string enum: - reactivated_by_failure - type: string enum: - reactivated_by_manual_review - type: string enum: - unused_for_cycles additionalProperties: false required: - testCaseId - currentState - nextState - reason additionalProperties: false required: - counts - recommendedTransitions warnings: type: array items: type: object properties: code: oneOf: - type: string enum: - coverage_cluster_dominance - type: string enum: - coverage_gap_detected - type: string enum: - dataset_behavior_imbalance - type: string enum: - dataset_behavior_unstable - type: string enum: - dataset_difficulty_imbalance - type: string enum: - dataset_failure_origin_low - type: string enum: - dataset_synthetic_only - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected - type: string enum: - metric_high_variance - type: string enum: - metric_inconsistency_detected - type: string enum: - metric_invalid_sample - type: string enum: - metric_low_confidence - type: string enum: - metric_missing_data - type: string enum: - shadow_harness_contract_mismatch - type: string enum: - shadow_harness_validation_failed severity: oneOf: - type: string enum: - critical - type: string enum: - info - type: string enum: - warning message: type: string context: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - code - severity - message events: type: array items: type: object properties: type: oneOf: - type: string enum: - dataset_integrity_warning - type: string enum: - eval_case_archived - type: string enum: - eval_case_deprecated - type: string enum: - eval_case_promoted - type: string enum: - eval_case_reactivated - type: string enum: - metric_drift_detected - type: string enum: - metric_gaming_detected message: type: string context: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - type - message metadata: type: object properties: totalWarnings: type: number refreshedAt: type: string sourceTrajectoryMetricsAvailable: type: boolean overallPassRate: type: number averageTrajectoryRedundancy: oneOf: - type: 'null' - type: number additionalProperties: false required: - totalWarnings - refreshedAt - sourceTrajectoryMetricsAvailable - overallPassRate - averageTrajectoryRedundancy additionalProperties: false required: - summary - datasetIntegrity - metricValidity - antiGaming - coverageIntegrity - lifecycle - warnings - events - metadata workflowCoverage: oneOf: - type: 'null' - type: object properties: totalCases: type: number agentWorkflowCases: type: number coverageRate: type: number workflowClassDistribution: type: object properties: approval_gated_mutation: type: number background_execution: type: number handoff_heavy: type: number memory_sensitive: type: number parallel_tool_use: type: number policy_conflict: type: number recovery_required: type: number additionalProperties: false required: - approval_gated_mutation - background_execution - handoff_heavy - memory_sensitive - parallel_tool_use - policy_conflict - recovery_required autonomyDistribution: type: object properties: autopilot: type: number bypass_approvals: type: number default_approvals: type: number manual: type: number additionalProperties: false required: - autopilot - bypass_approvals - default_approvals - manual generatedCandidateCount: type: number experimentRecipeCount: type: number gaps: type: array items: type: object properties: id: type: string workflowClass: oneOf: - type: string enum: - approval_gated_mutation - type: string enum: - background_execution - type: string enum: - handoff_heavy - type: string enum: - memory_sensitive - type: string enum: - parallel_tool_use - type: string enum: - policy_conflict - type: string enum: - recovery_required severity: oneOf: - type: string enum: - critical - type: string enum: - info - type: string enum: - warning description: type: string recommendation: type: string additionalProperties: false required: - id - workflowClass - severity - description - recommendation additionalProperties: false required: - totalCases - agentWorkflowCases - coverageRate - workflowClassDistribution - autonomyDistribution - generatedCandidateCount - experimentRecipeCount - gaps gateResult: oneOf: - type: 'null' - type: object properties: passed: type: boolean severity: oneOf: - type: string enum: - fail - type: string enum: - pass - type: string enum: - skipped - type: string enum: - warn reasonCode: type: string reasonMessage: oneOf: - type: 'null' - type: string gateSkipped: oneOf: - type: boolean enum: - false - type: boolean enum: - true providerAdmission: type: object properties: policyVersion: type: string enum: - evalgate.provider-availability.v1 qualityDetermined: type: boolean providerState: oneOf: - type: string enum: - authentication_failed - type: string enum: - cache_reused - type: string enum: - cancelled - type: string enum: - deferred_by_policy - type: string enum: - executed - type: string enum: - malformed_response - type: string enum: - model_retired - type: string enum: - not_required - type: string enum: - policy_blocked - type: string enum: - provider_unavailable - type: string enum: - rate_limited - type: string enum: - skipped_by_policy - type: string enum: - timed_out - type: string enum: - unknown releaseReadiness: oneOf: - type: string enum: - blocked - type: string enum: - inconclusive - type: string enum: - ready passed: type: boolean code: oneOf: - type: string enum: - PROVIDER_CACHE_STALE - type: string enum: - PROVIDER_DEFERRED - type: string enum: - PROVIDER_EVIDENCE_MISSING - type: string enum: - PROVIDER_NOT_REQUIRED - type: string enum: - PROVIDER_READY - type: string enum: - PROVIDER_UNAVAILABLE - type: string enum: - STALE_CALIBRATION message: type: string additionalProperties: false required: - policyVersion - qualityDetermined - providerState - releaseReadiness - passed - code - message redTeamAdmission: type: object properties: policyVersion: type: string enum: - evalgate.red-team-admission.v1 required: type: boolean passed: type: boolean qualityDetermined: type: boolean releaseReadiness: oneOf: - type: string enum: - blocked - type: string enum: - inconclusive - type: string enum: - ready code: oneOf: - type: string enum: - RED_TEAM_ACCEPTED_RISK_UNGOVERNED - type: string enum: - RED_TEAM_ATTACK_THRESHOLD_FAILED - type: string enum: - RED_TEAM_ATTESTATION_INVALID - type: string enum: - RED_TEAM_BENIGN_UTILITY_FAILED - type: string enum: - RED_TEAM_BENIGN_UTILITY_MISSING - type: string enum: - RED_TEAM_CAMPAIGN_INCOMPLETE - type: string enum: - RED_TEAM_CONTROL_FAILED - type: string enum: - RED_TEAM_COST_UNATTRIBUTED - type: string enum: - RED_TEAM_COVERAGE_MISSING - type: string enum: - RED_TEAM_CRITICAL_FINDING - type: string enum: - RED_TEAM_EVIDENCE_MALFORMED - type: string enum: - RED_TEAM_EVIDENCE_MISSING - type: string enum: - RED_TEAM_EVIDENCE_STALE - type: string enum: - RED_TEAM_HIGH_FINDING - type: string enum: - RED_TEAM_INCONCLUSIVE - type: string enum: - RED_TEAM_MEDIUM_FINDING - type: string enum: - RED_TEAM_NEW_FINDINGS - type: string enum: - RED_TEAM_NOT_REQUIRED - type: string enum: - RED_TEAM_ORGANIZATION_MISMATCH - type: string enum: - RED_TEAM_PROVIDER_UNAVAILABLE - type: string enum: - RED_TEAM_READY - type: string enum: - RED_TEAM_REMEDIATION_INCOMPLETE - type: string enum: - RED_TEAM_REPORT_MISSING - type: string enum: - RED_TEAM_TARGET_MISMATCH message: type: string evidenceHash: type: string stalenessReasons: type: array items: oneOf: - type: string enum: - campaign_snapshot_changed - type: string enum: - completion_time_in_future - type: string enum: - completion_time_missing - type: string enum: - evidence_age_exceeded - type: string enum: - target_content_changed additionalProperties: false required: - policyVersion - required - passed - qualityDetermined - releaseReadiness - code - message redTeamEvidence: type: object properties: target: type: object properties: artifactId: type: string versionId: type: string contentHash: type: string additionalProperties: false required: - artifactId - versionId - contentHash campaign: type: object properties: campaignId: type: string campaignVersionId: type: string snapshotHash: type: string runId: type: string completedAt: type: string additionalProperties: false required: - campaignId - campaignVersionId - snapshotHash - runId evidenceIdentity: type: object properties: evidenceHash: type: string policyVersion: type: string generatedAt: type: string additionalProperties: false required: - evidenceHash - policyVersion - generatedAt additionalProperties: false required: - target - campaign - evidenceIdentity releaseReadiness: oneOf: - type: string enum: - blocked - type: string enum: - inconclusive - type: string enum: - ready additionalProperties: false required: - passed - severity - reasonCode - reasonMessage checkReport: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number evaluationId: type: string runId: type: number verdict: oneOf: - type: string enum: - fail - type: string enum: - pass - type: string enum: - warn gateApplied: type: boolean gateMode: oneOf: - type: string enum: - enforced - type: string enum: - neutral reasonCode: type: string reasonMessage: oneOf: - type: 'null' - type: string actionableMessage: type: string score: type: number baselineScore: oneOf: - type: 'null' - type: number delta: oneOf: - type: 'null' - type: number baselineMissing: oneOf: - type: boolean enum: - false - type: boolean enum: - true baselineRunId: oneOf: - type: 'null' - type: number flags: type: array items: type: string thresholds: type: object properties: minScore: type: number maxDrop: type: number warnDrop: type: number baseline: oneOf: - type: string enum: - auto - type: string enum: - previous - type: string enum: - production - type: string enum: - published providerRequiredNow: oneOf: - type: boolean enum: - false - type: boolean enum: - true freshCacheAllowed: oneOf: - type: boolean enum: - false - type: boolean enum: - true deferredAllowed: oneOf: - type: boolean enum: - false - type: boolean enum: - true requireFreshCalibration: oneOf: - type: boolean enum: - false - type: boolean enum: - true additionalProperties: false required: - minScore - baseline providerAvailability: type: object properties: state: oneOf: - type: string enum: - authentication_failed - type: string enum: - cache_reused - type: string enum: - cancelled - type: string enum: - deferred_by_policy - type: string enum: - executed - type: string enum: - malformed_response - type: string enum: - model_retired - type: string enum: - not_required - type: string enum: - policy_blocked - type: string enum: - provider_unavailable - type: string enum: - rate_limited - type: string enum: - skipped_by_policy - type: string enum: - timed_out - type: string enum: - unknown freshCache: oneOf: - type: boolean enum: - false - type: boolean enum: - true calibrationState: oneOf: - type: string enum: - fresh - type: string enum: - not_comparable - type: string enum: - stale provider: type: string model: type: string runId: type: number attempted: type: number succeeded: type: number failed: type: number source: oneOf: - type: string enum: - cache - type: string enum: - evaluation_run - type: string enum: - model_gateway_ledger - type: string enum: - unknown additionalProperties: false required: - state providerAdmission: type: object properties: policyVersion: type: string enum: - evalgate.provider-availability.v1 qualityDetermined: type: boolean providerState: oneOf: - type: string enum: - authentication_failed - type: string enum: - cache_reused - type: string enum: - cancelled - type: string enum: - deferred_by_policy - type: string enum: - executed - type: string enum: - malformed_response - type: string enum: - model_retired - type: string enum: - not_required - type: string enum: - policy_blocked - type: string enum: - provider_unavailable - type: string enum: - rate_limited - type: string enum: - skipped_by_policy - type: string enum: - timed_out - type: string enum: - unknown releaseReadiness: oneOf: - type: string enum: - blocked - type: string enum: - inconclusive - type: string enum: - ready passed: type: boolean code: oneOf: - type: string enum: - PROVIDER_CACHE_STALE - type: string enum: - PROVIDER_DEFERRED - type: string enum: - PROVIDER_EVIDENCE_MISSING - type: string enum: - PROVIDER_NOT_REQUIRED - type: string enum: - PROVIDER_READY - type: string enum: - PROVIDER_UNAVAILABLE - type: string enum: - STALE_CALIBRATION message: type: string additionalProperties: false required: - policyVersion - qualityDetermined - providerState - releaseReadiness - passed - code - message redTeamAdmission: type: object properties: policyVersion: type: string enum: - evalgate.red-team-admission.v1 required: type: boolean passed: type: boolean qualityDetermined: type: boolean releaseReadiness: oneOf: - type: string enum: - blocked - type: string enum: - inconclusive - type: string enum: - ready code: oneOf: - type: string enum: - RED_TEAM_ACCEPTED_RISK_UNGOVERNED - type: string enum: - RED_TEAM_ATTACK_THRESHOLD_FAILED - type: string enum: - RED_TEAM_ATTESTATION_INVALID - type: string enum: - RED_TEAM_BENIGN_UTILITY_FAILED - type: string enum: - RED_TEAM_BENIGN_UTILITY_MISSING - type: string enum: - RED_TEAM_CAMPAIGN_INCOMPLETE - type: string enum: - RED_TEAM_CONTROL_FAILED - type: string enum: - RED_TEAM_COST_UNATTRIBUTED - type: string enum: - RED_TEAM_COVERAGE_MISSING - type: string enum: - RED_TEAM_CRITICAL_FINDING - type: string enum: - RED_TEAM_EVIDENCE_MALFORMED - type: string enum: - RED_TEAM_EVIDENCE_MISSING - type: string enum: - RED_TEAM_EVIDENCE_STALE - type: string enum: - RED_TEAM_HIGH_FINDING - type: string enum: - RED_TEAM_INCONCLUSIVE - type: string enum: - RED_TEAM_MEDIUM_FINDING - type: string enum: - RED_TEAM_NEW_FINDINGS - type: string enum: - RED_TEAM_NOT_REQUIRED - type: string enum: - RED_TEAM_ORGANIZATION_MISMATCH - type: string enum: - RED_TEAM_PROVIDER_UNAVAILABLE - type: string enum: - RED_TEAM_READY - type: string enum: - RED_TEAM_REMEDIATION_INCOMPLETE - type: string enum: - RED_TEAM_REPORT_MISSING - type: string enum: - RED_TEAM_TARGET_MISMATCH message: type: string evidenceHash: type: string stalenessReasons: type: array items: oneOf: - type: string enum: - campaign_snapshot_changed - type: string enum: - completion_time_in_future - type: string enum: - completion_time_missing - type: string enum: - evidence_age_exceeded - type: string enum: - target_content_changed additionalProperties: false required: - policyVersion - required - passed - qualityDetermined - releaseReadiness - code - message redTeamEvidence: type: object properties: target: type: object properties: artifactId: type: string versionId: type: string contentHash: type: string additionalProperties: false required: - artifactId - versionId - contentHash campaign: type: object properties: campaignId: type: string campaignVersionId: type: string snapshotHash: type: string runId: type: string completedAt: type: string additionalProperties: false required: - campaignId - campaignVersionId - snapshotHash - runId evidenceIdentity: type: object properties: evidenceHash: type: string policyVersion: type: string generatedAt: type: string additionalProperties: false required: - evidenceHash - policyVersion - generatedAt additionalProperties: false required: - target - campaign - evidenceIdentity releaseReadiness: oneOf: - type: string enum: - blocked - type: string enum: - inconclusive - type: string enum: - ready 'n': oneOf: - type: 'null' - type: number evidenceLevel: oneOf: - type: 'null' - type: string dashboardUrl: type: string checkedAt: type: string source: type: string enum: - app-gate additionalProperties: false required: - schemaVersion - evaluationId - runId - verdict - gateApplied - gateMode - reasonCode - checkedAt - source qualityScore: type: object properties: score: type: number breakdown: type: object properties: passRate: type: number safety: type: number judge: type: number schema: type: number latency: type: number cost: type: number additionalProperties: false required: - passRate flags: type: array items: type: string evidenceLevel: oneOf: - type: string enum: - medium - type: string enum: - strong - type: string enum: - weak scoringVersion: type: string enum: - v2 scoringSpecHash: type: string additionalProperties: false required: - score - breakdown - flags - evidenceLevel - scoringVersion - scoringSpecHash additionalProperties: false required: - success - runId - integrity - workflowCoverage - gateResult - checkReport - qualityScore JsonValue: oneOf: - type: 'null' - type: boolean - type: number - type: string - type: array items: $ref: '#/components/schemas/JsonValue' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' ApiError: type: object properties: error: type: object properties: code: type: string enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - INVALID_REFERENCE - VALIDATION_ERROR - RATE_LIMITED - CONFLICT - INTERNAL_ERROR - SERVICE_UNAVAILABLE - QUOTA_EXCEEDED - FEATURE_LIMIT_REACHED - NO_ORG_MEMBERSHIP - SHARE_REVOKED - SHARE_EXPIRED - SHARE_UNAVAILABLE - PROVIDER_KEY_MISSING - PROVIDER_KEY_EXPIRED - PROVIDER_RATE_LIMITED - PROVIDER_UNAUTHORIZED - PROVIDER_UNAVAILABLE - AGENT_HANDOFF_INVALID - AGENT_HANDOFF_EXPIRED - AGENT_HANDOFF_DENIED - AGENT_HANDOFF_CONSUMED - PAYLOAD_TOO_LARGE - METHOD_NOT_ALLOWED message: type: string details: {} requestId: type: string format: uuid redaction: type: object properties: applied: type: boolean enum: - true fieldCount: type: integer minimum: 0 secretCount: type: integer minimum: 0 required: - applied - fieldCount - secretCount required: - code - message required: - error responses: ValidationError: description: Validation error content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Unauthorized: description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string WWW-Authenticate: description: >- Bearer challenge. Agent-facing routes include the RFC 9728 resource_metadata URL. schema: type: string Forbidden: description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string NotFound: description: Resource not found content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Conflict: description: Conflict content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string RateLimited: description: Rate limit exceeded content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Retry-After: description: Delay in seconds before retrying when present. schema: type: integer minimum: 1 InternalError: description: Internal server error content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string ServiceUnavailable: description: Service unavailable content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: EvalGate API key description: >- Organization-scoped EvalGate API key. Discover the supported agent scopes at /.well-known/oauth-protected-resource. Per-operation grants are listed in x-evalgate-required-scopes; OpenAPI bearer security requirement arrays remain empty as required for non-OAuth schemes. x-evalgate-scopes-supported: - eval:read - eval:write - objectives:read - objectives:write - runs:read - runs:write - traces:read - traces:write - exports:download - reports:write - prompt:publish - scorer:publish - red_team:read - red_team:run - red_team:approve - docs:read - agent:read - agent:execute - resources:read - resources:write - contexts:read - contexts:write - observations:write - checks:read - checks:run - standards:read - standards:write - standards:approve - grants:read - grants:write - admin:keys - admin:workspace - admin:org x-evalgate-protected-resource-metadata: https://www.evalgate.com/.well-known/oauth-protected-resource ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.