> ## Documentation Index > Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Get evaluations runId > Requires scopes: runs:read ## OpenAPI ````yaml /api-reference/openapi.json get /api/evaluations/{id}/runs/{runId} openapi: 3.1.2 info: title: EvalGate API version: 18.13.0 description: EvalGate API. See docs/api-contract.md for stability commitments. servers: - url: https://evalgate.com description: Production - url: http://localhost:3000 description: Local security: [] paths: /api/evaluations/{id}/runs/{runId}: get: tags: - evaluations summary: Get evaluations runId description: 'Requires scopes: runs:read' operationId: get_evaluations_id_runs_runId parameters: - name: id in: path required: true schema: type: integer - name: runId in: path required: true schema: type: integer - name: compareRunId in: query required: false schema: type: string responses: '200': description: Successful response content: application/json: schema: $ref: >- #/components/schemas/GetApiEvaluationsIdRunsRunIdResponse200ApplicationJson '400': $ref: '#/components/responses/ValidationError' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/Conflict' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' '503': $ref: '#/components/responses/ServiceUnavailable' security: - bearerAuth: [] components: schemas: GetApiEvaluationsIdRunsRunIdResponse200ApplicationJson: type: object properties: enforcementReceipt: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number enum: - 1 id: type: string source: type: string enum: - evaluation_runs organizationId: $ref: '#/components/schemas/OrganizationId' target: type: string boundary: type: string enum: - post_run_evaluation action: type: object properties: evaluationId: type: number runId: type: number manifestHash: oneOf: - type: 'null' - type: string additionalProperties: false required: - evaluationId - runId - manifestHash environment: oneOf: - type: 'null' - type: string policy: type: object properties: gateDecision: oneOf: - type: 'null' - type: string enum: - BLOCK - type: string enum: - INDETERMINATE - type: string enum: - NO_DECISION - type: string enum: - PASS - type: string enum: - WARN gateEnforcementAction: oneOf: - type: 'null' - type: string enum: - ALLOW - type: string enum: - BLOCK - type: string enum: - NONE - type: string enum: - WARN manifestHash: oneOf: - type: 'null' - type: string additionalProperties: false required: - gateDecision - gateEnforcementAction - manifestHash timing: type: string enum: - after_execution status: type: string outcome: type: string scopeComplete: type: boolean attemptEvidence: type: object properties: complete: type: boolean count: oneOf: - type: 'null' - type: number digest: oneOf: - type: 'null' - type: string additionalProperties: false required: - complete - count - digest reconciliationHash: oneOf: - type: 'null' - type: string observedAt: type: string coverage: type: string receiptHash: type: string additionalProperties: false required: - schemaVersion - id - source - organizationId - target - boundary - action - environment - policy - timing - status - outcome - scopeComplete - attemptEvidence - reconciliationHash - observedAt - coverage - receiptHash run: type: object properties: id: type: number evaluationId: type: number organizationId: $ref: '#/components/schemas/OrganizationId' idempotencyKey: oneOf: - type: 'null' - type: string status: type: string executionMode: type: string runManifest: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number enum: - 1 evaluationId: type: number organizationId: $ref: '#/components/schemas/OrganizationId' dataset: type: object properties: id: oneOf: - type: 'null' - type: string versionId: oneOf: - type: 'null' - type: string cases: type: array items: type: object properties: testCaseId: type: number caseHash: type: string snapshot: type: object properties: name: type: string input: type: string expectedOutput: oneOf: - type: 'null' - type: string datasetVersionId: oneOf: - type: 'null' - type: string datasetRowContentHash: oneOf: - type: 'null' - type: string metadata: oneOf: - type: 'null' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - name - input - expectedOutput - datasetVersionId - datasetRowContentHash additionalProperties: false required: - testCaseId - caseHash additionalProperties: false required: - cases expectedTrialsPerCase: type: number execution: type: object properties: requestHash: type: string executionSettings: oneOf: - type: 'null' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' executionSettingsHash: type: string executorConfigHash: type: string experimentEvidence: type: object properties: snapshotHash: type: string executionId: type: string variantKey: type: string variantConfigurationHash: type: string additionalProperties: false required: - snapshotHash - executionId - variantKey - variantConfigurationHash harnessCommit: oneOf: - type: 'null' - type: string harnessSourceState: oneOf: - type: 'null' - type: string enum: - EXACT_COMMIT - type: string enum: - LOCAL_DIRTY_SNAPSHOT - type: string enum: - UNVERIFIED harnessDirtyDigest: oneOf: - type: 'null' - type: string model: oneOf: - type: 'null' - type: string provider: oneOf: - type: 'null' - type: string promptHash: oneOf: - type: 'null' - type: string systemPrompt: oneOf: - type: 'null' - type: string toolSchemaHash: oneOf: - type: 'null' - type: string judgeCertificateHash: oneOf: - type: 'null' - type: string runtimeImage: oneOf: - type: 'null' - type: string budgets: type: object properties: maxCostUsd: type: number maxWallTimeMs: type: number maxModelCalls: type: number additionalProperties: false timeoutRules: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' baselineConfigurationHash: oneOf: - type: 'null' - type: string additionalProperties: false createdAt: type: string additionalProperties: false required: - schemaVersion - evaluationId - organizationId - dataset - expectedTrialsPerCase - execution - createdAt runManifestHash: oneOf: - type: 'null' - type: string reconciliationSummary: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number enum: - 1 manifestHash: type: string expectedCaseCount: type: number expectedLogicalTrialCount: type: number receivedPhysicalAttemptCount: type: number completedLogicalTrialCount: type: number passedLogicalTrialCount: type: number failedLogicalTrialCount: type: number infrastructureFailureCount: type: number missingLogicalTrials: type: array items: type: object properties: testCaseId: type: number logicalTrialIndex: type: number additionalProperties: false required: - testCaseId - logicalTrialIndex invalidAttempts: type: array items: type: object properties: testCaseId: type: number logicalTrialIndex: type: number physicalAttemptIndex: type: number reason: oneOf: - type: string enum: - ATTEMPT_AFTER_COMPLETION - type: string enum: - ATTEMPT_INDEX_GAP - type: string enum: - COMPLETED_WITHOUT_ARTIFACT - type: string enum: - COMPLETED_WITHOUT_COMPLETION_TIME - type: string enum: - COMPLETED_WITHOUT_OUTCOME - type: string enum: - COMPLETED_WITH_INVALID_ARTIFACT - type: string enum: - DUPLICATE_ATTEMPT_INDEX - type: string enum: - FIRST_ATTEMPT_MARKED_RETRY - type: string enum: - MISSING_INITIAL_ATTEMPT - type: string enum: - MULTIPLE_COMPLETED_ATTEMPTS - type: string enum: - NON_COMPLETED_WITH_ARTIFACT - type: string enum: - NON_COMPLETED_WITH_OUTCOME - type: string enum: - NON_POSITIVE_ATTEMPT_INDEX - type: string enum: - RETRY_MARKED_INITIAL - type: string enum: - UNEXPECTED_CASE - type: string enum: - UNEXPECTED_TRIAL additionalProperties: false required: - testCaseId - logicalTrialIndex - physicalAttemptIndex - reason invalidRunReasons: type: array items: type: string terminalState: oneOf: - type: string enum: - CANCELLED - type: string enum: - COMPLETE - type: string enum: - COMPLETE_WITH_CASE_FAILURES - type: string enum: - INVALID - type: string enum: - PARTIAL reconciledAt: type: string additionalProperties: false required: - schemaVersion - manifestHash - expectedCaseCount - expectedLogicalTrialCount - receivedPhysicalAttemptCount - completedLogicalTrialCount - passedLogicalTrialCount - failedLogicalTrialCount - infrastructureFailureCount - missingLogicalTrials - invalidAttempts - invalidRunReasons - terminalState - reconciledAt terminalState: oneOf: - type: 'null' - type: string enum: - CANCELLED - type: string enum: - COMPLETE - type: string enum: - COMPLETE_WITH_CASE_FAILURES - type: string enum: - INVALID - type: string enum: - PARTIAL gateDecision: oneOf: - type: 'null' - type: string enum: - BLOCK - type: string enum: - INDETERMINATE - type: string enum: - NO_DECISION - type: string enum: - PASS - type: string enum: - WARN gateEnforcementAction: oneOf: - type: 'null' - type: string enum: - ALLOW - type: string enum: - BLOCK - type: string enum: - NONE - type: string enum: - WARN totalCases: oneOf: - type: 'null' - type: number passedCases: oneOf: - type: 'null' - type: number failedCases: oneOf: - type: 'null' - type: number processedCount: oneOf: - type: 'null' - type: number traceLog: oneOf: - type: 'null' - type: object properties: portableMeteringSettlement: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' resultUnitReservation: type: object properties: version: type: number enum: - 1 lockId: type: string units: type: number producerFinished: oneOf: - type: boolean enum: - false - type: boolean enum: - true additionalProperties: false required: - version - lockId - units messages: type: array items: type: object properties: role: oneOf: - type: string enum: - assistant - type: string enum: - developer - type: string enum: - system - type: string enum: - tool - type: string enum: - tool_result - type: string enum: - user content: type: string timestamp: type: string additionalProperties: false required: - role - content originalEvaluationId: type: number shadowEvalType: type: string resolvedTraceIds: type: array items: type: string jobId: type: number requestedAt: type: string enqueuedAt: type: string enqueueError: type: string lastError: type: string completedAt: type: string averageScore: type: number trajectoryCoverageRate: type: number trajectoryAverageScore: oneOf: - type: 'null' - type: number trajectoryFailureCounts: type: object additionalProperties: type: number error: type: string failedAt: type: string totalCases: type: number processedCount: type: number manifestHash: type: string reconciliation: type: object properties: schemaVersion: type: number enum: - 1 manifestHash: type: string expectedCaseCount: type: number expectedLogicalTrialCount: type: number receivedPhysicalAttemptCount: type: number completedLogicalTrialCount: type: number passedLogicalTrialCount: type: number failedLogicalTrialCount: type: number infrastructureFailureCount: type: number missingLogicalTrials: type: array items: type: object properties: testCaseId: type: number logicalTrialIndex: type: number additionalProperties: false required: - testCaseId - logicalTrialIndex invalidAttempts: type: array items: type: object properties: testCaseId: type: number logicalTrialIndex: type: number physicalAttemptIndex: type: number reason: oneOf: - type: string enum: - ATTEMPT_AFTER_COMPLETION - type: string enum: - ATTEMPT_INDEX_GAP - type: string enum: - COMPLETED_WITHOUT_ARTIFACT - type: string enum: - COMPLETED_WITHOUT_COMPLETION_TIME - type: string enum: - COMPLETED_WITHOUT_OUTCOME - type: string enum: - COMPLETED_WITH_INVALID_ARTIFACT - type: string enum: - DUPLICATE_ATTEMPT_INDEX - type: string enum: - FIRST_ATTEMPT_MARKED_RETRY - type: string enum: - MISSING_INITIAL_ATTEMPT - type: string enum: - MULTIPLE_COMPLETED_ATTEMPTS - type: string enum: - NON_COMPLETED_WITH_ARTIFACT - type: string enum: - NON_COMPLETED_WITH_OUTCOME - type: string enum: - NON_POSITIVE_ATTEMPT_INDEX - type: string enum: - RETRY_MARKED_INITIAL - type: string enum: - UNEXPECTED_CASE - type: string enum: - UNEXPECTED_TRIAL additionalProperties: false required: - testCaseId - logicalTrialIndex - physicalAttemptIndex - reason invalidRunReasons: type: array items: type: string terminalState: oneOf: - type: string enum: - CANCELLED - type: string enum: - COMPLETE - type: string enum: - COMPLETE_WITH_CASE_FAILURES - type: string enum: - INVALID - type: string enum: - PARTIAL reconciledAt: type: string additionalProperties: false required: - schemaVersion - manifestHash - expectedCaseCount - expectedLogicalTrialCount - receivedPhysicalAttemptCount - completedLogicalTrialCount - passedLogicalTrialCount - failedLogicalTrialCount - infrastructureFailureCount - missingLogicalTrials - invalidAttempts - invalidRunReasons - terminalState - reconciledAt cancelledAt: oneOf: - type: string - type: string format: date-time traceIds: type: array items: type: string import: type: object properties: sourceTraceId: type: string sourceOrgId: type: string importedAt: type: string source: type: string checkReport: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' gateCheckedAt: type: string requestHash: type: string manifestHash: type: string reconciliation: type: object properties: schemaVersion: type: number enum: - 1 manifestHash: type: string expectedCaseCount: type: number expectedLogicalTrialCount: type: number receivedPhysicalAttemptCount: type: number completedLogicalTrialCount: type: number passedLogicalTrialCount: type: number failedLogicalTrialCount: type: number infrastructureFailureCount: type: number missingLogicalTrials: type: array items: type: object properties: testCaseId: type: number logicalTrialIndex: type: number additionalProperties: false required: - testCaseId - logicalTrialIndex invalidAttempts: type: array items: type: object properties: testCaseId: type: number logicalTrialIndex: type: number physicalAttemptIndex: type: number reason: oneOf: - type: string enum: - ATTEMPT_AFTER_COMPLETION - type: string enum: - ATTEMPT_INDEX_GAP - type: string enum: - COMPLETED_WITHOUT_ARTIFACT - type: string enum: - COMPLETED_WITHOUT_COMPLETION_TIME - type: string enum: - COMPLETED_WITHOUT_OUTCOME - type: string enum: - COMPLETED_WITH_INVALID_ARTIFACT - type: string enum: - DUPLICATE_ATTEMPT_INDEX - type: string enum: - FIRST_ATTEMPT_MARKED_RETRY - type: string enum: - MISSING_INITIAL_ATTEMPT - type: string enum: - MULTIPLE_COMPLETED_ATTEMPTS - type: string enum: - NON_COMPLETED_WITH_ARTIFACT - type: string enum: - NON_COMPLETED_WITH_OUTCOME - type: string enum: - NON_POSITIVE_ATTEMPT_INDEX - type: string enum: - RETRY_MARKED_INITIAL - type: string enum: - UNEXPECTED_CASE - type: string enum: - UNEXPECTED_TRIAL additionalProperties: false required: - testCaseId - logicalTrialIndex - physicalAttemptIndex - reason invalidRunReasons: type: array items: type: string terminalState: oneOf: - type: string enum: - CANCELLED - type: string enum: - COMPLETE - type: string enum: - COMPLETE_WITH_CASE_FAILURES - type: string enum: - INVALID - type: string enum: - PARTIAL reconciledAt: type: string additionalProperties: false required: - schemaVersion - manifestHash - expectedCaseCount - expectedLogicalTrialCount - receivedPhysicalAttemptCount - completedLogicalTrialCount - passedLogicalTrialCount - failedLogicalTrialCount - infrastructureFailureCount - missingLogicalTrials - invalidAttempts - invalidRunReasons - terminalState - reconciledAt gatePolicy: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false type: type: string scoreImprovement: type: number dateRange: type: object properties: start: type: string end: type: string additionalProperties: false filters: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' createdBy: type: string additionalProperties: false trajectory: type: object properties: schemaVersion: type: number entries: type: array items: type: object properties: testCaseId: type: number status: oneOf: - type: string enum: - failed - type: string enum: - passed behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string trajectory: type: object properties: id: type: string input: type: string finalOutput: type: string steps: type: array items: type: object properties: index: type: number type: oneOf: - type: string enum: - final - type: string enum: - reasoning - type: string enum: - tool_call - type: string enum: - tool_result content: type: string toolName: type: string toolArgs: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' timestamp: type: number additionalProperties: false required: - index - type - content totalSteps: type: number totalToolCalls: type: number durationMs: type: number terminatedEarly: oneOf: - type: boolean enum: - false - type: boolean enum: - true error: type: string additionalProperties: false required: - id - input - finalOutput - steps - totalSteps - totalToolCalls - durationMs idealTrajectory: type: object properties: expectedSteps: type: number expectedToolCalls: type: number requiredTools: type: array items: type: string forbiddenTools: type: array items: type: string allowParallel: oneOf: - type: boolean enum: - false - type: boolean enum: - true maxRedundantSteps: type: number referenceToolCalls: type: array items: type: object properties: name: type: string args: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - name trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered toolArgsMatchMode: oneOf: - type: string enum: - exact - type: string enum: - ignore - type: string enum: - subset - type: string enum: - superset toolArgsMatchOverrides: type: object additionalProperties: oneOf: - type: string enum: - exact - type: string enum: - ignore - type: string enum: - subset - type: string enum: - superset - type: array items: type: string additionalProperties: false required: - expectedSteps - expectedToolCalls baseline: oneOf: - type: 'null' - type: object properties: behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string stepDistribution: type: object properties: p50: type: number p75: type: number p90: type: number additionalProperties: false required: - p50 - p75 - p90 toolCallDistribution: type: object properties: p50: type: number p75: type: number additionalProperties: false required: - p50 - p75 durationDistribution: type: object properties: p50: type: number p75: type: number additionalProperties: false required: - p50 - p75 commonToolSequences: type: array items: type: array items: type: string redundancyProfile: type: object properties: mean: type: number p90: type: number additionalProperties: false required: - mean - p90 parallelToolGroups: type: array items: type: array items: type: string sampleSize: type: number updatedAt: type: string version: type: string model: type: string additionalProperties: false required: - behavior - stepDistribution - toolCallDistribution - durationDistribution - commonToolSequences - redundancyProfile - sampleSize - updatedAt evaluation: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number trajectoryId: type: string behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string valid: type: boolean score: oneOf: - type: 'null' - type: number baselineVersion: oneOf: - type: 'null' - type: string failureModes: type: array items: oneOf: - type: string enum: - inefficient_path - type: string enum: - looping_behavior - type: string enum: - missed_tool_call - type: string enum: - non_parallel_execution - type: string enum: - premature_termination - type: string enum: - trajectory_mismatch - type: string enum: - unnecessary_tool_call metrics: oneOf: - type: 'null' - type: object properties: actualSteps: type: number actualToolCalls: type: number actualDurationMs: type: number redundancyRatio: type: number redundancySteps: type: number sequenceSimilarity: type: number efficiency: type: number toolUsage: type: number redundancy: type: number latency: type: number parallelizationPenalty: type: number parallelizationScore: type: number requiredToolsMissing: type: array items: type: string forbiddenToolsUsed: type: array items: type: string extraToolsUsed: type: array items: type: string selectedBaselineVersion: oneOf: - type: 'null' - type: string baselineStepP50: type: number baselineStepP75: type: number baselineToolCallP50: type: number baselineToolCallP75: type: number baselineDurationP50: type: number baselineDurationP75: type: number idealExpectedSteps: type: number idealExpectedToolCalls: type: number trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered trajectoryMatched: oneOf: - type: boolean enum: - false - type: boolean enum: - true referenceToolCallCount: type: number additionalProperties: false required: - actualSteps - actualToolCalls - actualDurationMs - redundancyRatio - redundancySteps - sequenceSimilarity - efficiency - toolUsage - redundancy - latency - parallelizationPenalty - parallelizationScore - requiredToolsMissing - forbiddenToolsUsed - extraToolsUsed additionalProperties: false required: - schemaVersion - valid - score - failureModes - metrics additionalProperties: false required: - testCaseId - status - trajectory additionalProperties: false required: - schemaVersion - entries trajectoryMetrics: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number totalResults: type: number evaluatedResults: type: number coverageRate: type: number averageTrajectoryScore: oneOf: - type: 'null' - type: number failureCounts: type: object properties: inefficient_path: type: number looping_behavior: type: number missed_tool_call: type: number non_parallel_execution: type: number premature_termination: type: number trajectory_mismatch: type: number unnecessary_tool_call: type: number additionalProperties: false byTestCase: type: array items: type: object properties: testCaseId: type: number status: oneOf: - type: string enum: - failed - type: string enum: - passed trajectoryId: type: string behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string score: oneOf: - type: 'null' - type: number baselineVersion: oneOf: - type: 'null' - type: string failureModes: type: array items: oneOf: - type: string enum: - inefficient_path - type: string enum: - looping_behavior - type: string enum: - missed_tool_call - type: string enum: - non_parallel_execution - type: string enum: - premature_termination - type: string enum: - trajectory_mismatch - type: string enum: - unnecessary_tool_call metrics: oneOf: - type: 'null' - type: object properties: actualSteps: type: number actualToolCalls: type: number actualDurationMs: type: number redundancyRatio: type: number redundancySteps: type: number sequenceSimilarity: type: number efficiency: type: number toolUsage: type: number redundancy: type: number latency: type: number parallelizationPenalty: type: number parallelizationScore: type: number requiredToolsMissing: type: array items: type: string forbiddenToolsUsed: type: array items: type: string extraToolsUsed: type: array items: type: string selectedBaselineVersion: oneOf: - type: 'null' - type: string baselineStepP50: type: number baselineStepP75: type: number baselineToolCallP50: type: number baselineToolCallP75: type: number baselineDurationP50: type: number baselineDurationP75: type: number idealExpectedSteps: type: number idealExpectedToolCalls: type: number trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered trajectoryMatched: oneOf: - type: boolean enum: - false - type: boolean enum: - true referenceToolCallCount: type: number additionalProperties: false required: - actualSteps - actualToolCalls - actualDurationMs - redundancyRatio - redundancySteps - sequenceSimilarity - efficiency - toolUsage - redundancy - latency - parallelizationPenalty - parallelizationScore - requiredToolsMissing - forbiddenToolsUsed - extraToolsUsed additionalProperties: false required: - testCaseId - status - score - failureModes - metrics additionalProperties: false required: - schemaVersion - totalResults - evaluatedResults - coverageRate - averageTrajectoryScore - failureCounts - byTestCase baselineVersion: oneOf: - type: 'null' - type: string scoreProvenance: oneOf: - type: 'null' - type: object properties: scoringVersion: type: string assertionSetVersion: type: string judgeConfigHash: type: string promptTemplateHash: type: string aggregationVersion: type: string embeddingModel: type: string taskType: type: string behaviorVersion: type: string calibrationStudyId: type: string additionalProperties: false required: - scoringVersion - assertionSetVersion - taskType measurementStatus: oneOf: - type: 'null' - type: string measurementSummary: oneOf: - type: 'null' - type: object properties: version: type: string enum: - evalgate.measurement.v1 status: oneOf: - type: string enum: - complete - type: string enum: - failed - type: string enum: - insufficient - type: string enum: - invalid - type: string enum: - partial - type: string enum: - pending - type: string enum: - queued - type: string enum: - unstable - type: string enum: - valid sampleSize: type: number uncertainty: type: object properties: lower: type: number upper: type: number confidenceLevel: type: number method: oneOf: - type: string enum: - newcombe-hybrid-score - type: string enum: - percentile-bootstrap - type: string enum: - welch-t - type: string enum: - wilson resamples: type: number seed: type: number additionalProperties: false required: - lower - upper - confidenceLevel - method metrics: type: object additionalProperties: oneOf: - type: 'null' - type: number warnings: type: array items: type: string computationHash: type: string additionalProperties: false required: - version - status - sampleSize - metrics - warnings - computationHash measurementCompletedAt: oneOf: - type: 'null' - type: string format: date-time activeMeasurementRevisionId: oneOf: - type: 'null' - type: string qualitySummary: oneOf: - type: 'null' - type: object properties: version: type: string enum: - evalgate.measurement.v1 status: oneOf: - type: string enum: - complete - type: string enum: - failed - type: string enum: - insufficient - type: string enum: - invalid - type: string enum: - partial - type: string enum: - pending - type: string enum: - queued - type: string enum: - unstable - type: string enum: - valid sampleSize: type: number uncertainty: type: object properties: lower: type: number upper: type: number confidenceLevel: type: number method: oneOf: - type: string enum: - newcombe-hybrid-score - type: string enum: - percentile-bootstrap - type: string enum: - welch-t - type: string enum: - wilson resamples: type: number seed: type: number additionalProperties: false required: - lower - upper - confidenceLevel - method metrics: type: object additionalProperties: oneOf: - type: 'null' - type: number warnings: type: array items: type: string computationHash: type: string additionalProperties: false required: - version - status - sampleSize - metrics - warnings - computationHash qualitySummaryCompletedAt: oneOf: - type: 'null' - type: string format: date-time orchestrationGraph: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number enum: - 1 rootNodeId: type: string nodes: type: array items: type: object properties: id: type: string type: oneOf: - type: string enum: - baseline - type: string enum: - candidate - type: string enum: - knowledge - type: string enum: - rejected - type: string enum: - winning label: type: string agentIndex: type: number strategyTrack: type: string passRate: type: number kept: oneOf: - type: boolean enum: - false - type: boolean enum: - true metadata: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - id - type - label edges: type: array items: type: object properties: from: type: string to: type: string label: type: string additionalProperties: false required: - from - to capturedAt: type: string gateDecisions: type: array items: type: object properties: timestamp: type: string passed: type: boolean exitCode: type: number reasonCode: type: string reasonMessage: oneOf: - type: 'null' - type: string policy: oneOf: - type: 'null' - type: string thresholds: type: object properties: minScore: type: number maxDrop: type: number warnDrop: type: number additionalProperties: false score: oneOf: - type: 'null' - type: number baselineScore: oneOf: - type: 'null' - type: number regressionDelta: oneOf: - type: 'null' - type: number baselineRunId: oneOf: - type: 'null' - type: number ciRunUrl: oneOf: - type: 'null' - type: string failedTestCaseIds: type: array items: type: number additionalProperties: false required: - timestamp - passed - exitCode - reasonCode - thresholds additionalProperties: false required: - schemaVersion - rootNodeId - nodes - edges - capturedAt startedAt: oneOf: - type: 'null' - type: string format: date-time completedAt: oneOf: - type: 'null' - type: string format: date-time environment: oneOf: - type: 'null' - type: string playgroundId: oneOf: - type: 'null' - type: number variantId: oneOf: - type: 'null' - type: number promptVersionId: oneOf: - type: 'null' - type: string baselineVariantId: oneOf: - type: 'null' - type: number triggerSource: oneOf: - type: 'null' - type: string createdAt: type: string format: date-time qualityScore: type: number qualityFlags: type: object properties: lowSampleSize: oneOf: - type: boolean enum: - false - type: boolean enum: - true highVariance: oneOf: - type: boolean enum: - false - type: boolean enum: - true additionalProperties: false additionalProperties: false required: - id - evaluationId - organizationId - idempotencyKey - status - executionMode - runManifest - runManifestHash - reconciliationSummary - terminalState - gateDecision - gateEnforcementAction - totalCases - passedCases - failedCases - processedCount - traceLog - trajectory - trajectoryMetrics - baselineVersion - scoreProvenance - measurementStatus - measurementSummary - measurementCompletedAt - activeMeasurementRevisionId - qualitySummary - qualitySummaryCompletedAt - orchestrationGraph - startedAt - completedAt - environment - playgroundId - variantId - promptVersionId - baselineVariantId - triggerSource - createdAt - qualityScore - qualityFlags results: type: array items: type: object properties: id: type: number evaluationRunId: type: number attemptId: oneOf: - type: 'null' - type: string testCaseId: type: number organizationId: $ref: '#/components/schemas/OrganizationId' logicalTrialIndex: type: number physicalAttemptIndex: type: number status: type: string remoteGradingState: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number enum: - 1 key: type: string jobId: type: string claimId: type: string runnerId: type: string dispatchAttempt: type: number finalStreamChecksum: oneOf: - type: 'null' - type: string state: oneOf: - type: string enum: - claimed - type: string enum: - completed - type: string enum: - indeterminate startedAt: type: string completedAt: type: string result: type: object properties: outcome: oneOf: - type: 'null' - type: string enum: - failed - type: string enum: - passed score: oneOf: - type: 'null' - type: number error: oneOf: - type: 'null' - type: string errorCode: oneOf: - type: 'null' - type: string assertionsJson: oneOf: - type: 'null' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - outcome - score - error - errorCode - assertionsJson additionalProperties: false required: - schemaVersion - key - jobId - claimId - runnerId - dispatchAttempt - finalStreamChecksum - state - startedAt output: oneOf: - type: 'null' - type: string score: oneOf: - type: 'null' - type: number error: oneOf: - type: 'null' - type: string assertionsJson: oneOf: - type: 'null' - type: object properties: version: type: string enum: - v1 taxonomyVersion: type: string assertions: type: array items: type: object properties: key: oneOf: - type: string enum: - functional - type: string enum: - hallucination - type: string enum: - instruction_following - type: string enum: - json_schema - type: string enum: - judge - type: string enum: - pii - type: string enum: - readability - type: string enum: - safety - type: string enum: - toxicity category: oneOf: - type: string enum: - format - type: string enum: - policy - type: string enum: - privacy - type: string enum: - quality - type: string enum: - safety passed: type: boolean score: type: number severity: oneOf: - type: string enum: - critical - type: string enum: - high - type: string enum: - low - type: string enum: - medium details: type: string applicable: oneOf: - type: boolean enum: - false - type: boolean enum: - true evidenceSources: type: array items: oneOf: - type: string enum: - assertion_result - type: string enum: - expected_output - type: string enum: - human_feedback - type: string enum: - input - type: string enum: - messages - type: string enum: - output - type: string enum: - tool_calls - type: string enum: - trajectory additionalProperties: false required: - key - category - passed additionalProperties: false required: - version - assertions - type: object properties: pii: oneOf: - type: boolean enum: - false - type: boolean enum: - true toxicity: oneOf: - type: boolean enum: - false - type: boolean enum: - true harmful: oneOf: - type: boolean enum: - false - type: boolean enum: - true additionalProperties: false - type: object additionalProperties: oneOf: - type: string - type: number - type: boolean enum: - false - type: boolean enum: - true traceLinkedMatched: oneOf: - type: 'null' - type: boolean enum: - false - type: boolean enum: - true traceId: oneOf: - type: 'null' - type: number hasProvenance: oneOf: - type: 'null' - type: boolean enum: - false - type: boolean enum: - true scoreProvenance: oneOf: - type: 'null' - type: object properties: scoringVersion: type: string assertionSetVersion: type: string judgeConfigHash: type: string promptTemplateHash: type: string aggregationVersion: type: string embeddingModel: type: string taskType: type: string behaviorVersion: type: string calibrationStudyId: type: string additionalProperties: false required: - scoringVersion - assertionSetVersion - taskType durationMs: oneOf: - type: 'null' - type: number messages: oneOf: - type: 'null' - type: array items: type: object properties: role: oneOf: - type: string enum: - assistant - type: string enum: - developer - type: string enum: - system - type: string enum: - tool - type: string enum: - tool_result - type: string enum: - user content: type: string timestamp: type: string additionalProperties: false required: - role - content toolCalls: oneOf: - type: 'null' - type: array items: type: object properties: name: type: string arguments: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' output: type: string timestamp: type: string success: oneOf: - type: 'null' - type: boolean enum: - false - type: boolean enum: - true toolClass: type: string callId: type: string target: type: string additionalProperties: false required: - name - arguments createdAt: type: string format: date-time test_cases: oneOf: - type: 'null' - type: object properties: name: type: string input: type: string expectedOutput: oneOf: - type: 'null' - type: string datasetVersionId: oneOf: - type: 'null' - type: string datasetRowContentHash: oneOf: - type: 'null' - type: string metadata: oneOf: - type: 'null' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - name - input - expectedOutput - datasetVersionId - datasetRowContentHash regressionPreparation: oneOf: - type: object properties: available: type: boolean enum: - false code: type: string reason: type: string additionalProperties: false required: - available - code - reason - type: object properties: available: type: boolean enum: - true sourceFingerprint: type: string reason: type: string additionalProperties: false required: - available - sourceFingerprint - reason trajectory: type: object properties: testCaseId: type: number status: oneOf: - type: string enum: - failed - type: string enum: - passed behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string trajectory: type: object properties: id: type: string input: type: string finalOutput: type: string steps: type: array items: type: object properties: index: type: number type: oneOf: - type: string enum: - final - type: string enum: - reasoning - type: string enum: - tool_call - type: string enum: - tool_result content: type: string toolName: type: string toolArgs: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' timestamp: type: number additionalProperties: false required: - index - type - content totalSteps: type: number totalToolCalls: type: number durationMs: type: number terminatedEarly: oneOf: - type: boolean enum: - false - type: boolean enum: - true error: type: string additionalProperties: false required: - id - input - finalOutput - steps - totalSteps - totalToolCalls - durationMs idealTrajectory: type: object properties: expectedSteps: type: number expectedToolCalls: type: number requiredTools: type: array items: type: string forbiddenTools: type: array items: type: string allowParallel: oneOf: - type: boolean enum: - false - type: boolean enum: - true maxRedundantSteps: type: number referenceToolCalls: type: array items: type: object properties: name: type: string args: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - name trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered toolArgsMatchMode: oneOf: - type: string enum: - exact - type: string enum: - ignore - type: string enum: - subset - type: string enum: - superset toolArgsMatchOverrides: type: object additionalProperties: oneOf: - type: string enum: - exact - type: string enum: - ignore - type: string enum: - subset - type: string enum: - superset - type: array items: type: string additionalProperties: false required: - expectedSteps - expectedToolCalls baseline: oneOf: - type: 'null' - type: object properties: behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string stepDistribution: type: object properties: p50: type: number p75: type: number p90: type: number additionalProperties: false required: - p50 - p75 - p90 toolCallDistribution: type: object properties: p50: type: number p75: type: number additionalProperties: false required: - p50 - p75 durationDistribution: type: object properties: p50: type: number p75: type: number additionalProperties: false required: - p50 - p75 commonToolSequences: type: array items: type: array items: type: string redundancyProfile: type: object properties: mean: type: number p90: type: number additionalProperties: false required: - mean - p90 parallelToolGroups: type: array items: type: array items: type: string sampleSize: type: number updatedAt: type: string version: type: string model: type: string additionalProperties: false required: - behavior - stepDistribution - toolCallDistribution - durationDistribution - commonToolSequences - redundancyProfile - sampleSize - updatedAt evaluation: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number trajectoryId: type: string behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string valid: type: boolean score: oneOf: - type: 'null' - type: number baselineVersion: oneOf: - type: 'null' - type: string failureModes: type: array items: oneOf: - type: string enum: - inefficient_path - type: string enum: - looping_behavior - type: string enum: - missed_tool_call - type: string enum: - non_parallel_execution - type: string enum: - premature_termination - type: string enum: - trajectory_mismatch - type: string enum: - unnecessary_tool_call metrics: oneOf: - type: 'null' - type: object properties: actualSteps: type: number actualToolCalls: type: number actualDurationMs: type: number redundancyRatio: type: number redundancySteps: type: number sequenceSimilarity: type: number efficiency: type: number toolUsage: type: number redundancy: type: number latency: type: number parallelizationPenalty: type: number parallelizationScore: type: number requiredToolsMissing: type: array items: type: string forbiddenToolsUsed: type: array items: type: string extraToolsUsed: type: array items: type: string selectedBaselineVersion: oneOf: - type: 'null' - type: string baselineStepP50: type: number baselineStepP75: type: number baselineToolCallP50: type: number baselineToolCallP75: type: number baselineDurationP50: type: number baselineDurationP75: type: number idealExpectedSteps: type: number idealExpectedToolCalls: type: number trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered trajectoryMatched: oneOf: - type: boolean enum: - false - type: boolean enum: - true referenceToolCallCount: type: number additionalProperties: false required: - actualSteps - actualToolCalls - actualDurationMs - redundancyRatio - redundancySteps - sequenceSimilarity - efficiency - toolUsage - redundancy - latency - parallelizationPenalty - parallelizationScore - requiredToolsMissing - forbiddenToolsUsed - extraToolsUsed additionalProperties: false required: - schemaVersion - valid - score - failureModes - metrics additionalProperties: false required: - testCaseId - status - trajectory trajectoryMetrics: type: object properties: testCaseId: type: number status: oneOf: - type: string enum: - failed - type: string enum: - passed trajectoryId: type: string behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string score: oneOf: - type: 'null' - type: number baselineVersion: oneOf: - type: 'null' - type: string failureModes: type: array items: oneOf: - type: string enum: - inefficient_path - type: string enum: - looping_behavior - type: string enum: - missed_tool_call - type: string enum: - non_parallel_execution - type: string enum: - premature_termination - type: string enum: - trajectory_mismatch - type: string enum: - unnecessary_tool_call metrics: oneOf: - type: 'null' - type: object properties: actualSteps: type: number actualToolCalls: type: number actualDurationMs: type: number redundancyRatio: type: number redundancySteps: type: number sequenceSimilarity: type: number efficiency: type: number toolUsage: type: number redundancy: type: number latency: type: number parallelizationPenalty: type: number parallelizationScore: type: number requiredToolsMissing: type: array items: type: string forbiddenToolsUsed: type: array items: type: string extraToolsUsed: type: array items: type: string selectedBaselineVersion: oneOf: - type: 'null' - type: string baselineStepP50: type: number baselineStepP75: type: number baselineToolCallP50: type: number baselineToolCallP75: type: number baselineDurationP50: type: number baselineDurationP75: type: number idealExpectedSteps: type: number idealExpectedToolCalls: type: number trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered trajectoryMatched: oneOf: - type: boolean enum: - false - type: boolean enum: - true referenceToolCallCount: type: number additionalProperties: false required: - actualSteps - actualToolCalls - actualDurationMs - redundancyRatio - redundancySteps - sequenceSimilarity - efficiency - toolUsage - redundancy - latency - parallelizationPenalty - parallelizationScore - requiredToolsMissing - forbiddenToolsUsed - extraToolsUsed additionalProperties: false required: - testCaseId - status - score - failureModes - metrics regressionCandidateId: oneOf: - type: 'null' - type: number regressionReviewItemId: oneOf: - type: 'null' - type: number additionalProperties: false required: - id - evaluationRunId - attemptId - testCaseId - organizationId - logicalTrialIndex - physicalAttemptIndex - status - remoteGradingState - output - score - error - assertionsJson - traceLinkedMatched - traceId - hasProvenance - scoreProvenance - durationMs - messages - toolCalls - createdAt - test_cases - regressionPreparation - trajectory - trajectoryMetrics - regressionCandidateId - regressionReviewItemId baselineResults: type: array items: type: object properties: id: type: number evaluationRunId: type: number attemptId: oneOf: - type: 'null' - type: string testCaseId: type: number organizationId: $ref: '#/components/schemas/OrganizationId' logicalTrialIndex: type: number physicalAttemptIndex: type: number status: type: string remoteGradingState: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number enum: - 1 key: type: string jobId: type: string claimId: type: string runnerId: type: string dispatchAttempt: type: number finalStreamChecksum: oneOf: - type: 'null' - type: string state: oneOf: - type: string enum: - claimed - type: string enum: - completed - type: string enum: - indeterminate startedAt: type: string completedAt: type: string result: type: object properties: outcome: oneOf: - type: 'null' - type: string enum: - failed - type: string enum: - passed score: oneOf: - type: 'null' - type: number error: oneOf: - type: 'null' - type: string errorCode: oneOf: - type: 'null' - type: string assertionsJson: oneOf: - type: 'null' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - outcome - score - error - errorCode - assertionsJson additionalProperties: false required: - schemaVersion - key - jobId - claimId - runnerId - dispatchAttempt - finalStreamChecksum - state - startedAt output: oneOf: - type: 'null' - type: string score: oneOf: - type: 'null' - type: number error: oneOf: - type: 'null' - type: string assertionsJson: oneOf: - type: 'null' - type: object properties: version: type: string enum: - v1 taxonomyVersion: type: string assertions: type: array items: type: object properties: key: oneOf: - type: string enum: - functional - type: string enum: - hallucination - type: string enum: - instruction_following - type: string enum: - json_schema - type: string enum: - judge - type: string enum: - pii - type: string enum: - readability - type: string enum: - safety - type: string enum: - toxicity category: oneOf: - type: string enum: - format - type: string enum: - policy - type: string enum: - privacy - type: string enum: - quality - type: string enum: - safety passed: type: boolean score: type: number severity: oneOf: - type: string enum: - critical - type: string enum: - high - type: string enum: - low - type: string enum: - medium details: type: string applicable: oneOf: - type: boolean enum: - false - type: boolean enum: - true evidenceSources: type: array items: oneOf: - type: string enum: - assertion_result - type: string enum: - expected_output - type: string enum: - human_feedback - type: string enum: - input - type: string enum: - messages - type: string enum: - output - type: string enum: - tool_calls - type: string enum: - trajectory additionalProperties: false required: - key - category - passed additionalProperties: false required: - version - assertions - type: object properties: pii: oneOf: - type: boolean enum: - false - type: boolean enum: - true toxicity: oneOf: - type: boolean enum: - false - type: boolean enum: - true harmful: oneOf: - type: boolean enum: - false - type: boolean enum: - true additionalProperties: false - type: object additionalProperties: oneOf: - type: string - type: number - type: boolean enum: - false - type: boolean enum: - true traceLinkedMatched: oneOf: - type: 'null' - type: boolean enum: - false - type: boolean enum: - true traceId: oneOf: - type: 'null' - type: number hasProvenance: oneOf: - type: 'null' - type: boolean enum: - false - type: boolean enum: - true scoreProvenance: oneOf: - type: 'null' - type: object properties: scoringVersion: type: string assertionSetVersion: type: string judgeConfigHash: type: string promptTemplateHash: type: string aggregationVersion: type: string embeddingModel: type: string taskType: type: string behaviorVersion: type: string calibrationStudyId: type: string additionalProperties: false required: - scoringVersion - assertionSetVersion - taskType durationMs: oneOf: - type: 'null' - type: number messages: oneOf: - type: 'null' - type: array items: type: object properties: role: oneOf: - type: string enum: - assistant - type: string enum: - developer - type: string enum: - system - type: string enum: - tool - type: string enum: - tool_result - type: string enum: - user content: type: string timestamp: type: string additionalProperties: false required: - role - content toolCalls: oneOf: - type: 'null' - type: array items: type: object properties: name: type: string arguments: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' output: type: string timestamp: type: string success: oneOf: - type: 'null' - type: boolean enum: - false - type: boolean enum: - true toolClass: type: string callId: type: string target: type: string additionalProperties: false required: - name - arguments createdAt: type: string format: date-time test_cases: oneOf: - type: 'null' - type: object properties: name: type: string input: type: string expectedOutput: oneOf: - type: 'null' - type: string datasetVersionId: oneOf: - type: 'null' - type: string datasetRowContentHash: oneOf: - type: 'null' - type: string metadata: oneOf: - type: 'null' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - name - input - expectedOutput - datasetVersionId - datasetRowContentHash regressionPreparation: oneOf: - type: object properties: available: type: boolean enum: - false code: type: string reason: type: string additionalProperties: false required: - available - code - reason - type: object properties: available: type: boolean enum: - true sourceFingerprint: type: string reason: type: string additionalProperties: false required: - available - sourceFingerprint - reason trajectory: type: object properties: testCaseId: type: number status: oneOf: - type: string enum: - failed - type: string enum: - passed behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string trajectory: type: object properties: id: type: string input: type: string finalOutput: type: string steps: type: array items: type: object properties: index: type: number type: oneOf: - type: string enum: - final - type: string enum: - reasoning - type: string enum: - tool_call - type: string enum: - tool_result content: type: string toolName: type: string toolArgs: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' timestamp: type: number additionalProperties: false required: - index - type - content totalSteps: type: number totalToolCalls: type: number durationMs: type: number terminatedEarly: oneOf: - type: boolean enum: - false - type: boolean enum: - true error: type: string additionalProperties: false required: - id - input - finalOutput - steps - totalSteps - totalToolCalls - durationMs idealTrajectory: type: object properties: expectedSteps: type: number expectedToolCalls: type: number requiredTools: type: array items: type: string forbiddenTools: type: array items: type: string allowParallel: oneOf: - type: boolean enum: - false - type: boolean enum: - true maxRedundantSteps: type: number referenceToolCalls: type: array items: type: object properties: name: type: string args: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' additionalProperties: false required: - name trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered toolArgsMatchMode: oneOf: - type: string enum: - exact - type: string enum: - ignore - type: string enum: - subset - type: string enum: - superset toolArgsMatchOverrides: type: object additionalProperties: oneOf: - type: string enum: - exact - type: string enum: - ignore - type: string enum: - subset - type: string enum: - superset - type: array items: type: string additionalProperties: false required: - expectedSteps - expectedToolCalls baseline: oneOf: - type: 'null' - type: object properties: behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string stepDistribution: type: object properties: p50: type: number p75: type: number p90: type: number additionalProperties: false required: - p50 - p75 - p90 toolCallDistribution: type: object properties: p50: type: number p75: type: number additionalProperties: false required: - p50 - p75 durationDistribution: type: object properties: p50: type: number p75: type: number additionalProperties: false required: - p50 - p75 commonToolSequences: type: array items: type: array items: type: string redundancyProfile: type: object properties: mean: type: number p90: type: number additionalProperties: false required: - mean - p90 parallelToolGroups: type: array items: type: array items: type: string sampleSize: type: number updatedAt: type: string version: type: string model: type: string additionalProperties: false required: - behavior - stepDistribution - toolCallDistribution - durationDistribution - commonToolSequences - redundancyProfile - sampleSize - updatedAt evaluation: oneOf: - type: 'null' - type: object properties: schemaVersion: type: number trajectoryId: type: string behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string valid: type: boolean score: oneOf: - type: 'null' - type: number baselineVersion: oneOf: - type: 'null' - type: string failureModes: type: array items: oneOf: - type: string enum: - inefficient_path - type: string enum: - looping_behavior - type: string enum: - missed_tool_call - type: string enum: - non_parallel_execution - type: string enum: - premature_termination - type: string enum: - trajectory_mismatch - type: string enum: - unnecessary_tool_call metrics: oneOf: - type: 'null' - type: object properties: actualSteps: type: number actualToolCalls: type: number actualDurationMs: type: number redundancyRatio: type: number redundancySteps: type: number sequenceSimilarity: type: number efficiency: type: number toolUsage: type: number redundancy: type: number latency: type: number parallelizationPenalty: type: number parallelizationScore: type: number requiredToolsMissing: type: array items: type: string forbiddenToolsUsed: type: array items: type: string extraToolsUsed: type: array items: type: string selectedBaselineVersion: oneOf: - type: 'null' - type: string baselineStepP50: type: number baselineStepP75: type: number baselineToolCallP50: type: number baselineToolCallP75: type: number baselineDurationP50: type: number baselineDurationP75: type: number idealExpectedSteps: type: number idealExpectedToolCalls: type: number trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered trajectoryMatched: oneOf: - type: boolean enum: - false - type: boolean enum: - true referenceToolCallCount: type: number additionalProperties: false required: - actualSteps - actualToolCalls - actualDurationMs - redundancyRatio - redundancySteps - sequenceSimilarity - efficiency - toolUsage - redundancy - latency - parallelizationPenalty - parallelizationScore - requiredToolsMissing - forbiddenToolsUsed - extraToolsUsed additionalProperties: false required: - schemaVersion - valid - score - failureModes - metrics additionalProperties: false required: - testCaseId - status - trajectory trajectoryMetrics: type: object properties: testCaseId: type: number status: oneOf: - type: string enum: - failed - type: string enum: - passed trajectoryId: type: string behavior: oneOf: - type: string enum: - classification - type: string enum: - extraction - type: string enum: - multi_step_reasoning - type: string enum: - retrieval - type: string enum: - tool_use taskType: type: string score: oneOf: - type: 'null' - type: number baselineVersion: oneOf: - type: 'null' - type: string failureModes: type: array items: oneOf: - type: string enum: - inefficient_path - type: string enum: - looping_behavior - type: string enum: - missed_tool_call - type: string enum: - non_parallel_execution - type: string enum: - premature_termination - type: string enum: - trajectory_mismatch - type: string enum: - unnecessary_tool_call metrics: oneOf: - type: 'null' - type: object properties: actualSteps: type: number actualToolCalls: type: number actualDurationMs: type: number redundancyRatio: type: number redundancySteps: type: number sequenceSimilarity: type: number efficiency: type: number toolUsage: type: number redundancy: type: number latency: type: number parallelizationPenalty: type: number parallelizationScore: type: number requiredToolsMissing: type: array items: type: string forbiddenToolsUsed: type: array items: type: string extraToolsUsed: type: array items: type: string selectedBaselineVersion: oneOf: - type: 'null' - type: string baselineStepP50: type: number baselineStepP75: type: number baselineToolCallP50: type: number baselineToolCallP75: type: number baselineDurationP50: type: number baselineDurationP75: type: number idealExpectedSteps: type: number idealExpectedToolCalls: type: number trajectoryMatchMode: oneOf: - type: string enum: - strict - type: string enum: - subset - type: string enum: - superset - type: string enum: - unordered trajectoryMatched: oneOf: - type: boolean enum: - false - type: boolean enum: - true referenceToolCallCount: type: number additionalProperties: false required: - actualSteps - actualToolCalls - actualDurationMs - redundancyRatio - redundancySteps - sequenceSimilarity - efficiency - toolUsage - redundancy - latency - parallelizationPenalty - parallelizationScore - requiredToolsMissing - forbiddenToolsUsed - extraToolsUsed additionalProperties: false required: - testCaseId - status - score - failureModes - metrics additionalProperties: false required: - id - evaluationRunId - attemptId - testCaseId - organizationId - logicalTrialIndex - physicalAttemptIndex - status - remoteGradingState - output - score - error - assertionsJson - traceLinkedMatched - traceId - hasProvenance - scoreProvenance - durationMs - messages - toolCalls - createdAt - test_cases - regressionPreparation - trajectory - trajectoryMetrics compareRunId: type: number additionalProperties: false required: - enforcementReceipt - run - results OrganizationId: type: string format: uuid JsonValue: oneOf: - type: 'null' - type: boolean - type: number - type: string - type: array items: $ref: '#/components/schemas/JsonValue' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' ApiError: type: object properties: error: type: object properties: code: type: string enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - INVALID_REFERENCE - VALIDATION_ERROR - RATE_LIMITED - CONFLICT - INTERNAL_ERROR - SERVICE_UNAVAILABLE - QUOTA_EXCEEDED - FEATURE_LIMIT_REACHED - NO_ORG_MEMBERSHIP - SHARE_REVOKED - SHARE_EXPIRED - SHARE_UNAVAILABLE - PROVIDER_KEY_MISSING - PROVIDER_KEY_EXPIRED - PROVIDER_RATE_LIMITED - PROVIDER_UNAUTHORIZED - PROVIDER_UNAVAILABLE - AGENT_HANDOFF_INVALID - AGENT_HANDOFF_EXPIRED - AGENT_HANDOFF_DENIED - AGENT_HANDOFF_CONSUMED - PAYLOAD_TOO_LARGE - METHOD_NOT_ALLOWED message: type: string details: {} requestId: type: string format: uuid redaction: type: object properties: applied: type: boolean enum: - true fieldCount: type: integer minimum: 0 secretCount: type: integer minimum: 0 required: - applied - fieldCount - secretCount required: - code - message required: - error responses: ValidationError: description: Validation error content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Unauthorized: description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string WWW-Authenticate: description: >- Bearer challenge. Agent-facing routes include the RFC 9728 resource_metadata URL. schema: type: string Forbidden: description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string NotFound: description: Resource not found content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Conflict: description: Conflict content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string RateLimited: description: Rate limit exceeded content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Retry-After: description: Delay in seconds before retrying when present. schema: type: integer minimum: 1 InternalError: description: Internal server error content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string ServiceUnavailable: description: Service unavailable content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: EvalGate API key description: >- Organization-scoped EvalGate API key. Discover the supported agent scopes at /.well-known/oauth-protected-resource. Per-operation grants are listed in x-evalgate-required-scopes; OpenAPI bearer security requirement arrays remain empty as required for non-OAuth schemes. x-evalgate-scopes-supported: - eval:read - eval:write - objectives:read - objectives:write - runs:read - runs:write - traces:read - traces:write - exports:download - reports:write - prompt:publish - scorer:publish - red_team:read - red_team:run - red_team:approve - docs:read - agent:read - agent:execute - resources:read - resources:write - contexts:read - contexts:write - observations:write - checks:read - checks:run - standards:read - standards:write - standards:approve - grants:read - grants:write - admin:keys - admin:workspace - admin:org x-evalgate-protected-resource-metadata: https://www.evalgate.com/.well-known/oauth-protected-resource ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.