> ## Documentation Index > Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Get experiments compare > Requires scopes: runs:read ## OpenAPI ````yaml /api-reference/openapi.json get /api/experiments/{id}/compare openapi: 3.1.2 info: title: EvalGate API version: 18.13.0 description: EvalGate API. See docs/api-contract.md for stability commitments. servers: - url: https://evalgate.com description: Production - url: http://localhost:3000 description: Local security: [] paths: /api/experiments/{id}/compare: get: tags: - experiments summary: Get experiments compare description: 'Requires scopes: runs:read' operationId: get_experiments_id_compare parameters: - name: id in: path required: true schema: type: integer responses: '200': description: Successful response content: application/json: schema: $ref: >- #/components/schemas/GetApiExperimentsIdCompareResponse200ApplicationJson '400': $ref: '#/components/responses/ValidationError' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/Conflict' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' '503': $ref: '#/components/responses/ServiceUnavailable' security: - bearerAuth: [] components: schemas: GetApiExperimentsIdCompareResponse200ApplicationJson: oneOf: - type: object properties: experimentId: type: string execution: type: 'null' comparisons: type: array items: $ref: '#/components/schemas/JsonValue' state: type: string additionalProperties: false required: - experimentId - execution - comparisons - state - type: object properties: experimentId: type: string execution: type: object properties: id: type: string organizationId: $ref: '#/components/schemas/OrganizationId' experimentId: type: string parentExecutionId: oneOf: - type: 'null' - type: string attempt: type: number status: type: string idempotencyKey: type: string requestHash: type: string totalVariants: type: number completedVariants: type: number cancelRequestedAt: oneOf: - type: 'null' - type: string format: date-time cancelledAt: oneOf: - type: 'null' - type: string format: date-time startedAt: oneOf: - type: 'null' - type: string format: date-time completedAt: oneOf: - type: 'null' - type: string format: date-time createdBy: type: string createdAt: type: string format: date-time additionalProperties: false required: - id - organizationId - experimentId - parentExecutionId - attempt - status - idempotencyKey - requestHash - totalVariants - completedVariants - cancelRequestedAt - cancelledAt - startedAt - completedAt - createdBy - createdAt comparisons: type: array items: type: object properties: bindingVersion: type: string enum: - evalgate.experiment-decision-binding.v2 baselineVariantKey: type: string candidateVariantKey: type: string classification: oneOf: - type: string enum: - invalid - type: string enum: - not_comparable - type: string enum: - regression - type: string enum: - inconclusive - type: string enum: - improvement - type: string enum: - tradeoff - type: string enum: - tie decisionLogic: type: array items: type: string comparable: type: boolean evidenceState: oneOf: - type: string enum: - invalid - type: string enum: - complete - type: string enum: - incomplete comparisonHash: type: string policyHash: type: string resolvedPolicy: type: object properties: schemaVersion: type: string enum: - evalgate.experiment-winner-policy.v2 minimumPassRateDelta: type: number maximumCostIncreasePercent: oneOf: - type: 'null' - type: number maximumP95LatencyIncreasePercent: oneOf: - type: 'null' - type: number allowIncomparableJudges: type: boolean enum: - false allowRegressions: type: boolean minimumSampleSize: oneOf: - type: 'null' - type: number confidenceLevel: type: number requireConfidenceInterval: type: boolean enum: - true statisticalMethod: type: string enum: - paired-bonferroni-wilson-v1 additionalProperties: false required: - schemaVersion - minimumPassRateDelta - maximumCostIncreasePercent - maximumP95LatencyIncreasePercent - allowIncomparableJudges - allowRegressions - minimumSampleSize - confidenceLevel - requireConfidenceInterval - statisticalMethod reconciliation: type: object properties: expectedLogicalObservations: type: number reconciledLogicalObservations: type: number acceptedPhysicalAttempts: type: number deduplicatedDeliveries: type: number diagnostics: type: array items: type: object properties: code: oneOf: - type: string enum: - missing_expected_frame - type: string enum: - invalid_expected_frame - type: string enum: - unsupported_logical_trial - type: string enum: - missing_observation - type: string enum: - unexpected_observation - type: string enum: - observation_binding_mismatch - type: string enum: - duplicate_attempt - type: string enum: - conflicting_duplicate_attempt - type: string enum: - invalid_retry_sequence - type: string enum: - multiple_completed_attempts - type: string enum: - missing_required_artifact - type: string enum: - incomplete_evidence - type: string enum: - missing_actual_provenance - type: string enum: - mismatched_actual_provenance - type: string enum: - ineligible_execution_state - type: string enum: - ineligible_run_state severity: oneOf: - type: string enum: - invalid - type: string enum: - warning - type: string enum: - incomplete message: type: string variantKey: type: string caseId: type: number logicalTrialIndex: type: number attemptId: oneOf: - type: 'null' - type: string additionalProperties: false required: - code - severity - message additionalProperties: false required: - expectedLogicalObservations - reconciledLogicalObservations - acceptedPhysicalAttempts - deduplicatedDeliveries - diagnostics metrics: type: object properties: quality: type: object properties: baselinePassRate: oneOf: - type: 'null' - type: number candidatePassRate: oneOf: - type: 'null' - type: number delta: oneOf: - type: 'null' - type: number confidence95: oneOf: - type: 'null' - type: object properties: low: type: number high: type: number additionalProperties: false required: - low - high independentPairCount: type: number additionalProperties: false required: - baselinePassRate - candidatePassRate - delta - confidence95 - independentPairCount scoreDistribution: type: object properties: baseline: type: array items: type: number candidate: type: array items: type: number meanDelta: oneOf: - type: 'null' - type: number additionalProperties: false required: - baseline - candidate - meanDelta cost: type: object properties: baselineUsd: oneOf: - type: 'null' - type: number candidateUsd: oneOf: - type: 'null' - type: number deltaUsd: oneOf: - type: 'null' - type: number unknownBaselineRows: type: number unknownCandidateRows: type: number additionalProperties: false required: - baselineUsd - candidateUsd - deltaUsd - unknownBaselineRows - unknownCandidateRows latency: type: object properties: baselineP50Ms: oneOf: - type: 'null' - type: number candidateP50Ms: oneOf: - type: 'null' - type: number p50DeltaMs: oneOf: - type: 'null' - type: number baselineP95Ms: oneOf: - type: 'null' - type: number candidateP95Ms: oneOf: - type: 'null' - type: number p95DeltaMs: oneOf: - type: 'null' - type: number additionalProperties: false required: - baselineP50Ms - candidateP50Ms - p50DeltaMs - baselineP95Ms - candidateP95Ms - p95DeltaMs failures: type: object properties: baseline: type: object additionalProperties: type: number candidate: type: object additionalProperties: type: number newRegressions: type: number fixedRegressions: type: number toolCallDelta: type: number toolErrorDelta: type: number policyBlockDelta: type: number additionalProperties: false required: - baseline - candidate - newRegressions - fixedRegressions - toolCallDelta - toolErrorDelta - policyBlockDelta additionalProperties: false required: - quality - scoreDistribution - cost - latency - failures pairedRows: type: array items: type: object properties: caseId: type: number logicalTrialIndex: type: number baseline: oneOf: - type: 'null' - type: object properties: variantKey: type: string caseId: type: number testResultId: oneOf: - type: 'null' - type: number runId: type: number executionId: type: string executionAttempt: type: number logicalTrialIndex: type: number physicalAttemptIndex: type: number attemptId: oneOf: - type: 'null' - type: string attemptKind: oneOf: - type: string enum: - initial - type: string enum: - infrastructure_retry attemptStatus: oneOf: - type: string enum: - completed - type: string enum: - infrastructure_failure - type: string enum: - cancelled - type: string enum: - invalid attemptCompletedAt: oneOf: - type: 'null' - type: string artifactHash: oneOf: - type: 'null' - type: string caseContentHash: type: string variantConfigurationHash: type: string status: type: string output: oneOf: - type: 'null' - type: string score: oneOf: - type: 'null' - type: number durationMs: oneOf: - type: 'null' - type: number costUsd: oneOf: - type: 'null' - type: number costKnown: type: boolean failureMode: oneOf: - type: 'null' - type: string toolError: type: boolean toolCallCount: type: number policyBlocked: type: boolean actualEvaluatorIdentity: oneOf: - type: 'null' - type: string actualCalibrationIdentity: oneOf: - type: 'null' - type: string artifacts: type: array items: type: object properties: kind: oneOf: - type: string enum: - score_provenance - type: string enum: - result - type: string enum: - model_call_provenance contentHash: type: string additionalProperties: false required: - kind - contentHash judgeIdentity: oneOf: - type: 'null' - type: string traceId: oneOf: - type: 'null' - type: number modelCallId: oneOf: - type: 'null' - type: string evidenceComplete: type: boolean additionalProperties: false required: - variantKey - caseId - testResultId - runId - status - output - score - durationMs - costUsd - costKnown - failureMode - toolError - toolCallCount - policyBlocked - judgeIdentity - traceId - modelCallId - evidenceComplete candidate: oneOf: - type: 'null' - type: object properties: variantKey: type: string caseId: type: number testResultId: oneOf: - type: 'null' - type: number runId: type: number executionId: type: string executionAttempt: type: number logicalTrialIndex: type: number physicalAttemptIndex: type: number attemptId: oneOf: - type: 'null' - type: string attemptKind: oneOf: - type: string enum: - initial - type: string enum: - infrastructure_retry attemptStatus: oneOf: - type: string enum: - completed - type: string enum: - infrastructure_failure - type: string enum: - cancelled - type: string enum: - invalid attemptCompletedAt: oneOf: - type: 'null' - type: string artifactHash: oneOf: - type: 'null' - type: string caseContentHash: type: string variantConfigurationHash: type: string status: type: string output: oneOf: - type: 'null' - type: string score: oneOf: - type: 'null' - type: number durationMs: oneOf: - type: 'null' - type: number costUsd: oneOf: - type: 'null' - type: number costKnown: type: boolean failureMode: oneOf: - type: 'null' - type: string toolError: type: boolean toolCallCount: type: number policyBlocked: type: boolean actualEvaluatorIdentity: oneOf: - type: 'null' - type: string actualCalibrationIdentity: oneOf: - type: 'null' - type: string artifacts: type: array items: type: object properties: kind: oneOf: - type: string enum: - score_provenance - type: string enum: - result - type: string enum: - model_call_provenance contentHash: type: string additionalProperties: false required: - kind - contentHash judgeIdentity: oneOf: - type: 'null' - type: string traceId: oneOf: - type: 'null' - type: number modelCallId: oneOf: - type: 'null' - type: string evidenceComplete: type: boolean additionalProperties: false required: - variantKey - caseId - testResultId - runId - status - output - score - durationMs - costUsd - costKnown - failureMode - toolError - toolCallCount - policyBlocked - judgeIdentity - traceId - modelCallId - evidenceComplete qualityDelta: oneOf: - type: 'null' - type: number costDeltaUsd: oneOf: - type: 'null' - type: number latencyDeltaMs: oneOf: - type: 'null' - type: number classification: oneOf: - type: string enum: - regression - type: string enum: - improvement - type: string enum: - unchanged - type: string enum: - missing - type: string enum: - disagreement warnings: type: array items: type: string additionalProperties: false required: - caseId - baseline - candidate - qualityDelta - costDeltaUsd - latencyDeltaMs - classification - warnings warnings: type: array items: type: string missingData: type: array items: type: string additionalProperties: false required: - bindingVersion - baselineVariantKey - candidateVariantKey - classification - decisionLogic - comparable - evidenceState - comparisonHash - policyHash - resolvedPolicy - reconciliation - metrics - pairedRows - warnings - missingData state: type: string additionalProperties: false required: - experimentId - execution - comparisons - state JsonValue: oneOf: - type: 'null' - type: boolean - type: number - type: string - type: array items: $ref: '#/components/schemas/JsonValue' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' OrganizationId: type: string format: uuid ApiError: type: object properties: error: type: object properties: code: type: string enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - INVALID_REFERENCE - VALIDATION_ERROR - RATE_LIMITED - CONFLICT - INTERNAL_ERROR - SERVICE_UNAVAILABLE - QUOTA_EXCEEDED - FEATURE_LIMIT_REACHED - NO_ORG_MEMBERSHIP - SHARE_REVOKED - SHARE_EXPIRED - SHARE_UNAVAILABLE - PROVIDER_KEY_MISSING - PROVIDER_KEY_EXPIRED - PROVIDER_RATE_LIMITED - PROVIDER_UNAUTHORIZED - PROVIDER_UNAVAILABLE - AGENT_HANDOFF_INVALID - AGENT_HANDOFF_EXPIRED - AGENT_HANDOFF_DENIED - AGENT_HANDOFF_CONSUMED - PAYLOAD_TOO_LARGE - METHOD_NOT_ALLOWED message: type: string details: {} requestId: type: string format: uuid redaction: type: object properties: applied: type: boolean enum: - true fieldCount: type: integer minimum: 0 secretCount: type: integer minimum: 0 required: - applied - fieldCount - secretCount required: - code - message required: - error responses: ValidationError: description: Validation error content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Unauthorized: description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string WWW-Authenticate: description: >- Bearer challenge. Agent-facing routes include the RFC 9728 resource_metadata URL. schema: type: string Forbidden: description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string NotFound: description: Resource not found content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Conflict: description: Conflict content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string RateLimited: description: Rate limit exceeded content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Retry-After: description: Delay in seconds before retrying when present. schema: type: integer minimum: 1 InternalError: description: Internal server error content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string ServiceUnavailable: description: Service unavailable content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: EvalGate API key description: >- Organization-scoped EvalGate API key. Discover the supported agent scopes at /.well-known/oauth-protected-resource. Per-operation grants are listed in x-evalgate-required-scopes; OpenAPI bearer security requirement arrays remain empty as required for non-OAuth schemes. x-evalgate-scopes-supported: - eval:read - eval:write - objectives:read - objectives:write - runs:read - runs:write - traces:read - traces:write - exports:download - reports:write - prompt:publish - scorer:publish - red_team:read - red_team:run - red_team:approve - docs:read - agent:read - agent:execute - resources:read - resources:write - contexts:read - contexts:write - observations:write - checks:read - checks:run - standards:read - standards:write - standards:approve - grants:read - grants:write - admin:keys - admin:workspace - admin:org x-evalgate-protected-resource-metadata: https://www.evalgate.com/.well-known/oauth-protected-resource ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.