> ## Documentation Index > Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt > Use this file to discover all available pages before exploring further. # Create online evals simulate > Requires scopes: eval:write ## OpenAPI ````yaml /api-reference/openapi.json post /api/online-evals/monitors/{id}/simulate openapi: 3.1.2 info: title: EvalGate API version: 18.13.0 description: EvalGate API. See docs/api-contract.md for stability commitments. servers: - url: https://evalgate.com description: Production - url: http://localhost:3000 description: Local security: [] paths: /api/online-evals/monitors/{id}/simulate: post: tags: - online evals summary: Create online evals simulate description: 'Requires scopes: eval:write' operationId: post_online_evals_monitors_id_simulate parameters: - name: id in: path required: true schema: type: integer requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostApiOnlineEvalsMonitorsIdSimulateRequest' responses: '200': description: Successful response content: application/json: schema: $ref: >- #/components/schemas/PostApiOnlineEvalsMonitorsIdSimulateResponse200ApplicationJson '201': description: Successful response content: application/json: schema: $ref: >- #/components/schemas/PostApiOnlineEvalsMonitorsIdSimulateResponse201ApplicationJson '400': $ref: '#/components/responses/ValidationError' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '409': $ref: '#/components/responses/Conflict' '429': $ref: '#/components/responses/RateLimited' '500': $ref: '#/components/responses/InternalError' '503': $ref: '#/components/responses/ServiceUnavailable' security: - bearerAuth: [] components: schemas: PostApiOnlineEvalsMonitorsIdSimulateRequest: type: object properties: idempotencyKey: type: string windowStart: type: string format: date-time windowEnd: type: string format: date-time versionId: type: string limit: type: number additionalProperties: false required: - idempotencyKey - windowStart - windowEnd PostApiOnlineEvalsMonitorsIdSimulateResponse200ApplicationJson: oneOf: - type: object properties: idempotentReplay: type: boolean id: type: string organizationId: $ref: '#/components/schemas/OrganizationId' monitorId: type: number monitorVersionId: type: string runType: type: string status: oneOf: - type: string enum: - budget_blocked - type: string enum: - cancel_requested - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - completed_with_failures - type: string enum: - failed - type: string enum: - provider_outage - type: string enum: - queued - type: string enum: - running - type: string enum: - stale_version idempotencyKey: type: string requestHash: type: string claimToken: oneOf: - type: 'null' - type: string claimedBy: oneOf: - type: 'null' - type: string attempt: type: number windowStart: oneOf: - type: 'null' - type: string format: date-time windowEnd: oneOf: - type: 'null' - type: string format: date-time matchedTraces: type: number sampledTraces: type: number processedSamples: type: number failedSamples: type: number alertCount: type: number estimatedCostUsd: oneOf: - type: 'null' - type: string actualCostUsd: oneOf: - type: 'null' - type: string simulationResult: oneOf: - type: 'null' - type: object properties: observationFrame: type: object properties: version: type: string enum: - evalgate.production-observation-frame.v1 frameHash: type: string organizationId: $ref: '#/components/schemas/OrganizationId' scope: oneOf: - type: string enum: - captured_filtered_traces - type: string enum: - captured_production_traces predicateHash: type: string windowStart: type: string windowEnd: type: string capturedAt: type: string state: oneOf: - type: string enum: - censored - type: string enum: - complete selection: oneOf: - type: string enum: - census - type: string enum: - oldest_first_capped limit: type: number observedCount: type: number eligibleCount: oneOf: - type: 'null' - type: number eligibleLowerBound: type: number upstreamCaptureProbability: type: 'null' populationFrequency: type: 'null' units: type: array items: type: object properties: traceId: type: number traceKey: type: string traceRevisionId: oneOf: - type: 'null' - type: string traceRevisionHash: oneOf: - type: 'null' - type: string createdAt: type: string outcome: oneOf: - type: string enum: - failed - type: string enum: - pending - type: string enum: - succeeded - type: string enum: - unknown taskKey: oneOf: - type: 'null' - type: string inclusionProbability: oneOf: - type: 'null' - type: number additionalProperties: false required: - traceId - traceKey - traceRevisionId - traceRevisionHash - createdAt - outcome - taskKey - inclusionProbability uncertainty: type: array items: type: string additionalProperties: false required: - version - frameHash - organizationId - scope - predicateHash - windowStart - windowEnd - capturedAt - state - selection - limit - observedCount - eligibleCount - eligibleLowerBound - upstreamCaptureProbability - populationFrequency - units - uncertainty matchedTraceIds: type: array items: type: number sampledTraceIds: type: array items: type: number excluded: type: object additionalProperties: type: number estimatedModelCalls: type: number estimatedCostUsd: oneOf: - type: 'null' - type: number expectedAlertCount: type: number historicalFailureCount: type: number falsePositiveReviewTraceIds: type: array items: type: number warnings: type: array items: type: string additionalProperties: false required: - matchedTraceIds - sampledTraceIds - excluded - estimatedModelCalls - estimatedCostUsd - expectedAlertCount - historicalFailureCount - falsePositiveReviewTraceIds - warnings evidenceJson: type: object properties: observationFrame: type: object properties: version: type: string enum: - evalgate.production-observation-frame.v1 frameHash: type: string organizationId: $ref: '#/components/schemas/OrganizationId' scope: oneOf: - type: string enum: - captured_filtered_traces - type: string enum: - captured_production_traces predicateHash: type: string windowStart: type: string windowEnd: type: string capturedAt: type: string state: oneOf: - type: string enum: - censored - type: string enum: - complete selection: oneOf: - type: string enum: - census - type: string enum: - oldest_first_capped limit: type: number observedCount: type: number eligibleCount: oneOf: - type: 'null' - type: number eligibleLowerBound: type: number upstreamCaptureProbability: type: 'null' populationFrequency: type: 'null' units: type: array items: type: object properties: traceId: type: number traceKey: type: string traceRevisionId: oneOf: - type: 'null' - type: string traceRevisionHash: oneOf: - type: 'null' - type: string createdAt: type: string outcome: oneOf: - type: string enum: - failed - type: string enum: - pending - type: string enum: - succeeded - type: string enum: - unknown taskKey: oneOf: - type: 'null' - type: string inclusionProbability: oneOf: - type: 'null' - type: number additionalProperties: false required: - traceId - traceKey - traceRevisionId - traceRevisionHash - createdAt - outcome - taskKey - inclusionProbability uncertainty: type: array items: type: string additionalProperties: false required: - version - frameHash - organizationId - scope - predicateHash - windowStart - windowEnd - capturedAt - state - selection - limit - observedCount - eligibleCount - eligibleLowerBound - upstreamCaptureProbability - populationFrequency - units - uncertainty monitorVersionId: type: string monitorVersionNumber: type: number configHash: type: string policySnapshot: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' sampledTraceIds: type: array items: type: number scorerVersions: type: array items: type: object properties: artifactId: type: number versionId: type: number contentHash: type: string name: type: string type: type: string definition: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' calibrationBindingId: oneOf: - type: 'null' - type: string calibrationSetId: oneOf: - type: 'null' - type: string calibrationRunId: oneOf: - type: 'null' - type: string scoreMappingId: oneOf: - type: 'null' - type: string comparabilityStatus: oneOf: - type: string enum: - calibrated - type: string enum: - exact - type: string enum: - not_comparable additionalProperties: false required: - artifactId - versionId - contentHash - name - type - definition - calibrationBindingId - calibrationSetId - calibrationRunId - scoreMappingId - comparabilityStatus prompt: oneOf: - type: 'null' - type: object properties: artifactId: type: number versionId: type: number contentHash: type: string payloadHash: type: string name: type: string additionalProperties: false required: - artifactId - versionId - contentHash - payloadHash - name dataset: oneOf: - type: 'null' - type: object properties: datasetId: type: string versionId: type: string versionNumber: type: number contentHash: type: string rowCount: type: number additionalProperties: false required: - datasetId - versionId - versionNumber - contentHash - rowCount modelCallIds: type: array items: type: string costRecordIds: type: array items: type: number judgeResultIds: type: array items: type: number missingEvidence: type: array items: type: string retryHistory: type: array items: type: object properties: attempt: type: number startedAt: type: string completedAt: oneOf: - type: 'null' - type: string status: oneOf: - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - failed - type: string enum: - running errorCode: oneOf: - type: 'null' - type: string errorMessage: oneOf: - type: 'null' - type: string providerReached: type: boolean additionalProperties: false required: - attempt - startedAt - completedAt - status - errorCode - errorMessage - providerReached additionalProperties: false required: - monitorVersionId - monitorVersionNumber - configHash - policySnapshot - sampledTraceIds - scorerVersions - prompt - dataset - modelCallIds - costRecordIds - judgeResultIds - missingEvidence - retryHistory missingEvidence: type: array items: type: string retryHistory: type: array items: type: object properties: attempt: type: number startedAt: type: string completedAt: oneOf: - type: 'null' - type: string status: oneOf: - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - failed - type: string enum: - running errorCode: oneOf: - type: 'null' - type: string errorMessage: oneOf: - type: 'null' - type: string providerReached: type: boolean additionalProperties: false required: - attempt - startedAt - completedAt - status - errorCode - errorMessage - providerReached alertsEnabled: type: boolean candidateGenerationEnabled: type: boolean createdBy: type: string createdAt: type: string format: date-time startedAt: oneOf: - type: 'null' - type: string format: date-time completedAt: oneOf: - type: 'null' - type: string format: date-time cancelRequestedAt: oneOf: - type: 'null' - type: string format: date-time cancelledAt: oneOf: - type: 'null' - type: string format: date-time additionalProperties: false required: - idempotentReplay - id - organizationId - monitorId - monitorVersionId - runType - status - idempotencyKey - requestHash - claimToken - claimedBy - attempt - windowStart - windowEnd - matchedTraces - sampledTraces - processedSamples - failedSamples - alertCount - estimatedCostUsd - actualCostUsd - simulationResult - evidenceJson - missingEvidence - retryHistory - alertsEnabled - candidateGenerationEnabled - createdBy - createdAt - startedAt - completedAt - cancelRequestedAt - cancelledAt - type: object properties: budgetPreflight: type: object properties: allowed: type: boolean reasons: type: array items: type: string remainingCostUsd: oneOf: - type: 'null' - type: number remainingModelCalls: oneOf: - type: 'null' - type: number additionalProperties: false required: - allowed - reasons - remainingCostUsd - remainingModelCalls idempotentReplay: type: boolean actualCostUsd: oneOf: - type: 'null' - type: string alertCount: type: number alertsEnabled: type: boolean attempt: type: number cancelRequestedAt: oneOf: - type: 'null' - type: string format: date-time cancelledAt: oneOf: - type: 'null' - type: string format: date-time candidateGenerationEnabled: type: boolean claimToken: oneOf: - type: 'null' - type: string claimedBy: oneOf: - type: 'null' - type: string completedAt: oneOf: - type: 'null' - type: string format: date-time createdAt: type: string format: date-time createdBy: type: string estimatedCostUsd: oneOf: - type: 'null' - type: string evidenceJson: type: object properties: observationFrame: type: object properties: version: type: string enum: - evalgate.production-observation-frame.v1 frameHash: type: string organizationId: $ref: '#/components/schemas/OrganizationId' scope: oneOf: - type: string enum: - captured_filtered_traces - type: string enum: - captured_production_traces predicateHash: type: string windowStart: type: string windowEnd: type: string capturedAt: type: string state: oneOf: - type: string enum: - censored - type: string enum: - complete selection: oneOf: - type: string enum: - census - type: string enum: - oldest_first_capped limit: type: number observedCount: type: number eligibleCount: oneOf: - type: 'null' - type: number eligibleLowerBound: type: number upstreamCaptureProbability: type: 'null' populationFrequency: type: 'null' units: type: array items: type: object properties: traceId: type: number traceKey: type: string traceRevisionId: oneOf: - type: 'null' - type: string traceRevisionHash: oneOf: - type: 'null' - type: string createdAt: type: string outcome: oneOf: - type: string enum: - failed - type: string enum: - pending - type: string enum: - succeeded - type: string enum: - unknown taskKey: oneOf: - type: 'null' - type: string inclusionProbability: oneOf: - type: 'null' - type: number additionalProperties: false required: - traceId - traceKey - traceRevisionId - traceRevisionHash - createdAt - outcome - taskKey - inclusionProbability uncertainty: type: array items: type: string additionalProperties: false required: - version - frameHash - organizationId - scope - predicateHash - windowStart - windowEnd - capturedAt - state - selection - limit - observedCount - eligibleCount - eligibleLowerBound - upstreamCaptureProbability - populationFrequency - units - uncertainty monitorVersionId: type: string monitorVersionNumber: type: number configHash: type: string policySnapshot: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' sampledTraceIds: type: array items: type: number scorerVersions: type: array items: type: object properties: artifactId: type: number versionId: type: number contentHash: type: string name: type: string type: type: string definition: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' calibrationBindingId: oneOf: - type: 'null' - type: string calibrationSetId: oneOf: - type: 'null' - type: string calibrationRunId: oneOf: - type: 'null' - type: string scoreMappingId: oneOf: - type: 'null' - type: string comparabilityStatus: oneOf: - type: string enum: - calibrated - type: string enum: - exact - type: string enum: - not_comparable additionalProperties: false required: - artifactId - versionId - contentHash - name - type - definition - calibrationBindingId - calibrationSetId - calibrationRunId - scoreMappingId - comparabilityStatus prompt: oneOf: - type: 'null' - type: object properties: artifactId: type: number versionId: type: number contentHash: type: string payloadHash: type: string name: type: string additionalProperties: false required: - artifactId - versionId - contentHash - payloadHash - name dataset: oneOf: - type: 'null' - type: object properties: datasetId: type: string versionId: type: string versionNumber: type: number contentHash: type: string rowCount: type: number additionalProperties: false required: - datasetId - versionId - versionNumber - contentHash - rowCount modelCallIds: type: array items: type: string costRecordIds: type: array items: type: number judgeResultIds: type: array items: type: number missingEvidence: type: array items: type: string retryHistory: type: array items: type: object properties: attempt: type: number startedAt: type: string completedAt: oneOf: - type: 'null' - type: string status: oneOf: - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - failed - type: string enum: - running errorCode: oneOf: - type: 'null' - type: string errorMessage: oneOf: - type: 'null' - type: string providerReached: type: boolean additionalProperties: false required: - attempt - startedAt - completedAt - status - errorCode - errorMessage - providerReached additionalProperties: false required: - monitorVersionId - monitorVersionNumber - configHash - policySnapshot - sampledTraceIds - scorerVersions - prompt - dataset - modelCallIds - costRecordIds - judgeResultIds - missingEvidence - retryHistory failedSamples: type: number id: type: string idempotencyKey: type: string matchedTraces: type: number missingEvidence: type: array items: type: string monitorId: type: number monitorVersionId: type: string organizationId: $ref: '#/components/schemas/OrganizationId' processedSamples: type: number requestHash: type: string retryHistory: type: array items: type: object properties: attempt: type: number startedAt: type: string completedAt: oneOf: - type: 'null' - type: string status: oneOf: - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - failed - type: string enum: - running errorCode: oneOf: - type: 'null' - type: string errorMessage: oneOf: - type: 'null' - type: string providerReached: type: boolean additionalProperties: false required: - attempt - startedAt - completedAt - status - errorCode - errorMessage - providerReached runType: type: string sampledTraces: type: number simulationResult: oneOf: - type: 'null' - type: object properties: observationFrame: type: object properties: version: type: string enum: - evalgate.production-observation-frame.v1 frameHash: type: string organizationId: $ref: '#/components/schemas/OrganizationId' scope: oneOf: - type: string enum: - captured_filtered_traces - type: string enum: - captured_production_traces predicateHash: type: string windowStart: type: string windowEnd: type: string capturedAt: type: string state: oneOf: - type: string enum: - censored - type: string enum: - complete selection: oneOf: - type: string enum: - census - type: string enum: - oldest_first_capped limit: type: number observedCount: type: number eligibleCount: oneOf: - type: 'null' - type: number eligibleLowerBound: type: number upstreamCaptureProbability: type: 'null' populationFrequency: type: 'null' units: type: array items: type: object properties: traceId: type: number traceKey: type: string traceRevisionId: oneOf: - type: 'null' - type: string traceRevisionHash: oneOf: - type: 'null' - type: string createdAt: type: string outcome: oneOf: - type: string enum: - failed - type: string enum: - pending - type: string enum: - succeeded - type: string enum: - unknown taskKey: oneOf: - type: 'null' - type: string inclusionProbability: oneOf: - type: 'null' - type: number additionalProperties: false required: - traceId - traceKey - traceRevisionId - traceRevisionHash - createdAt - outcome - taskKey - inclusionProbability uncertainty: type: array items: type: string additionalProperties: false required: - version - frameHash - organizationId - scope - predicateHash - windowStart - windowEnd - capturedAt - state - selection - limit - observedCount - eligibleCount - eligibleLowerBound - upstreamCaptureProbability - populationFrequency - units - uncertainty matchedTraceIds: type: array items: type: number sampledTraceIds: type: array items: type: number excluded: type: object additionalProperties: type: number estimatedModelCalls: type: number estimatedCostUsd: oneOf: - type: 'null' - type: number expectedAlertCount: type: number historicalFailureCount: type: number falsePositiveReviewTraceIds: type: array items: type: number warnings: type: array items: type: string additionalProperties: false required: - matchedTraceIds - sampledTraceIds - excluded - estimatedModelCalls - estimatedCostUsd - expectedAlertCount - historicalFailureCount - falsePositiveReviewTraceIds - warnings startedAt: oneOf: - type: 'null' - type: string format: date-time status: oneOf: - type: string enum: - budget_blocked - type: string enum: - cancel_requested - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - completed_with_failures - type: string enum: - failed - type: string enum: - provider_outage - type: string enum: - queued - type: string enum: - running - type: string enum: - stale_version windowEnd: oneOf: - type: 'null' - type: string format: date-time windowStart: oneOf: - type: 'null' - type: string format: date-time additionalProperties: false required: - budgetPreflight - idempotentReplay - actualCostUsd - alertCount - alertsEnabled - attempt - cancelRequestedAt - cancelledAt - candidateGenerationEnabled - claimToken - claimedBy - completedAt - createdAt - createdBy - estimatedCostUsd - evidenceJson - failedSamples - id - idempotencyKey - matchedTraces - missingEvidence - monitorId - monitorVersionId - organizationId - processedSamples - requestHash - retryHistory - runType - sampledTraces - simulationResult - startedAt - status - windowEnd - windowStart PostApiOnlineEvalsMonitorsIdSimulateResponse201ApplicationJson: oneOf: - type: object properties: idempotentReplay: type: boolean id: type: string organizationId: $ref: '#/components/schemas/OrganizationId' monitorId: type: number monitorVersionId: type: string runType: type: string status: oneOf: - type: string enum: - budget_blocked - type: string enum: - cancel_requested - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - completed_with_failures - type: string enum: - failed - type: string enum: - provider_outage - type: string enum: - queued - type: string enum: - running - type: string enum: - stale_version idempotencyKey: type: string requestHash: type: string claimToken: oneOf: - type: 'null' - type: string claimedBy: oneOf: - type: 'null' - type: string attempt: type: number windowStart: oneOf: - type: 'null' - type: string format: date-time windowEnd: oneOf: - type: 'null' - type: string format: date-time matchedTraces: type: number sampledTraces: type: number processedSamples: type: number failedSamples: type: number alertCount: type: number estimatedCostUsd: oneOf: - type: 'null' - type: string actualCostUsd: oneOf: - type: 'null' - type: string simulationResult: oneOf: - type: 'null' - type: object properties: observationFrame: type: object properties: version: type: string enum: - evalgate.production-observation-frame.v1 frameHash: type: string organizationId: $ref: '#/components/schemas/OrganizationId' scope: oneOf: - type: string enum: - captured_filtered_traces - type: string enum: - captured_production_traces predicateHash: type: string windowStart: type: string windowEnd: type: string capturedAt: type: string state: oneOf: - type: string enum: - censored - type: string enum: - complete selection: oneOf: - type: string enum: - census - type: string enum: - oldest_first_capped limit: type: number observedCount: type: number eligibleCount: oneOf: - type: 'null' - type: number eligibleLowerBound: type: number upstreamCaptureProbability: type: 'null' populationFrequency: type: 'null' units: type: array items: type: object properties: traceId: type: number traceKey: type: string traceRevisionId: oneOf: - type: 'null' - type: string traceRevisionHash: oneOf: - type: 'null' - type: string createdAt: type: string outcome: oneOf: - type: string enum: - failed - type: string enum: - pending - type: string enum: - succeeded - type: string enum: - unknown taskKey: oneOf: - type: 'null' - type: string inclusionProbability: oneOf: - type: 'null' - type: number additionalProperties: false required: - traceId - traceKey - traceRevisionId - traceRevisionHash - createdAt - outcome - taskKey - inclusionProbability uncertainty: type: array items: type: string additionalProperties: false required: - version - frameHash - organizationId - scope - predicateHash - windowStart - windowEnd - capturedAt - state - selection - limit - observedCount - eligibleCount - eligibleLowerBound - upstreamCaptureProbability - populationFrequency - units - uncertainty matchedTraceIds: type: array items: type: number sampledTraceIds: type: array items: type: number excluded: type: object additionalProperties: type: number estimatedModelCalls: type: number estimatedCostUsd: oneOf: - type: 'null' - type: number expectedAlertCount: type: number historicalFailureCount: type: number falsePositiveReviewTraceIds: type: array items: type: number warnings: type: array items: type: string additionalProperties: false required: - matchedTraceIds - sampledTraceIds - excluded - estimatedModelCalls - estimatedCostUsd - expectedAlertCount - historicalFailureCount - falsePositiveReviewTraceIds - warnings evidenceJson: type: object properties: observationFrame: type: object properties: version: type: string enum: - evalgate.production-observation-frame.v1 frameHash: type: string organizationId: $ref: '#/components/schemas/OrganizationId' scope: oneOf: - type: string enum: - captured_filtered_traces - type: string enum: - captured_production_traces predicateHash: type: string windowStart: type: string windowEnd: type: string capturedAt: type: string state: oneOf: - type: string enum: - censored - type: string enum: - complete selection: oneOf: - type: string enum: - census - type: string enum: - oldest_first_capped limit: type: number observedCount: type: number eligibleCount: oneOf: - type: 'null' - type: number eligibleLowerBound: type: number upstreamCaptureProbability: type: 'null' populationFrequency: type: 'null' units: type: array items: type: object properties: traceId: type: number traceKey: type: string traceRevisionId: oneOf: - type: 'null' - type: string traceRevisionHash: oneOf: - type: 'null' - type: string createdAt: type: string outcome: oneOf: - type: string enum: - failed - type: string enum: - pending - type: string enum: - succeeded - type: string enum: - unknown taskKey: oneOf: - type: 'null' - type: string inclusionProbability: oneOf: - type: 'null' - type: number additionalProperties: false required: - traceId - traceKey - traceRevisionId - traceRevisionHash - createdAt - outcome - taskKey - inclusionProbability uncertainty: type: array items: type: string additionalProperties: false required: - version - frameHash - organizationId - scope - predicateHash - windowStart - windowEnd - capturedAt - state - selection - limit - observedCount - eligibleCount - eligibleLowerBound - upstreamCaptureProbability - populationFrequency - units - uncertainty monitorVersionId: type: string monitorVersionNumber: type: number configHash: type: string policySnapshot: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' sampledTraceIds: type: array items: type: number scorerVersions: type: array items: type: object properties: artifactId: type: number versionId: type: number contentHash: type: string name: type: string type: type: string definition: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' calibrationBindingId: oneOf: - type: 'null' - type: string calibrationSetId: oneOf: - type: 'null' - type: string calibrationRunId: oneOf: - type: 'null' - type: string scoreMappingId: oneOf: - type: 'null' - type: string comparabilityStatus: oneOf: - type: string enum: - calibrated - type: string enum: - exact - type: string enum: - not_comparable additionalProperties: false required: - artifactId - versionId - contentHash - name - type - definition - calibrationBindingId - calibrationSetId - calibrationRunId - scoreMappingId - comparabilityStatus prompt: oneOf: - type: 'null' - type: object properties: artifactId: type: number versionId: type: number contentHash: type: string payloadHash: type: string name: type: string additionalProperties: false required: - artifactId - versionId - contentHash - payloadHash - name dataset: oneOf: - type: 'null' - type: object properties: datasetId: type: string versionId: type: string versionNumber: type: number contentHash: type: string rowCount: type: number additionalProperties: false required: - datasetId - versionId - versionNumber - contentHash - rowCount modelCallIds: type: array items: type: string costRecordIds: type: array items: type: number judgeResultIds: type: array items: type: number missingEvidence: type: array items: type: string retryHistory: type: array items: type: object properties: attempt: type: number startedAt: type: string completedAt: oneOf: - type: 'null' - type: string status: oneOf: - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - failed - type: string enum: - running errorCode: oneOf: - type: 'null' - type: string errorMessage: oneOf: - type: 'null' - type: string providerReached: type: boolean additionalProperties: false required: - attempt - startedAt - completedAt - status - errorCode - errorMessage - providerReached additionalProperties: false required: - monitorVersionId - monitorVersionNumber - configHash - policySnapshot - sampledTraceIds - scorerVersions - prompt - dataset - modelCallIds - costRecordIds - judgeResultIds - missingEvidence - retryHistory missingEvidence: type: array items: type: string retryHistory: type: array items: type: object properties: attempt: type: number startedAt: type: string completedAt: oneOf: - type: 'null' - type: string status: oneOf: - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - failed - type: string enum: - running errorCode: oneOf: - type: 'null' - type: string errorMessage: oneOf: - type: 'null' - type: string providerReached: type: boolean additionalProperties: false required: - attempt - startedAt - completedAt - status - errorCode - errorMessage - providerReached alertsEnabled: type: boolean candidateGenerationEnabled: type: boolean createdBy: type: string createdAt: type: string format: date-time startedAt: oneOf: - type: 'null' - type: string format: date-time completedAt: oneOf: - type: 'null' - type: string format: date-time cancelRequestedAt: oneOf: - type: 'null' - type: string format: date-time cancelledAt: oneOf: - type: 'null' - type: string format: date-time additionalProperties: false required: - idempotentReplay - id - organizationId - monitorId - monitorVersionId - runType - status - idempotencyKey - requestHash - claimToken - claimedBy - attempt - windowStart - windowEnd - matchedTraces - sampledTraces - processedSamples - failedSamples - alertCount - estimatedCostUsd - actualCostUsd - simulationResult - evidenceJson - missingEvidence - retryHistory - alertsEnabled - candidateGenerationEnabled - createdBy - createdAt - startedAt - completedAt - cancelRequestedAt - cancelledAt - type: object properties: budgetPreflight: type: object properties: allowed: type: boolean reasons: type: array items: type: string remainingCostUsd: oneOf: - type: 'null' - type: number remainingModelCalls: oneOf: - type: 'null' - type: number additionalProperties: false required: - allowed - reasons - remainingCostUsd - remainingModelCalls idempotentReplay: type: boolean actualCostUsd: oneOf: - type: 'null' - type: string alertCount: type: number alertsEnabled: type: boolean attempt: type: number cancelRequestedAt: oneOf: - type: 'null' - type: string format: date-time cancelledAt: oneOf: - type: 'null' - type: string format: date-time candidateGenerationEnabled: type: boolean claimToken: oneOf: - type: 'null' - type: string claimedBy: oneOf: - type: 'null' - type: string completedAt: oneOf: - type: 'null' - type: string format: date-time createdAt: type: string format: date-time createdBy: type: string estimatedCostUsd: oneOf: - type: 'null' - type: string evidenceJson: type: object properties: observationFrame: type: object properties: version: type: string enum: - evalgate.production-observation-frame.v1 frameHash: type: string organizationId: $ref: '#/components/schemas/OrganizationId' scope: oneOf: - type: string enum: - captured_filtered_traces - type: string enum: - captured_production_traces predicateHash: type: string windowStart: type: string windowEnd: type: string capturedAt: type: string state: oneOf: - type: string enum: - censored - type: string enum: - complete selection: oneOf: - type: string enum: - census - type: string enum: - oldest_first_capped limit: type: number observedCount: type: number eligibleCount: oneOf: - type: 'null' - type: number eligibleLowerBound: type: number upstreamCaptureProbability: type: 'null' populationFrequency: type: 'null' units: type: array items: type: object properties: traceId: type: number traceKey: type: string traceRevisionId: oneOf: - type: 'null' - type: string traceRevisionHash: oneOf: - type: 'null' - type: string createdAt: type: string outcome: oneOf: - type: string enum: - failed - type: string enum: - pending - type: string enum: - succeeded - type: string enum: - unknown taskKey: oneOf: - type: 'null' - type: string inclusionProbability: oneOf: - type: 'null' - type: number additionalProperties: false required: - traceId - traceKey - traceRevisionId - traceRevisionHash - createdAt - outcome - taskKey - inclusionProbability uncertainty: type: array items: type: string additionalProperties: false required: - version - frameHash - organizationId - scope - predicateHash - windowStart - windowEnd - capturedAt - state - selection - limit - observedCount - eligibleCount - eligibleLowerBound - upstreamCaptureProbability - populationFrequency - units - uncertainty monitorVersionId: type: string monitorVersionNumber: type: number configHash: type: string policySnapshot: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' sampledTraceIds: type: array items: type: number scorerVersions: type: array items: type: object properties: artifactId: type: number versionId: type: number contentHash: type: string name: type: string type: type: string definition: type: object additionalProperties: $ref: '#/components/schemas/JsonValue' calibrationBindingId: oneOf: - type: 'null' - type: string calibrationSetId: oneOf: - type: 'null' - type: string calibrationRunId: oneOf: - type: 'null' - type: string scoreMappingId: oneOf: - type: 'null' - type: string comparabilityStatus: oneOf: - type: string enum: - calibrated - type: string enum: - exact - type: string enum: - not_comparable additionalProperties: false required: - artifactId - versionId - contentHash - name - type - definition - calibrationBindingId - calibrationSetId - calibrationRunId - scoreMappingId - comparabilityStatus prompt: oneOf: - type: 'null' - type: object properties: artifactId: type: number versionId: type: number contentHash: type: string payloadHash: type: string name: type: string additionalProperties: false required: - artifactId - versionId - contentHash - payloadHash - name dataset: oneOf: - type: 'null' - type: object properties: datasetId: type: string versionId: type: string versionNumber: type: number contentHash: type: string rowCount: type: number additionalProperties: false required: - datasetId - versionId - versionNumber - contentHash - rowCount modelCallIds: type: array items: type: string costRecordIds: type: array items: type: number judgeResultIds: type: array items: type: number missingEvidence: type: array items: type: string retryHistory: type: array items: type: object properties: attempt: type: number startedAt: type: string completedAt: oneOf: - type: 'null' - type: string status: oneOf: - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - failed - type: string enum: - running errorCode: oneOf: - type: 'null' - type: string errorMessage: oneOf: - type: 'null' - type: string providerReached: type: boolean additionalProperties: false required: - attempt - startedAt - completedAt - status - errorCode - errorMessage - providerReached additionalProperties: false required: - monitorVersionId - monitorVersionNumber - configHash - policySnapshot - sampledTraceIds - scorerVersions - prompt - dataset - modelCallIds - costRecordIds - judgeResultIds - missingEvidence - retryHistory failedSamples: type: number id: type: string idempotencyKey: type: string matchedTraces: type: number missingEvidence: type: array items: type: string monitorId: type: number monitorVersionId: type: string organizationId: $ref: '#/components/schemas/OrganizationId' processedSamples: type: number requestHash: type: string retryHistory: type: array items: type: object properties: attempt: type: number startedAt: type: string completedAt: oneOf: - type: 'null' - type: string status: oneOf: - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - failed - type: string enum: - running errorCode: oneOf: - type: 'null' - type: string errorMessage: oneOf: - type: 'null' - type: string providerReached: type: boolean additionalProperties: false required: - attempt - startedAt - completedAt - status - errorCode - errorMessage - providerReached runType: type: string sampledTraces: type: number simulationResult: oneOf: - type: 'null' - type: object properties: observationFrame: type: object properties: version: type: string enum: - evalgate.production-observation-frame.v1 frameHash: type: string organizationId: $ref: '#/components/schemas/OrganizationId' scope: oneOf: - type: string enum: - captured_filtered_traces - type: string enum: - captured_production_traces predicateHash: type: string windowStart: type: string windowEnd: type: string capturedAt: type: string state: oneOf: - type: string enum: - censored - type: string enum: - complete selection: oneOf: - type: string enum: - census - type: string enum: - oldest_first_capped limit: type: number observedCount: type: number eligibleCount: oneOf: - type: 'null' - type: number eligibleLowerBound: type: number upstreamCaptureProbability: type: 'null' populationFrequency: type: 'null' units: type: array items: type: object properties: traceId: type: number traceKey: type: string traceRevisionId: oneOf: - type: 'null' - type: string traceRevisionHash: oneOf: - type: 'null' - type: string createdAt: type: string outcome: oneOf: - type: string enum: - failed - type: string enum: - pending - type: string enum: - succeeded - type: string enum: - unknown taskKey: oneOf: - type: 'null' - type: string inclusionProbability: oneOf: - type: 'null' - type: number additionalProperties: false required: - traceId - traceKey - traceRevisionId - traceRevisionHash - createdAt - outcome - taskKey - inclusionProbability uncertainty: type: array items: type: string additionalProperties: false required: - version - frameHash - organizationId - scope - predicateHash - windowStart - windowEnd - capturedAt - state - selection - limit - observedCount - eligibleCount - eligibleLowerBound - upstreamCaptureProbability - populationFrequency - units - uncertainty matchedTraceIds: type: array items: type: number sampledTraceIds: type: array items: type: number excluded: type: object additionalProperties: type: number estimatedModelCalls: type: number estimatedCostUsd: oneOf: - type: 'null' - type: number expectedAlertCount: type: number historicalFailureCount: type: number falsePositiveReviewTraceIds: type: array items: type: number warnings: type: array items: type: string additionalProperties: false required: - matchedTraceIds - sampledTraceIds - excluded - estimatedModelCalls - estimatedCostUsd - expectedAlertCount - historicalFailureCount - falsePositiveReviewTraceIds - warnings startedAt: oneOf: - type: 'null' - type: string format: date-time status: oneOf: - type: string enum: - budget_blocked - type: string enum: - cancel_requested - type: string enum: - cancelled - type: string enum: - completed - type: string enum: - completed_with_failures - type: string enum: - failed - type: string enum: - provider_outage - type: string enum: - queued - type: string enum: - running - type: string enum: - stale_version windowEnd: oneOf: - type: 'null' - type: string format: date-time windowStart: oneOf: - type: 'null' - type: string format: date-time additionalProperties: false required: - budgetPreflight - idempotentReplay - actualCostUsd - alertCount - alertsEnabled - attempt - cancelRequestedAt - cancelledAt - candidateGenerationEnabled - claimToken - claimedBy - completedAt - createdAt - createdBy - estimatedCostUsd - evidenceJson - failedSamples - id - idempotencyKey - matchedTraces - missingEvidence - monitorId - monitorVersionId - organizationId - processedSamples - requestHash - retryHistory - runType - sampledTraces - simulationResult - startedAt - status - windowEnd - windowStart OrganizationId: type: string format: uuid JsonValue: oneOf: - type: 'null' - type: boolean - type: number - type: string - type: array items: $ref: '#/components/schemas/JsonValue' - type: object additionalProperties: $ref: '#/components/schemas/JsonValue' ApiError: type: object properties: error: type: object properties: code: type: string enum: - UNAUTHORIZED - FORBIDDEN - NOT_FOUND - INVALID_REFERENCE - VALIDATION_ERROR - RATE_LIMITED - CONFLICT - INTERNAL_ERROR - SERVICE_UNAVAILABLE - QUOTA_EXCEEDED - FEATURE_LIMIT_REACHED - NO_ORG_MEMBERSHIP - SHARE_REVOKED - SHARE_EXPIRED - SHARE_UNAVAILABLE - PROVIDER_KEY_MISSING - PROVIDER_KEY_EXPIRED - PROVIDER_RATE_LIMITED - PROVIDER_UNAUTHORIZED - PROVIDER_UNAVAILABLE - AGENT_HANDOFF_INVALID - AGENT_HANDOFF_EXPIRED - AGENT_HANDOFF_DENIED - AGENT_HANDOFF_CONSUMED - PAYLOAD_TOO_LARGE - METHOD_NOT_ALLOWED message: type: string details: {} requestId: type: string format: uuid redaction: type: object properties: applied: type: boolean enum: - true fieldCount: type: integer minimum: 0 secretCount: type: integer minimum: 0 required: - applied - fieldCount - secretCount required: - code - message required: - error responses: ValidationError: description: Validation error content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Unauthorized: description: Unauthorized content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string WWW-Authenticate: description: >- Bearer challenge. Agent-facing routes include the RFC 9728 resource_metadata URL. schema: type: string Forbidden: description: Forbidden content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string NotFound: description: Resource not found content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Conflict: description: Conflict content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string RateLimited: description: Rate limit exceeded content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string Retry-After: description: Delay in seconds before retrying when present. schema: type: integer minimum: 1 InternalError: description: Internal server error content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string ServiceUnavailable: description: Service unavailable content: application/json: schema: $ref: '#/components/schemas/ApiError' headers: RateLimit-Limit: description: Quota in requests for the active window. schema: type: integer minimum: 0 RateLimit-Remaining: description: Remaining requests in the active window. schema: type: integer minimum: 0 RateLimit-Reset: description: Reset delay in seconds (not a Unix timestamp). schema: type: integer minimum: 1 X-RateLimit-Limit: schema: type: integer minimum: 0 X-RateLimit-Remaining: schema: type: integer minimum: 0 X-RateLimit-Reset: description: Legacy reset timestamp in Unix milliseconds. schema: type: integer minimum: 0 RateLimit-Policy: description: IETF HTTPAPI structured quota policy (work in progress). schema: type: string RateLimit: description: >- IETF HTTPAPI structured remaining quota and reset delay (work in progress). schema: type: string securitySchemes: bearerAuth: type: http scheme: bearer bearerFormat: EvalGate API key description: >- Organization-scoped EvalGate API key. Discover the supported agent scopes at /.well-known/oauth-protected-resource. Per-operation grants are listed in x-evalgate-required-scopes; OpenAPI bearer security requirement arrays remain empty as required for non-OAuth schemes. x-evalgate-scopes-supported: - eval:read - eval:write - objectives:read - objectives:write - runs:read - runs:write - traces:read - traces:write - exports:download - reports:write - prompt:publish - scorer:publish - red_team:read - red_team:run - red_team:approve - docs:read - agent:read - agent:execute - resources:read - resources:write - contexts:read - contexts:write - observations:write - checks:read - checks:run - standards:read - standards:write - standards:approve - grants:read - grants:write - admin:keys - admin:workspace - admin:org x-evalgate-protected-resource-metadata: https://www.evalgate.com/.well-known/oauth-protected-resource ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.