> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluations

# Evaluations API — create and run evals

> Create evaluation definitions, retrieve them with test cases and runs, and start new runs against dev, staging, or production environments.

The Evaluations API is the core of EvalGate's quality loop. Use it to define what you want to measure, add test cases, and trigger runs that compute a pass/fail score you can gate on in CI or track over time in the dashboard.

## List evaluations

Returns a page of evaluations for the authenticated organization. Filter with `status`, `type`, or `search` (name).

```bash theme={null} theme={null}
curl "https://evalgate.com/api/evaluations" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

### Query parameters

**status** `string`

Filter by evaluation status. Accepted values: `draft`, `active`, `archived`.

**limit** `integer`

Maximum number of results to return. Defaults to 50, maximum 1000.

**offset** `integer`

Number of results to skip for pagination. Defaults to 0.

### Response

```json theme={null} theme={null}
{
  "evaluations": [
    {
      "id": 42,
      "name": "Chatbot regression",
      "description": null,
      "type": "unit_test",
      "status": "draft",
      "organizationId": "00000000-0000-4000-8000-000000000001",
      "createdAt": "2026-03-15T10:30:00.000Z",
      "updatedAt": "2026-03-15T10:30:00.000Z"
    }
  ],
  "hasMore": false,
  "limit": 50,
  "offset": 0
}
```

**evaluations** `array`

**Evaluation object fields**

**id** `integer`

Unique numeric ID of the evaluation.

**name** `string`

Display name.

**description** `string | null`

Optional description.

**type** `string`

Evaluation type, such as `standard`, `unit_test`, `human_eval`, `model_eval`, or `ab_test`.

**status** `string`

One of `draft`, `active`, `archived`.

**organizationId** `string`

UUID of the owning organization.

**createdAt** `string`

ISO 8601 creation timestamp.

**updatedAt** `string`

ISO 8601 last-updated timestamp.

***

## Create an evaluation

Creates a new evaluation definition. The evaluation starts in `draft` status.

```bash theme={null} theme={null}
curl https://evalgate.com/api/evaluations \
  -X POST \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "My Evaluation",
    "type": "unit_test"
  }'
```

### Request body

**name** `string` *required*

Display name for the evaluation.

**type** `string`

Evaluation type, such as `unit_test`, `human_eval`, `model_eval`, or `ab_test`. Defaults to `standard` (reference-graded).

**description** `string`

Optional description explaining the purpose of this evaluation.

**executionSettings** `object`

Optional settings controlling how the evaluation is executed (parallelism, timeout, etc.).

**modelSettings** `object`

Optional model configuration applied when the evaluation runner makes LLM calls.

**customMetrics** `array`

Optional array of custom metric definitions to compute alongside built-in scoring.

**testCases** `array`

Optional inline test cases to attach at creation time.

### Response

Returns `201 Created` with the created evaluation object:

```json theme={null} theme={null}
{
  "id": 43,
  "name": "My Evaluation",
  "type": "unit_test",
  "status": "draft",
  "organizationId": "00000000-0000-4000-8000-000000000001",
  "createdAt": "2026-03-15T10:35:00.000Z",
  "updatedAt": "2026-03-15T10:35:00.000Z"
}
```

***

## Get a single evaluation

Retrieve one evaluation by its numeric ID. The response includes the evaluation's `testCases` and recent `runs` arrays, which the list endpoint omits.

```bash theme={null} theme={null}
curl "https://evalgate.com/api/evaluations?id=42" \
  -H "Authorization: Bearer YOUR_API_KEY"
```

### Query parameters

**id** `integer` *required*

Numeric ID of the evaluation to retrieve.

### Response

```json theme={null} theme={null}
{
  "id": 42,
  "name": "Chatbot regression",
  "type": "unit_test",
  "status": "active",
  "organizationId": "00000000-0000-4000-8000-000000000001",
  "createdAt": "2026-03-15T10:30:00.000Z",
  "updatedAt": "2026-03-15T10:30:00.000Z",
  "testCases": [
    {
      "id": 1,
      "name": "Case A",
      "input": "What is your refund policy?"
    }
  ],
  "runs": [
    {
      "id": 9001,
      "status": "completed",
      "evaluationId": 42
    }
  ]
}
```

**testCases** `array`

Test cases attached to this evaluation.

**runs** `array`

Recent evaluation runs. Ordered by creation time, most recent first.

***

## Start an evaluation run

Triggers a new run for an existing evaluation. Pass an `environment` value to tag the run for filtering in the dashboard and in CI comparisons.

```bash theme={null} theme={null}
curl https://evalgate.com/api/evaluations/42/runs \
  -X POST \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"environment": "dev"}'
```

### Path parameters

**id** `integer` *required*

Numeric ID of the evaluation to run.

### Request body

**environment** `string`

Target environment for this run. Accepted values: `dev` (default), `staging`, `prod`. You can also pass the environment via the `x-evalgate-env` request header, which takes precedence over the body. Only owners, admins, or API keys can tag a run as `prod`.

### Response

```json theme={null} theme={null}
{
  "id": 9002,
  "evaluationId": 42,
  "status": "running",
  "environment": "dev",
  "createdAt": "2026-03-15T10:40:00.000Z"
}
```

**id** `integer`

Unique ID of the new run.

**evaluationId** `integer`

ID of the parent evaluation.

**status** `string`

Run status. If it is not yet terminal, poll the run or use webhooks to detect completion.

**environment** `string`

The environment value this run was tagged with.

**createdAt** `string`

ISO 8601 timestamp when the run was created.

> [!NOTE]
>
> To import results from your own test runner instead of triggering a managed run, use `POST /api/evaluations/{id}/runs/import` with an optional `Idempotency-Key` header to prevent duplicate runs on CI retry.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.