> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# CI/CD regression gates

> Run EvalGate in GitHub Actions or GitLab CI with local gates, eval artifacts, and platform quality checks.

# Set up CI/CD regression gates with EvalGate

Start with **`gate`**. Complete the [five-minute loop](https://evalgate.com/docs/quickstart-typescript.md)
once, then run that same command on every pull request. Add the later modes
only when you need their different evidence contract.

| Mode | What it proves | Credentials and cost | What a red build means | Artifacts |
| - | - | - | - | - |
| First: `gate` | Reviewed local checks still pass against the accepted baseline | No EvalGate credential or result charge; deterministic cases need no provider spend | Regression (exit `2`), or a distinct configuration, evidence or infrastructure failure; inspect the reason | `.evalgate/last-report.json`, `evals/regression-report.json` |
| Later: `ci` | Discovered specs execute, with optional base/head comparison | No EvalGate credential; opted-in model execution can incur BYOK spend | Spec execution, coverage or comparison failed; not the same baseline decision as `gate` | `.evalgate/` run and diff artifacts with `--write-results` |
| Later: `check` | Hosted evidence satisfies configured case, component, provider, judge and policy requirements | Organization credential and hosted usage; model-backed evidence adds provider spend | Required evidence is missing, invalid or below a threshold; a high blended score cannot override it | Hosted check report and optional failure import |

The [quality scoring contract](https://evalgate.com/docs/concepts/quality-score.md) describes components,
missing measurements and evidence completeness. The blended score is descriptive,
not a standalone release decision. Cost thresholds on completed evidence are
not pre-execution spending limits.

## Local gate setup

Complete the [five-minute local gate](https://evalgate.com/docs/quickstart-typescript.md) first. It covers
preview, explicit apply, replacement of every placeholder, discovery, the
reviewed-source commit and acceptance of a non-empty passing baseline. CI
consumes those reviewed files; it never accepts a baseline for you.

Commit `@evalgate/sdk` as a dev dependency and its package lockfile. Use the
workflow below so `npm ci` installs the reviewed version and `npx --no-install`
fails if that dependency is missing.

The published 3.15.0 initializer pins its own 3.15.0 in the generated workflow.
The source generator derives that pin from its own SDK\_VERSION, so it does not
carry an independent stale version. The lockfile-only template below is an
alternative for projects that commit the SDK dependency. Compare older generated
workflows with your installed SDK before committing them.

```yaml .github/workflows/evalgate-gate.yml theme={null} theme={null}
name: EvalGate Gate
on:
  pull_request:
    branches: [main]
  push:
    branches: [main]
jobs:
  regression-gate:
    runs-on: ubuntu-latest
    permissions:
      contents: read
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '24'
          cache: npm
      - run: npm ci
      - name: EvalGate regression gate
        run: npx --no-install @evalgate/sdk gate --format github --offline
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: evalgate-report
          path: .evalgate/last-report.json
          if-no-files-found: error
```

The canonical `.evalgate/last-report.json` is the release decision envelope;
`evals/regression-report.json` is the detailed regression artifact. Baselines
are reviewed evidence and are never automatically updated, committed, or
pushed by generated CI. Accept a baseline explicitly after reviewing a clean
run with `evalgate baseline update`.

> [!NOTE]
>
> The built-in local gate compares test pass state and parsed test count against the baseline. If your test script runs AI evals, this gates AI behavior. Otherwise, it gates test health until you add eval specs or a custom `eval:regression-gate` script.

## Later: hosted checks

Use `check` after the local loop when you need shared hosted evidence, independent judge credibility and import-on-failure behavior. Review availability and configure every required component; a score alone cannot demonstrate unmeasured safety.

```yaml .github/workflows/evalgate-platform-gate.yml theme={null} theme={null}
name: EvalGate Platform Gate
on:
  pull_request:
    branches: [main]
jobs:
  eval-gate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '24'
      - run: npm ci
      - name: EvalGate quality gate
        env:
          EVALGATE_API_KEY: ${{ secrets.EVALGATE_API_KEY }}
        run: npx @evalgate/sdk check --format github --onFail import
```

`check --format github` emits GitHub annotations for failed cases and writes a step summary with the verdict, score, baseline score, delta, and top failing cases. `--onFail import` uploads the failed run context to the platform so you can review it in the dashboard.

## GitLab CI recipe

GitLab supports a **local gate recipe only**. Connecting a GitLab repository for Repository Intelligence or automatic AI discovery is unsupported. This is not a native GitLab integration. Use JSON output and retain both decision artifacts:

```yaml .gitlab-ci.yml theme={null} theme={null}
eval-gate:
  stage: test
  image: node:24
  script:
    - npm ci
    - npx --no-install @evalgate/sdk gate --format json
  artifacts:
    when: always
    paths:
      - evals/regression-report.json
      - .evalgate/last-report.json
  only:
    - merge_requests
    - main
```

## Setting quality thresholds

Define local gate thresholds in `evalgate.config.json`. The built-in local gate uses `evals/baseline.json`; platform gates use the quality and judge settings configured for the evaluation.

```json evalgate.config.json theme={null} theme={null}
{
  "gate": {
    "baseline": "evals/baseline.json",
    "report": "evals/regression-report.json"
  },
  "judge": {
    "tprMin": 0.7,
    "tnrMin": 0.7,
    "minLabeledSamples": 30
  }
}
```

> [!WARNING]
>
> Store `evalgate.config.json` and `evals/baseline.json` in version control. Changing thresholds or baselines without review changes what your CI gate proves.

## Exit codes

The process exit status is the shell/CI result. JSON reports may additionally
carry `effectiveExitCode` and `releaseReady` when preview mode or evidence
policy separates process completion from the release decision. The complete,
version-bound contract is available from `evalgate capabilities --format json`.

| Code | Symbol | Meaning | Release meaning |
| - | - | - | - |
| `0` | `PASS` | All required evidence passed the configured gate. | Release-ready when the report also says `releaseReady=true`. |
| `1` | `SCORE_BELOW` | The measured score is below the configured minimum. | Blocks release. |
| `2` | `REGRESSION` | Measured behavior regressed beyond the accepted baseline tolerance. | Blocks release. |
| `3` | `POLICY_VIOLATION` | A configured policy check failed. | Blocks release. |
| `4` | `API_ERROR` | The hosted API or network request failed. | Release readiness is inconclusive; retry or use an admissible cache. |
| `5` | `BAD_ARGS` | The command arguments are invalid. | Blocks release until configuration is corrected. |
| `6` | `LOW_N` | The measured sample is too small for the configured decision. | Blocks release unless policy explicitly allows weak evidence. |
| `7` | `WEAK_EVIDENCE` | Evidence exists but is not strong enough for the gate. | Blocks release. |
| `8` | `WARN_REGRESSION` | A metric entered the configured warning band. | Policy-dependent; never treat as a stronger pass than reported. |
| `9` | `FAILURE_MODE_THRESHOLD` | A failure-mode threshold was exceeded. | Blocks release. |
| `10` | `JUDGE_CREDIBILITY_UNTRUSTWORTHY` | Judge calibration is too weak to support a trusted decision. | Blocks release or remains inconclusive per policy. |
| `11` | `IO_ERROR` | Required report or evidence could not be persisted. | Blocks release; a missing canonical report is not a pass. |
| `12` | `BASELINE_TAMPER` | The baseline checksum or attestation is invalid. | Blocks release until the reviewed baseline is restored. |
| `13` | `PROVIDER_UNAVAILABLE` | Required provider evidence was unavailable or stale. | Inconclusive or blocked; never release-green. |
| `14` | `PROVIDER_INCONCLUSIVE` | Provider work was deferred under an explicit policy. | Inconclusive unless policy explicitly admits deferred evidence. |
| `15` | `BASELINE_NOT_ACCEPTED` | The current run is measurable, but no reviewed baseline has been accepted. | Blocks release until a reviewed baseline is accepted. |
| `99` | `INTERNAL_ERROR` | An unexpected internal error occurred. | Blocks release. |
| `130` | `CANCELLED` | The operation was cancelled by the user or orchestrator. | No release decision was made. |

These meanings are authoritative for the installed runtime. Agents and CI
integrations can read the complete version-bound contract with
`npx @evalgate/sdk capabilities --format json`.

## CLI commands reference

```bash Local gate theme={null} theme={null}
npx @evalgate/sdk gate
npx @evalgate/sdk gate --format github
npx @evalgate/sdk gate --format json
```

```bash Eval artifact CI theme={null} theme={null}
npx @evalgate/sdk ci --format github --write-results --base main
npx @evalgate/sdk ci --format github --write-results --base main --impacted-only
```

```bash Platform gate theme={null} theme={null}
npx @evalgate/sdk check --format github --onFail import
npx @evalgate/sdk check --format json
```

## Best practices

> [card] Start local
>
> Use the local gate first so every repo gets a baseline before the team adopts platform workflows.

> [card] Upload artifacts
>
> Keep `.evalgate/` or `evals/regression-report.json` as CI artifacts so failures are debuggable after the job exits.

> [card] Use impacted-only deliberately
>
> Add `--impacted-only` only after your spec manifest is stable. Unknown changes fall back to broader runs.

> [card] Promote reviewed cases
>
> Treat generated candidate cases as drafts until a human or promotion workflow approves them for gating.

## Later: eval artifact CI

Use `ci` when your repo contains EvalGate spec files and you want each pull request to write `.evalgate/` run artifacts.

```yaml .github/workflows/evalgate.yml theme={null} theme={null}
name: EvalGate CI
on: [push, pull_request]
jobs:
  evalgate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
      - run: npm ci
      - run: npx @evalgate/sdk ci --format github --write-results --base main
      - uses: actions/upload-artifact@v4
        if: always()
        with:
          name: evalgate-results
          path: .evalgate/
```

That command:

* Discovers evaluation specs automatically
* Runs all specs by default
* Writes run artifacts when `--write-results` is set
* Compares against the base branch when `--base` is set
* Writes a GitHub step summary and annotations when `--format github` is set
* Exits with the appropriate code (see [exit codes](#exit-codes))

> [!NOTE]
>
> Add `--impacted-only` to run only specs affected by the current diff. Impact analysis requires a base branch, so use it with `--base main` or another base ref.

## Other CI hosts and languages

Jenkins and Buildkite can invoke the same local command and capture artifacts.
JUnit XML is a source candidate until its release is listed in feature status;
use `gate --format junit > evalgate.xml` only with a build that advertises it.
This is generic CI execution, not a native integration. TypeScript and Python
are evaluation runtimes. Go/Ruby generated REST clients are not runners; Java
and .NET evaluation runners are unsupported.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.