Skip to main content

​
EvalGate CLI command reference

Complete reference for EvalGate CLI authentication, repository linking, gates, CI integration, trace labeling, failure analysis, judge orchestration, and auto loops.
The EvalGate CLI is the fastest way to run regression gates, analyze failure patterns, and automate prompt improvement without leaving your terminal. Run TypeScript CLI commands with npx @evalgate/sdk <command> for zero-install usage, or add @evalgate/sdk to your project and run the same commands through your package manager. The local CLI gate is Experimental, matching feature status. Start with the five-minute local gate. In the first hour, use gate; CI modes compares later ci and hosted check, their evidence, credentials, costs and blocking reasons. Removed commands return targeted, non-mutating migration guidance in both CLI routers; use the migration table below to select the current command. They never silently provision credentials or apply a scaffold.

​
Local core and hosted boundary

The MIT-licensed CLI gives cloud coding agents a credential-free local execution path. init --local performs project discovery and scaffolding with no account, browser, organization, GitHub remote, API key, or network request and creates no hosted identity. Spec execution, baseline updates, and regression gates then run in the checkout. Browser-approved login and durable repository link are a separate optional hosted path. Exact commit and remote-head details remain cloud activation snapshots, not durable link identity. Baselines and local evidence remain reviewable files that can run in any CI provider. The hosted control plane is additive. Connect it for shared history, managed traces, team review, access control, and organization policy. Local gates remain available when the hosted service is not configured.

​
Start a cloud-agent sandbox locally

Here local describes where evaluation executes; the intended caller may be a remote coding agent. JSON and non-TTY init is preview-only without explicit --apply.

​
Add hosted repository collaboration

The normal human path uses browser-approved device authorization and does not print or ask you to copy an EvalGate credential:
link detects the GitHub origin, current symbolic branch, exact local HEAD, and repository root. It previews the server-owned intake contract pinned to that exact target and asks before applying in a TTY. Branch movement does not unlink the project, and EvalGate never silently substitutes the remote branch head. For agents and CI, mutation remains explicit:
Repository configuration cannot supply a key or redirect a user session to another API origin. After installation, deterministic understand, init, health, baseline, and gate commands can run without a network round trip. status keeps durable linkage, checkout movement, cloud snapshot, and readiness separate. A checkout behind its tracked branch can remain linked, locally gate-ready, and cloud-targetable. Ahead or unpushed work needs the exact commit pushed before cloud evidence; diverged work needs an explicit appropriate branch. Dirty worktree state is reported independently because Git SHA alone cannot reproduce uncommitted changes.
The lifecycle on this page follows the currently published @evalgate/sdk package on npm, including credential-free init --local when that release advertises it. Discover commands with npx @evalgate/sdk --help and capabilities --format json; do not assume a source-tree command is published. The Python package evalgate-sdk==3.13.1 on PyPI ships the same release line; run pip show evalgate-sdk to confirm what is installed.
Both CLIs expose the same versioned capability map. Start here when you know the job you need to accomplish but not the internal command name:
The complete map of governed core workflows covers local product understanding, setup, evaluation, Playground, Prompt Hub, Dataset Hub, first-party evaluation packs, repository intelligence, experiments, scorer development, logs, continuous evaluation, Failure patterns, dashboards, Copilot, provider onboarding, red-team testing, remote runners, deployable assets, governance, and release. TypeScript and Python emit identical JSON from the generated contract. Every mapped workflow names its web route, public operation IDs, native automation command, artifacts, and next governed step; generation fails when any required workflow disappears.

​
Stable exit codes

The process exit status is the shell/CI result. JSON reports may additionally carry effectiveExitCode and releaseReady when preview mode or evidence policy separates process completion from the release decision. The complete, version-bound contract is available from evalgate capabilities --format json.

​
Call any public API operation

The TypeScript CLI’s api command consumes the generated public operation registry. This gives automation a single command for every operation in the API reference without falling back to private routes or database helpers. Treat it as an escape hatch for advanced automation; evalgate capabilities and the native workflow commands are the primary navigation surface.
Use --path name=value, --query name=value, and --header name=value more than once when an operation declares multiple inputs. --body accepts inline JSON or @path/to/body.json. Machine mode writes the exact successful API envelope to stdout; failures write a stable JSON error containing code, message, status, requestId, and exitCode. Ctrl-C aborts the active request and exits 130.

​
Red-team release evidence

To supply a local release gate with durable Red-Team evidence, request the specific completed run that the release is based on:
The first-class command exists in both SDK CLIs and uses the generated public API operation. It validates the strict artifact and canonical hash, verifies the server’s Ed25519 signature against redTeam.expectedAttestationPublicKeySha256, checks the configured organization, campaign snapshot, and exact release target, then atomically writes .evalgate/red-team-evidence.json. It compiles identity, completion, provider, attack, benign-utility, finding, remediation, golden, signed-report, and framework-reference evidence from that run. It does not execute a campaign or choose the latest run. A required local redTeam policy must pin the exact artifactId, versionId, and 64-character contentHash being released; the gate rejects missing, malformed, stale, mismatched, cross-organization, incomplete, or unverifiable evidence. Do not hand-author the JSON file or substitute a source-checkout digest for the hosted release target. A bare redTeam.required: true uses pre_release_strict: complete and fresh campaign evidence, attributable costs, compiler attestation, zero successful attacks, and 100% benign utility. It intentionally does not require a finding-scoped remediation report, so a clean first campaign can pass. Use post_remediation_strict when admitting a repaired finding; it additionally requires the baseline/fixed rerun comparison, no introduced fingerprints, active golden coverage, and an exact-hash signed report. The legacy strict name remains an alias for pre_release_strict. Pin expectedTarget, expectedOrganizationId, expectedSnapshotHash, expectedAttestationKeyId, and expectedAttestationPublicKeySha256 in evalgate.config.json. Use policyProfile: "custom" only for a reviewed policy exception. A fresh CI job runs evalgate red-team evidence --run <run-id> before evalgate gate; it never receives the server’s private signing key. In machine mode, export failures emit one JSON error envelope. Invalid arguments/configuration exit 5, API/auth/network failures exit 4, artifact integrity or identity-policy failures exit 3, local writes exit 11, and unexpected defects exit 99. Campaign creation, approval, and execution remain explicit actions requiring the appropriate Red-Team permissions. Fetching release evidence is a read-only compilation step.

​
Setup and initialization

Reads repository evidence and immediately returns a working theory of the product, its audience, the user’s job, the primary visible output, likely quality risks, confidence, unknowns, and one useful question. Inferred risks remain hypotheses until a person confirms them or representative behavior provides evidence.Quick mode is the default bounded scan of high-signal product files. Use --depth deep or --deep for a broader local scan. Both modes preview by default.
Human output is the default. --format json returns the complete plan envelope, while --format jsonl emits typed scan, evidence, theory, risk, question, and completion events for agents and automation. --json is an alias for JSON output.Preview writes nothing. It performs no network or model calls, executes no project commands, inspects or changes no Git state, and does not modify baselines, promote cases, or install telemetry. After review, --apply writes the canonical context to evalgate.product.json. A custom --output must remain inside the selected project root.
Product understanding is available in the current source tree. TypeScript and Python CLI parity is locally verified, but package-release evidence, REST and mounted web authoring, marketplace distribution, and controlled coding-agent compatibility proof are still pending.
Detects Node, Python, polyglot, and nested workspace projects. init --local is the credential-free cloud-agent path and creates no hosted context; plain init is the hosted-linked variant. In a TTY bare init previews and asks before applying. Explicit --preview never asks or writes. JSON and non-TTY calls remain non-mutating unless --apply is explicit. Init creates a pending baseline without executing tests or accepting results. It never mints or accepts an implicit activation key.The JSON plan includes project.aiUsage with evidence-backed agent frameworks, model providers, observability libraries, and tool protocols. Declared dependencies are high confidence; exact source imports are medium confidence. init --apply preserves the same evidence under agentRuntimeRegistry.discovery in evalgate.config.json and chooses a default runtime only when one framework is unambiguous.Discovery is local, bounded, and deterministic. It ignores dependency/build/virtual-environment directories, does not read .env files, and never guesses credentials or model names.
Supported first-class handlers are npm, pnpm, the legacy yarn alias, Yarn Classic, Yarn Modern, Bun, Deno, pip, uv, Poetry, Pipenv, PDM, Conda, Mamba, Hatch, Pixi, and explicit custom JSON argv. Conflicting JavaScript or Python/environment markers stop the plan. Repeat --package-handler <id> for an explicit polyglot composition; gate and baseline update execute every runnable handler from its owning root.
baseline update accepts only successful current evidence from the exact clean source that was evaluated. A pending baseline is not release-ready, and generated CI never updates, commits, or pushes one. The local gate writes its canonical release decision to .evalgate/last-report.json and detailed deltas to evals/regression-report.json.Then commit the generated files and push to trigger your first CI gate:
Removed command families fail without mutation using BAD_ARGS (exit 5) and targeted next actions. JSON mode emits one document and redacts secret-like arguments.An API key authorizes automation against an already established hosted context. It cannot bootstrap hosted repository linkage and is unnecessary for the local gate.
Diagnoses local project readiness without cloud access in quick mode. Omit --quick when you also want account and platform connectivity checks.
EvalGate is not limited to AI applications. A small local manifest can check safe HTTP routes, webhooks, background jobs, or command-based smoke tests. Checks run sequentially only when a developer or CI job calls them; no collector, daemon, cloud credential, or long-lived connection is required.
The JSON report contains bounded latency metrics, status/error counts, redacted reasons, and a suggested inspection path. In GitHub Actions, --format github appends the measured reasons to the PR Step Summary. The repository evalgate-application-health Skill helps a coding agent create the manifest and prepare a reviewed change; it does not edit or merge code automatically.Python projects use the parity command:
To evaluate delegation with the versioned evidence extension, see Evaluate your subagents. The existing command accepts an independent expected-root frame:
These comparisons are local and exploratory. Child sessions and retries are not independent root outcomes. Use a build containing the extension; generic exports do not certify native provider context inheritance.EvalGate can improve the harness used to build or operate an application, not only the application’s own AI behavior. Give the CLI a bounded JSONL file containing session outcomes and counters such as input/output tokens, repeated context, redundant tool calls, Skill loads, and cache hits.Each JSONL line contains counters only, for example:
The local report identifies likely waste such as repeated repository discovery, eager Skill loading, redundant reads, or overly long passing summaries. It waits for five outcome-labeled sessions, refuses to optimize when quality is below 70%, and shows a conservative token estimate. It never reads prompts, completions, tool arguments, traces, credentials, or starts a collector. It proposes a change; the existing eval suite must validate the base and candidate before the change is considered validated.
Python exposes the same commands as evalgate harness optimize and evalgate harness feedback. evalgate init --apply installs the portable evalgate-harness-optimization Skill in .agents/skills/ so a coding agent can explain the result and prepare the change without requiring a cloud connector.After the base-vs-candidate eval has passed, record validated feedback, then plan one managed Skill edit:
Apply requires the explicit --apply authority flag and the matching validated report hash. EvalGate changes only its own deterministic managed block, refuses stale/out-of-repository/symlink targets, and writes rollback metadata under .evalgate/harness-mutations/:
Python exposes the same mutation and rollback commands. Mutation remains local and on-demand: no daemon, cloud credentials, raw source snapshot, prompt, completion, or unrelated instruction-file rewrite is involved.

​
Gate and CI

​
Stability policy for probabilistic checks

An LLM call or model-judge verdict is not deterministic just because it runs in CI. Build hard gates on deterministic assertions where possible. For probabilistic checks, pin model and judge configuration, execute independent trials with the SDK’s repetitions option, set an explicit minimumPassRate, and fail or quarantine evidence reported in flakyCases. Retries are transient recovery attempts within a trial; they are surfaced as flaky evidence and should not be used to retry a verdict until it passes. A seed makes EvalGate’s case ordering reproducible but cannot prevent an external provider from changing a model. Recalibrate release-blocking judges against labeled examples, require the CLI’s judge TPR/TNR and minimum-sample thresholds, and keep the resulting credibility evidence with the gate report. Exit code 10 means the configured judge is not credible enough to block a release safely. Before any provider call, the auto editor and SDK model-assertion boundary reject common instruction overrides, role spoofing, and judge-score coercion in untrusted evaluation text. A security rejection fails closed and never falls back to an automatic prompt mutation. The check --onFail import network boundary recursively redacts nested secrets and common PII from the complete failed-run envelope; this convenience upload path has no raw-data opt-out.
Compares your current test results against the stored baseline and exits non-zero when the gate fails (for example 2 for a regression; see Stable exit codes).
Output format options:
Discovers eval specs, runs them, writes results, and compares against a base run when --base is provided. Add --impacted-only to run only specs affected by the current diff.
Full GitHub Actions workflow:
Runs the regression gate against the EvalGate platform (requires EVALGATE_API_KEY). Use --onFail import to upload failed run context to the dashboard for review.
Re-runs your tests and accepts the results as the new baseline only when they pass from a clean checkout. Run this after you intentionally change model behavior or fix a known issue.

​
Labeling and analysis

Steps through your unlabeled traces one by one. Use arrow keys to select pass/fail and pick a failure mode. Press u to undo the previous label. Press Ctrl-C to save progress and exit.
Each label you save becomes part of the golden dataset that can be used by later eval runs and gates.
Aggregates labeled traces and prints a frequency report of failure modes (counts and share of labeled failures).
Lists standard failure modes used by label, and optionally seeds or extends your evalgate.config.json alert weights.
Custom modes are stored under failureModeAlerts.modes in config and used by analyze and gate alerting.
Loads a candidate eval case from the platform and prints the minimized input for manual or scripted re-run. The legacy replay alias is deprecated — prefer replay-candidate.
Loads two saved run artifacts and emits a keep/discard decision for each case — useful for reviewing whether a prompt change improved or regressed specific failure modes.

​
Agent evidence and enterprise controls

Wraps an agent or CLI subprocess, captures stdout/stderr and tool events, and posts an evidence envelope to EvalGate (unless --local).
Useful flags:
Validates MCP manifests, audits connector readiness, or proxies a server while emitting evidence.
Add --format json for machine-readable audit output. Critical findings block by default; use --fail-on high (or low, medium, critical) to set the CI threshold. Use --strict-persistence separately when evidence must land on the platform.
Fetches org-scoped system controls: provider allowlists, PII scrub mode, retention policy, drift alerts, and guardrail state.
Requires EVALGATE_API_KEY with access to /api/system/controls.

​
Repository intelligence and evaluation packs

Use the local activation path first. After the gate is working and GitHub is connected, scan an exact commit, review the evidence, and then install coverage:
Repository scanning never executes repository code. Source findings and pack recommendations remain evidence-bounded and report incomplete scans honestly.

​
Advanced

Reads a saved run artifact and groups cases with similar failure patterns. Use the output to prioritize which failure mode to fix first.
Start with contrastive response examples or expand a labeled failure dataset. Profile mode previews locally without writing or calling a model.
Reads labeled failures and prior prompt history, generates the next candidate prompt edit, evaluates it against the specs that depend on the prompt, and keeps the edit only if none of those compared specs regress. The loop terminates on explicit guard conditions rather than running open-ended.auto run and every auto daemon cycle execute with .evalgate/auto/program.md itself, so its mutation families, adaptive-loop, swarm and stop conditions apply to unattended cycles too. budget.max_experiments caps a run; daemon.max_experiments_per_cycle sets the per-cycle budget.The loop splits the cases into three populations by source family (a family:<name> tag, otherwise the spec file), so cases derived from the same source never straddle populations. holdout.ratio sizes the selection population and holdout.confirmation_ratio the confirmation population; everything else is development. The assignment is written once to .evalgate/auto/populations.json and existing families never move; new families are placed as they appear.
  • The editor sees failures from development cases only.
  • Baseline and candidates run development and selection cases. A candidate that regresses any selection case is discarded, whatever its development gain.
  • Confirmation cases are never run, shown or read by the local loop. They are left for an independent confirmation of the frozen candidate.
The ledger (.evalgate/auto/ledger.jsonl, schema 2) records whether each dimension was measured. The selection population is recorded as the measured holdout, with its regression count; when there are too few source families to reserve one, the holdout is recorded as unavailable rather than as zero regressions. Cost is recorded only when the runs report it (for example with a cost budget source); otherwise it is unavailable, and auto history reports that spend as not measured instead of counting it as zero. Rows written by schema 1 are read back with holdout and cost unavailable.When program.md contains review: required: true (the auto init template does), auto run refuses to start until the plan is approved. auto review prints the plan: populations (confirmation only as counts and a digest), graders, protected slices, mutation target and spend bound, each with a risk level. auto review --approve records who approved which plan hash in .evalgate/auto/review.json; high-risk items must be accepted explicitly with --accept-risk <id>. Any change to the plan (a new case, a different budget) requires a new approval.
Every run is held to an Optimization Contract. auto contract --intent "…" compiles what you want (“Fix tone mismatch in the prompt. Keep accuracy within 1%. At most 5 experiments. Spend up to $3.”) together with program.md and the current measurement into .evalgate/auto/contract.json; without stated intent the contract is compiled from program.md. The contract is part of the reviewed plan, and auto run enforces it while executing:
  • Only the editable prompt may change. Spec files, grader helpers (dependsOn.code), datasets, evals/baseline.json, program.md and evalgate.config.json are immutable. If they changed since the contract was compiled, the run refuses to start. If a candidate’s execution changes any of them, the candidate is refused, the approved files are restored, and the refusal is kept in .evalgate/auto/details/*-execution.json (no ledger score is recorded).
  • The run may not ask for more trials than the contract allows. Once execution spend exceeds a stated limit, or can no longer be measured against it, the loop stops before the next candidate.
  • Confirmation cases are never requested.
  • Authority is fixed: the loop proposes and tests, EvalGate qualifies, and a person adopts and merges. Clauses the compiler does not understand are listed in the review and must be accepted by name (--accept-risk contract-unrecognized-intent).
Each ledger row records the contractHash it ran under.After a run, hand the kept candidate to people and to EvalGate:
  • auto freeze pins the kept candidate by content hash. It commits the candidate to an isolated branch (evalgate/auto/<session>) through a temporary git worktree and restores your working tree to the base content. If the working-tree prompt is not byte-identical to a kept candidate, the freeze is refused.
  • auto report writes .evalgate/auto/report.md. It covers every candidate tried and its hypothesis, and development and selection pass rates with 95% intervals. It also includes cost and latency with their boundaries, cheaper candidates that failed and the cases they broke, the exact sentences added and removed, spend, and limitations. The report states that the candidate is not confirmed.
  • auto export writes .evalgate/auto/campaign.json (evalgate.auto-campaign/v1). It carries the base and candidate commits and content hashes, the contract, and the development and selection observations. Confirmation cases appear only as a count and a digest.
  • auto propose runs all three. It prints a draft pull request command; --push pushes the branch and --open-pr opens the draft with gh. Nothing merges or deploys.
Import the export into EvalGate to confirm the candidate independently. EvalGate binds the candidate commit, then runs approved cases of a remote-runner evaluation on the base and candidate target versions. Only when quality holds on cases the optimizer never saw does the journey show the candidate as confirmed, with the cost claim checked from runner execution evidence. A person then starts release review and merges the pull request.
Scheduled campaigns. auto dispatch is the command a scheduler calls (a nightly CI job, a cron). It runs the reviewed plan exactly as approved, in a detached git worktree of the committed base with a copy of .evalgate/, so your checkout is never touched. It refuses with exit code 3 instead of running when:
  • program.md does not require review, or no plan is approved;
  • the plan changed since it was approved (cases, graders, budgets, contract or measured files), or the approval is older than --approval-max-age-days (default 14);
  • tracked files have uncommitted changes;
  • another dispatch holds the lock (a lock left by a dead or expired dispatch is recovered and that dispatch is recorded as abandoned);
  • this approval was already dispatched --max-per-approval times (default 3).
A scheduled checkout only sees what is committed, so commit the approved state: .evalgate/auto/program.md, populations.json, contract.json and review.json. Approving again (auto review --approve) and committing review.json is how a person renews or changes what the schedule may run.Dispatch state must outlive the checkout. Idempotency, the per-approval cap and overlap protection are only as durable as the record of earlier dispatches. A CI job starts from a fresh checkout, so --state-remote origin keeps that record as refs on the remote (refs/evalgate/dispatch/*): the lock, one record per dispatch, and cancel requests. Every write is a compare-and-swap push, so two scheduled jobs cannot both take the lock or the same attempt. Under CI (CI set) dispatch refuses with EPHEMERAL_STATE without --state-remote; pass --local-state only when the workspace persists between runs (a workstation, a self-hosted runner with a persistent workspace). If the remote cannot be read or written, dispatch refuses with STATE_UNAVAILABLE and nothing runs.The same approval and commit are dispatched once; a second trigger is a no-op. --max-runtime-minutes (default 120) stops a run on top of the contract’s trial and spend limits, and auto dispatch --cancel (with the same --state-remote, from any machine) stops the running one. Stopping terminates the loop’s whole process tree, including eval and provider subprocesses: SIGTERM, then SIGKILL after 10 seconds, and the worktree is removed only after every process has exited. A kept candidate is frozen on its own branch with a report and export under .evalgate/auto/dispatches/<id>/; --open-pr pushes it and opens a draft pull request. Nothing merges. auto dispatch --status lists recent dispatches.
When the contract’s objective is cost (“Cut cost by changing the prompt. Keep accuracy within 5%”), each candidate is judged on measured operating cost and preserved quality instead of a failure-mode delta. If cost is unknown on either run, the candidate is never kept as a saving. A candidate that is not cheaper, or that loses more pass rate than the protected margin, is discarded. A cheaper candidate within the margin is kept as selection evidence only. The cost claim itself needs per-case non-inferiority and independent confirmation, which EvalGate decides when the frozen candidate is qualified.
To repeat bounded cycles unattended (for example, overnight):
The program-driven loop above is implemented in the TypeScript CLI (npx @evalgate/sdk auto). The Python CLI’s evalgate auto run and evalgate auto daemon do not read program.md; they build a report from run artifacts you supply.
External-import intake is conditional on the installed CLI. Discover capabilities first:
import first shipped in @evalgate/sdk@3.12.5; 3.12.4 and earlier do not include it. If import is absent from the installed help and capability map, preserve the artifact and report the unsupported boundary; do not invent a hosted import route.When import is advertised, import inspect reads a result another tool produced and checks it independently. Supported today: Anthropic hillclimb/v2 (state.json from a build-eval or hillclimb run). EvalGate recomputes every score from the per-case observations and reports what the evidence supports and what it cannot establish.
An import is something EvalGate received, not something it proved: its authority is investigation only, it is not EvalGate evidence, and it never authorizes a release. The output separates observation integrity from the imported claim: a reported winner that the observations contradict is shown as DISAGREES WITH OBSERVATIONS, and EvalGate uses the recomputed winner. The paired delta is reported per case (reps averaged within each case, attempts counted separately) with a descriptive 95% interval, because the source cannot establish case independence and the candidate was selected on those cases. A content-addressed diff identifies the imported change, not the program that ran. Reported experiment spend, recomputed spend, candidate operating cost and verification cost are reported separately and never combined. A test split is a label and not a sealed holdout, a report hash identifies files and not the program that produced them, errored attempts are kept out of scores, and cost without model and token usage is unverified. Duplicate (case, rep) rows or undeclared variants are refused and nothing is imported. The command is local and read-only: no network, no credentials, no files written. Exit 0 means an assessment was produced, not that a candidate is ready; 5 means the document was refused; 11 means the file could not be read.
Scans your project for eval spec files, refreshes the manifest, and reports any redundant or overlapping specs. Run it after creating, editing, renaming, or deleting any eval file: until the manifest matches the files on disk, baseline update and gate refuse with COVERAGE_NOT_REGISTERED (exit 7) and name the untracked or stale files. Refreshing the manifest does not accept a baseline; review and commit the eval source, then run baseline update.

​
Judge commands

Prints all judges available in the EvalGate registry for your organization.
Prints the built-in judge presets (pre-configured provider + model + prompt combinations).
Runs a judge against a single input/output pair and prints the score, reasoning, and signals. Use this to validate a judge configuration before wiring it into your gate.
Example output:
Runs a judge against two candidate outputs for the same input and returns a preference decision with reasoning. Useful for A/B prompt comparisons.

​
Judge credibility config

Configure judge credibility thresholds and failure-mode alerts in evalgate.config.json at the root of your project:
Set bootstrapSeed to a fixed value (for example, 42) to make judge credibility calculations deterministic across CI runs. Without a fixed seed, bootstrap confidence intervals may vary slightly between runs.
When a judge’s discriminative power (TPR + TNR − 1) falls at or below 0.05, the gate skips score correction and exits with code 8 (WARN) instead of using a potentially biased score. When labeled sample count is below minLabeledSamples, bootstrap confidence intervals are also skipped — both conditions emit reason codes into the judgeCredibility block of the JSON report.

​
Complete command index

Every registered EvalGate CLI command. Run npx @evalgate/sdk <command> --help for the same text in your terminal.

​
Latency enforcement in SDK 3.13.1

SDK 3.13.1 supports tolerance.latencyEnforcement in an accepted local baseline. Upgrade older installed versions before relying on this policy.
advisory is the default for local and shared-CI evaluations. A measured increase above the tolerance appears as WARN, with the baseline, current measurement and delta retained. It cannot block solely because of latency. For a dedicated performance benchmark, explicitly choose blocking and a finite nonnegative maxLatencyIncreaseMs. This is a permitted increase from baseline, not an absolute response-time limit. Missing required comparison measurements block instead of appearing unchanged. Execution timeouts, unregistered coverage, incomplete execution evidence and behavioral failures remain blocking under both policies. Change baseline policy through your reviewed baseline workflow; editing an accepted checksummed baseline without reacceptance fails its integrity check. A performance policy is not a deployment permission or baseline approval.