> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Python quick start

> Block your first regression with evalgate-sdk in about five minutes, with no account.

# Python quick start

You will write reviewed evals for one function, accept a baseline, and watch
the gate block a change that breaks it. No EvalGate account, API key, or network
call to EvalGate is involved. Python 3.10+ is supported.

## 1. Install and scaffold

```bash theme={null}
pip install evalgate-sdk
pip install -e .                                  # so evals can import your app
evalgate init --local --format json               # preview; writes nothing
evalgate init --local --apply --format json       # write the scaffold
```

Confirm the installed version with `evalgate --version`.

## 2. Write real evals

The example gates a support agent's refund decision:

```python theme={null}
# refund_agent/__init__.py
def decide_refund(days_since_purchase, amount, reason, prior_refunds=0):
    if prior_refunds >= 2 or reason == "fraud_suspected":
        return {"action": "escalate", "refund": 0}
    if days_since_purchase > 30:
        return {"action": "deny", "refund": 0}
    if reason == "defective":
        return {"action": "approve", "refund": amount}
    return {"action": "approve", "refund": round(amount * 0.9, 2)}
```

Replace `evals/example_eval.py` with cases for your own code.

```python theme={null}
# evals/refund_policy_eval.py
import json

from evalgate_sdk import create_result, define_eval
from refund_agent import decide_refund


@define_eval("day 30 is still inside the window")
def day_thirty(ctx):
    decision = decide_refund(days_since_purchase=30, amount=20, reason="changed_mind")
    failures = []
    if decision["action"] != "approve":
        failures.append("expected action approve")
    if decision["refund"] != 18:
        failures.append("expected refund 18")
    return create_result(
        passed=not failures,
        score=0 if failures else 100,  # 0–100
        output=json.dumps(decision),
        metadata={"failures": failures},  # shown on a blocked gate
    )
```

* Both `@define_eval("name")` and `define_eval("name", fn)` work.
* **Names are literal strings.** Build cases with a helper, not a loop.
* Use ASCII names if the same specs might move to the TypeScript SDK.
* `evalgate discover --manifest` lists the specs. It exits `5` and reports
  `loadErrors` if an eval file fails to import.

## 3. Accept a baseline

```bash theme={null}
git add -A && git commit -m "add evals"
evalgate baseline update     # needs a clean worktree
git add evals/baseline.json && git commit -m "accept reviewed baseline"
```

## 4. Gate every change

Try it: change `days_since_purchase > 30` to `>= 30`, commit, and run the
gate. It exits `2` and shows `day 30 is still inside the window` with the
actual `{"action": "deny", "refund": 0}`.

```bash theme={null}
evalgate gate --format json
```

| Exit | Meaning |
| - | - |
| `0` | PASS: release-ready |
| `2` | REGRESSION: a reviewed behavior changed. The report lists `failedCases` with each case's actual output |
| `15` | Baseline not accepted yet |

When it blocks, run `evalgate explain --format json`. Fix the code; never
update the baseline to make a regression pass. In GitHub Actions, run
`pip install -e .`, `pip install evalgate-sdk`, then
`evalgate gate --format github --offline`. The local gate needs no API key.

Next: [Python SDK reference](https://evalgate.com/docs/sdk/python.md) or let your coding agent do this
with [Skills and MCP](https://evalgate.com/docs/guides/write-evals-with-your-agent.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.