> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Red-Team Workbench

> Review versioned attack evidence, execute immutable adversarial campaigns, adjudicate findings, and prove remediation with golden coverage and signed reports.

# Red-Team Workbench

Red-Team Workbench is EvalGate's organization-scoped control plane for adversarial evaluation. It turns a safety test from a collection of prompt strings into a reproducible evidence chain: reviewed risk-pack version, exact attack cases, immutable target and scorer versions, Model Gateway calls, stable findings, human decisions, remediation artifacts, a fixed-target rerun, active golden coverage, a release gate, and a signed report.

Open the workbench at `/red-team`.

> [!NOTE]
>
> A campaign never resolves “latest.” Its snapshot stores the target artifact ID, exact version ID, SHA-256 content hash, approved risk-pack and case hashes, variants, scorer bindings, optional dataset version, budget, concurrency, and seed. Approval and execution revalidate those identities.

## Permissions

| Action | Required scope | Default role access |
| - | - | - |
| Read packs, campaigns, runs, findings, decisions, and evidence | `red_team:read` | Viewer, member, admin, owner |
| Generate proposed cases, create campaigns/versions, run, cancel, rerun, compare | `red_team:run` | Member, admin, owner |
| Review cases/packs, approve campaign versions, transition findings, link remediation, promote regressions, sign reports | `red_team:approve` | Admin and owner |

API keys receive only the scopes explicitly granted to them. All database reads and writes are organization-scoped. Composite references are revalidated in the service and database, so a same-ID resource from another organization is rejected rather than disclosed.

## Evidence model

| Object | Immutable evidence | Mutable lifecycle fields |
| - | - | - |
| Risk-pack version | Taxonomy/version, pack snapshot, artifact/version IDs, content hash | Draft review to approved or rejected |
| Attack case | Category, technique, redacted input, expected safe behavior, indicators, severity, source, content hash, generated model-call ID | Proposed review to approved or rejected |
| Campaign version | Exact target/version/hash, cases, variants, scorers, dataset, budget, seed, snapshot hash | Draft approval; an older approved version may become superseded |
| Campaign result | Execution key, response hash/preview, scorer/model-call evidence, cost, finding fingerprint | None; append-only |
| Finding decision | From/to state, rationale, evidence, actor, decision time | None; append-only |
| Remediation link | Resource identity/version, server-verified evidence hash, actor | None; append-only |
| Run comparison | Baseline/candidate runs and content-hashed comparison | None; append-only |

Risk packs and campaign versions are backed by the normal release-artifact/version registry. Their content remains addressable after a newer version is created.

## Create an assessment from risk

The current workbench retains its governed risk-pack and campaign builders while the persisted assessment-plan contract is developed. The intended risk-first sequence begins with an exact published target and a plain-language failure mode, then reviews applicable defensive controls, evidence methods, bounded execution depth, and the complete plan. EvalGate does not expose that sequence as a primary production route until it can persist the plan through a shared API without executing provider calls or mutating the target.

The control-first foundation is available in the **AIDEFEND defense library**. AIDEFEND is a defensive-control reference and remediation framework, not an attack engine. EvalGate owns the attack scenarios, assessment planning, immutable campaigns, evidence, findings, remediation proof, regression gates, release decisions, and reports. The working model is:

```text theme={null}
risk or failure mode → attack scenario → applicable defense → verification plan
→ immutable campaign → evidence → finding → remediation proof → regression gate
```

Parent families are browsable organizers and may summarize child coverage, but cannot be assessed or receive independent credit. Standalone techniques and leaf sub-techniques are actionable. A control without a reviewed EvalGate verification profile remains an evidence obligation labelled **Verification design required**; EvalGate does not fabricate a generic prompt. Supported methods are adversarial probe, deterministic assertion, configuration review, trace/telemetry requirement, provenance or artifact review, resilience exercise, and human evidence review. A control may require several methods, and attack resistance remains separate from benign utility.

> [!WARNING]
>
> A mapping does not prove implementation. Missing evidence is not a pass. EvalGate never presents AIDEFEND compliance, certification, approval, endorsement, partnership, or affiliation.

## Advanced risk-pack implementation objects

Risk packs and campaigns remain the governed implementation objects behind the workbench. Advanced users can create a structured risk-pack draft with stable case keys, supported risk categories, explicit safe behavior and success indicators. Every framework-derived or model-generated suggestion remains proposed, editable, and human-reviewed before approval.

### Generate proposed cases

`POST /api/red-team/attack-cases/generate` sends the reviewed objective, category, taxonomy, count, and explicit provider/model intent through the canonical Model Gateway with the `synthesis` use case. It accepts only strict JSON, rejects cases outside the requested category, records the model-call ID, redacts content, and stores the candidates as `proposed`.

Generated content is never auto-approved. An admin or owner must inspect the exact input, safe behavior, indicators, severity, source, and hash before choosing **Approve case** or rejecting it with a rationale.

The pack can be approved only after every included case is approved. Approval transitions the backing release version to a candidate and freezes the risk-pack review state.

> [!WARNING]
>
> Edit a case by creating a new risk-pack version. Reviewed case content and historical risk-pack snapshots cannot be rewritten or deleted.

## Create an immutable campaign

Choose **New campaign** and bind:

1. Target type, release artifact ID, exact version ID, and content hash.
2. One or more approved risk-pack version IDs.
3. One or more execution variants with stable keys, configured Gateway models/routing profiles, and parameters.
4. Exact scorer artifact/version/hash bindings when scoring is required.
5. An optional immutable dataset-version ID.
6. Maximum cost, maximum model calls, concurrency, and deterministic seed.
7. A unique idempotency key.

The service expands the cross-product of approved cases and variants into stable execution keys. A reused campaign idempotency key returns the original campaign only when the complete immutable intent matches; different intent returns a conflict.

### Approve the snapshot

**Approve** reruns target, pack, case, and scorer validation before storing the actor, rationale, and time. Creating a draft does not supersede the currently approved version. Only successful approval supersedes the prior approved version.

The approval dialog identifies that the complete target/version/hash, attack evidence, variants, scorers, dataset, budget, concurrency, and seed are being frozen.

## Execute within reviewed bounds

Choose **Run** on an approved campaign version. The request repeats the accepted maximum cost and concurrency. Execution is refused when:

* The version is not human-approved.
* Target or scorer content no longer matches the approved hash.
* Expansion exceeds the reviewed model-call ceiling.
* The caller does not explicitly accept the approved cost or concurrency bound.
* A rerun baseline is incomplete, outside the organization, or belongs to another campaign.

Every provider/model request goes through the Model Gateway with `eval_executor`. The result retains the model-call ID, response hash, redacted preview, target/case/scorer hashes, provider status, cost source, and error code.

Calls execute at the snapshot's bounded concurrency. The worker stops scheduling new work when cancellation is requested, known actual cost exceeds the ceiling, or provider cost is unavailable. Unknown cost fails closed with `COST_UNAVAILABLE`; known overage fails with `BUDGET_EXCEEDED`. Provider failures remain explicit failed evidence rather than successful safe results.

### Cancel, rerun, and compare

**Cancel** requires a human rationale. In-flight calls may finish and remain durable; no new work is scheduled after the cancellation flag is observed.

**Rerun** uses a completed baseline from the same campaign and an approved campaign version, normally one that binds the fixed target. The workbench automatically requests a baseline/candidate comparison after a completed rerun.

Comparison reports:

* Baseline and candidate attack-success rate
* Rate delta
* Resolved fingerprints
* Newly introduced fingerprints
* Persisting fingerprints
* Regression state

## Compile release evidence

The release-admission artifact is compiled from one organization-scoped,
completed campaign run. It is not a second execution path and it does not
discover a run by recency. Fetch the run you intend to use:

```bash theme={null} theme={null}
evalgate red-team evidence --run <completed-run-id>
```

The command exists in both the TypeScript and Python CLIs and calls the same
`get_red_team_runs_id_release_evidence` operation exposed by both generated
clients. It validates the strict schema and canonical hash, verifies the
server's Ed25519 signature against the repository-pinned public-key
fingerprint, checks the configured organization, snapshot, and exact target,
then atomically writes `.evalgate/red-team-evidence.json`. The response contains the exact
artifact/version/content-hash identity, campaign snapshot and run identity,
execution completeness, provider state, attack and benign-utility outcomes,
finding governance, remediation comparison, active golden coverage, signed
report identity, framework references, and a content hash for the evidence
envelope. It contains decision metadata rather than raw prompts or model
outputs.

Pin the target the gate is deciding on in repository configuration. For
example:

```json theme={null} theme={null}
{
  "redTeam": {
    "required": true,
    "policyProfile": "pre_release_strict",
    "exactTargetRequired": true,
    "expectedTarget": {
      "artifactId": "42",
      "versionId": "7",
      "contentHash": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
    },
    "expectedOrganizationId": "org_123",
    "expectedSnapshotHash": "abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789",
    "expectedAttestationKeyId": "red-team-prod-v1",
    "expectedAttestationPublicKeySha256": "fedcba9876543210fedcba9876543210fedcba9876543210fedcba9876543210"
  }
}
```

For a required gate, `pre_release_strict` is the default profile. It requires
compiler attestation, a complete and fresh campaign, attributable costs, zero
successful attacks, and a 100% benign-utility pass rate. It does not require a
finding-scoped report, so a clean first campaign can pass. Use
`post_remediation_strict` for a repaired finding; that profile additionally
requires the baseline/fixed rerun comparison, no introduced fingerprints,
active golden coverage, and an exact-hash signed report. The legacy `strict`
name remains an alias for `pre_release_strict`; use `custom` only for a reviewed
exception. Both strict lifecycle profiles require exact organization, target,
campaign-snapshot, and compiler-key pins. The control plane
must configure the server-only `RED_TEAM_EVIDENCE_SIGNING_PRIVATE_KEY` and
`RED_TEAM_EVIDENCE_SIGNING_KEY_ID`; CI receives no signing secret.

Use the artifact and version identifiers from the release registry, not a
source checkout digest. A required gate fails closed when the evidence is
missing, malformed, stale, cross-organization, bound to another target or
snapshot, incomplete, provider-failed, cost-unattributed, or has an invalid
evidence hash or compiler signature. Hand-authored JSON and a convenient
“latest run” lookup are not
admissible substitutes. Campaign creation and execution remain explicit
human-approved actions; compiling evidence only reads durable state.

In a fresh CI checkout, fetch the exact artifact before the normal gate:

```bash theme={null} theme={null}
evalgate red-team evidence --run "$RED_TEAM_RUN_ID" --format json
evalgate gate --format json
```

## Findings and stable dedupe

A successful attack creates or updates a campaign-scoped finding. Its fingerprint contains organization, exact target version, case content hash, category, variant, and normalized evidence signature. The same governed failure recurs into one finding; evidence from another campaign cannot become its latest result.

Each new finding result occurrence also records an explicit source state. When
the campaign target is linked to Repository Intelligence, the occurrence binds
the organization-owned repository, exact commit snapshot, graph version, and
one or more typed locators (`test_target`, `observed_execution`, or
`suspected_contributor`). Node identity and revision are included only when the
locator is verified as evidence for that graph member. These relationships are
observations, not automatic causal attribution. Legacy, inaccessible, and
black-box targets remain explicitly `unlinked`, `unavailable`, or
`not_applicable`; EvalGate never substitutes today's repository HEAD.

Read the same evidence used by the finding detail UI through either generated
client or the TypeScript/Python source CLI:

```bash theme={null} theme={null}
evalgate red-team finding --finding <finding-id> --format json
```

Linked responses include commit-pinned GitHub and exact-graph navigation. The
original observation remains distinct from later repeat, rerun, and remediation
source contexts.

Severity is rank-preserving (`critical` > `high` > `medium` > `low`) rather than lexicographic. Confidence retains the maximum supported value. High and critical findings always require an explicit owner.

The finding lifecycle is:

```text theme={null}
open → triaged → confirmed → remediating → resolved
  └────────────→ false_positive
triaged/confirmed/remediating → accepted_risk
resolved/accepted_risk → remediating
```

Every transition requires a fresh append-only decision created after the current state began. The decision records actor, rationale, structured evidence, and from/to state in the same transaction as the state change. Historical decisions cannot be replayed after a finding is reopened.

Confirming a finding creates or reuses an organization-scoped adjudication item and links it to the finding. High/critical accepted risk requires an explicit owner and a verified release-gate record.

## Prove the remediation chain

A resolved finding requires seven distinct verified links:

| Link | Server verification |
| - | - |
| Playground | Completed immutable Playground run snapshot in the same organization |
| Experiment | Completed immutable experiment snapshot and hash |
| Candidate case | Durable reviewed candidate with provenance and dedupe hash |
| Golden case | Active `active_gate` regression case in the target evaluation |
| Release gate | Completed organization-scoped evaluation run |
| Rerun | Completed rerun of the same campaign, with zero successful attacks |
| Signed report | Active organization-scoped shared report signed by the current report key |

The reviewer supplies the durable resource ID. EvalGate reads the resource, verifies organization and lifecycle state, derives server evidence, hashes the complete evidence envelope, and appends the link. Caller-supplied metadata cannot make an incomplete resource verified.

### Promote a finding to golden coverage

Enter the target evaluation ID and choose **Promote**. EvalGate:

1. Reads the latest immutable attack/result/campaign evidence.
2. Creates a deterministic candidate with source run, failure mode, fingerprint cluster, exact target version, deterministic generator version, and dedupe hash.
3. Records a normal synthetic human-review decision.
4. Promotes through the canonical candidate-to-golden lifecycle.
5. Appends candidate and active-golden remediation links.

The flow is available only for confirmed or remediating findings. It never silently auto-promotes generated attack content.

### Sign the evidence report

After Playground, Experiment, Candidate, Golden, Release gate, and Rerun are verified, enter the evaluation and release-gate run IDs and choose **Sign report**.

The report includes:

* Finding ID, fingerprint, severity, and state
* Exact fixed target artifact/version/content hash and campaign snapshot hash
* Every remediation resource/version and evidence hash
* Baseline/rerun comparison
* Active golden-case and release-gate identity
* Signing algorithm, key version, audience, scope, replay policy, creation time, and optional expiry

Report persistence and signing-key selection occur transactionally. The signed report becomes the seventh verified link.

### Resolve

Choose **Resolve**, enter the decision rationale, and confirm. Both the service and database reject resolution unless all seven link types are verified and the linked campaign rerun contains zero successful attacks. Reopening clears the resolution timestamp but retains all historical decisions and evidence.

## Versioned framework references and attribution

The active AIDEFEND defense-library snapshot records framework/data version, public schema version, generation time, source URL and commit, SHA-256 source digest, computed parent/standalone/leaf/actionable/total counts, license, attribution, and the EvalGate change notice. A reviewed framework-derived risk-pack case also persists the complete control reference, including canonical ID/title, hierarchy, relationship, mapping source and rationale, and review state, in that immutable risk-pack snapshot.

Persisting that edition-qualified reference through assessment plans, campaign results, findings, remediation, regression gates, and signed reports remains tracked platform work. The current catalog does not pin migration-registry identity or digest, and the current OWASP and MITRE mappings remain derived display strings rather than reviewed structured references. Exact-target control-evidence states and semantic edition migration therefore remain intended assessment-contract behavior, not an end-to-end persistence claim.

AIDEFEND AI Defense Framework, created by Edward Lee, [https://aidefend.net](https://aidefend.net), licensed under CC BY 4.0. EvalGate classifies catalog nodes, summarizes purposes, and adds reviewed mappings and verification metadata; these are EvalGate modifications and adaptations. No logo or badge is used.

## API reference

| Method | Endpoint | Purpose |
| - | - | - |
| `GET` | `/api/red-team/workspace` | Read packs, campaigns, versions, runs, and findings |
| `GET`, `POST` | `/api/red-team/risk-packs` | List or create versioned risk packs |
| `POST` | `/api/red-team/attack-cases/generate` | Generate Model Gateway proposals |
| `POST` | `/api/red-team/attack-cases/{id}/review` | Approve or reject one proposed case |
| `POST` | `/api/red-team/risk-packs/{id}/review` | Approve or reject a pack version |
| `POST` | `/api/red-team/campaigns` | Create campaign and immutable draft version |
| `POST` | `/api/red-team/campaigns/{id}/versions` | Create a new immutable draft |
| `POST` | `/api/red-team/campaigns/{id}/versions/{versionId}/approve` | Revalidate and approve a snapshot |
| `POST` | `/api/red-team/campaigns/{id}/runs` | Execute or rerun through Model Gateway |
| `GET` | `/api/red-team/runs/{id}` | Read run and append-only results |
| `GET` | `/api/red-team/runs/{id}/release-evidence` | Compile canonical release-admission evidence for one run |
| `POST` | `/api/red-team/runs/{id}/cancel` | Request attributable cancellation |
| `POST` | `/api/red-team/runs/{id}/compare` | Persist baseline/candidate comparison |
| `GET` | `/api/red-team/findings/{id}` | Read finding evidence and decisions |
| `POST` | `/api/red-team/findings/{id}/transition` | Append a human decision and change state |
| `POST` | `/api/red-team/findings/{id}/remediation-links` | Verify and append downstream evidence |
| `POST` | `/api/red-team/findings/{id}/promote-regression` | Create reviewed candidate and golden coverage |
| `POST` | `/api/red-team/findings/{id}/signed-report` | Sign the completed remediation evidence |

## Failure behavior

| Condition | Response or run state |
| - | - |
| Referenced resource is missing or outside the organization | `404 NOT_FOUND` without cross-org disclosure |
| Invalid body, category, lifecycle, or evidence type | `400 VALIDATION_ERROR` |
| Idempotency key reused for different intent | `409 CONFLICT` |
| Target/scorer hash changed before approval or execution | `409 CONFLICT` with stale-target detail |
| Model-call plan or actual cost exceeds reviewed budget | `QUOTA_EXCEEDED` before start or failed run with `BUDGET_EXCEEDED` |
| Provider omits cost | Failed run with `COST_UNAVAILABLE` |
| Gateway blocks/fails a call | Failed run with `PROVIDER_FAILURE` and model-call evidence |
| Finding decision is stale or transition invalid | `409 CONFLICT` or database rejection |
| Remediation chain is incomplete | `409 CONFLICT` with missing evidence list |
| Severe accepted risk lacks release gate or owner | `409 CONFLICT` / validation rejection |

## Troubleshooting

* If campaign approval reports stale evidence, reload the exact target, pack, cases, and scorers; create a new campaign version for intentional changes.
* If execution is denied by budget, reduce case/variant expansion or approve a new campaign version with the intended bounded ceiling.
* If a run fails with unknown cost, inspect the Model Gateway ledger and configure pricing before rerunning. EvalGate does not treat unknown cost as zero.
* If promotion fails, confirm the finding and verify that the target evaluation belongs to the same organization.
* If signing fails, verify all six prerequisite links, the active golden lifecycle, and that the evaluation run exactly matches the release-gate link.
* If resolution is blocked, inspect the seven remediation cards and the candidate rerun's successful-attack count.

The implementation is covered by red-team contract/unit tests, route tests, real-Postgres immutability and organization-lineage tests, DOM interaction tests, and a non-skipped authenticated golden browser path under `tests/unit/red-team`, `tests/api/red-team`, `tests/integration/red-team`, `tests/dom/red-team`, and `e2e/red-team-golden-path.spec.ts`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.