> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Integration adapters

> The evalgate.integration.v1 contract, adapter manifests and readiness, and the Harness², GEPA, DSPy, and OpenInference adapters.

# Integration adapters

EvalGate connects to agent ecosystems through adapters that translate a
source's own records into one canonical event contract,
`evalgate.integration.v1`. Adapters do not run your agents or replace their
frameworks. They report what happened, what was scored, and what a source
claims, and they keep those three things apart.

> [!WARNING]
>
> The Harness², GEPA, and DSPy adapters and the `/api/integrations/events`
> endpoint are an **unreleased candidate**. They are in the source tree and
> tested against fixtures, but they are not in the published `evalgate-sdk`
> 3.14.1, and none of them is qualified against a live third-party run yet.

## Five stances, never one `passed` field

Every event declares exactly one stance:

| Stance | Meaning | Example |
| - | - | - |
| `observed` | The source saw it happen | A Harness² task run finished |
| `evaluated` | A judge or metric scored it | A judged candidate harness, a GEPA valset score |
| `inferred` | A conclusion drawn from evaluations | A probe comparison, a lesson, a source-side adoption |
| `authorized` | A claim that an action may proceed | Recorded as a pending approval |
| `promoted` | A claim that an artifact should become active | Recorded as a pending approval |

An observed event cannot assert an outcome, an evaluated event must carry a
metric or an outcome, and authorized or promoted events must state the claim
they ask EvalGate to approve. EvalGate never applies an external adapter's
authorization or promotion; it records a pending human approval.

## Ingestion guarantees

`POST /api/integrations/events` accepts up to 100 events and requires an API
key with `traces:write`.

* **Explicit approval for writes.** A batch is validated and lowered but not
  persisted unless it sets `"dryRun": false`. The SDK adapters are plan-only
  by default as well.
* **Idempotency.** Each event's key is
  `sha256(ecosystem, adapterId, eventType, nativeEventId)`. A replay returns
  `replayed`; the same key with different content returns `conflict`. The
  adapter version and `occurredAt` are not part of the conflict hash, so
  re-importing with an approximated time still replays.
* **Tenant isolation.** The organization comes only from the API key. An
  event that carries `organizationId` is rejected.
* **Redaction and provenance.** Events are stored through the agent-evidence
  ingest, which redacts payloads, evidence previews, and approval reasons
  server-side regardless of client claims, scans every other stored value for
  credentials, and keeps the source schema, adapter version, and parent event
  for each record.
* **Manifest enforcement.** An event is accepted only from a registered
  adapter, for an event type that adapter declares, with the stance the
  manifest declares for it.

Each event returns one result: `validated`, `created`, `replayed`,
`conflict`, or `rejected` with a reason. One invalid event does not fail the
batch.

## Adapter manifests and readiness

Every adapter has a manifest in the adapter registry
(`evalgate.integration-manifest.v1`) that declares its ecosystem, version, transport, capabilities (observe,
evaluate, enforce), authentication, supported events, source schema,
canonical mapping, replay behavior, limitations, test fixtures, and readiness.

Readiness is one of `planned`, `fixture_qualified`, `live_qualified`, or
`production_supported`. The schema rejects `live_qualified` or
`production_supported` without a recorded live qualification, and rejects
`enforce` for any adapter that only sees an action after it ran (OTLP
telemetry and file imports).

| Adapter | Transport | Readiness | Released | Live qualification needed |
| - | - | - | - | - |
| `openinference-otlp` | OTLP/HTTP | fixture\_qualified | yes | A live export from an `openinference-instrumentation-*` package |
| `otel-genai-otlp` | OTLP/HTTP | fixture\_qualified | yes | A live GenAI-instrumented export |
| `langchain-sdk-wrapper` | SDK | fixture\_qualified | yes | A live LangChain run |
| `crewai-sdk-wrapper` | SDK | fixture\_qualified | yes | A live crew run with a model key |
| `autogen-sdk-wrapper` | SDK | fixture\_qualified | yes | Release of AgentChat `run()` support, then a live run with a model key |
| `vercel-ai-model-wrapper` | SDK | fixture\_qualified | yes | A live provider run |
| `native-harness-exports` | File import | fixture\_qualified | yes | A client that writes the export natively |
| `harness2-runs` | File import | fixture\_qualified | no | A Harness² sweep imported into a staging organization |
| `gepa-optimize-anything` | SDK | fixture\_qualified | no | `optimize_anything` with a reflection LM key |
| `dspy-gepa-metric` | SDK | fixture\_qualified | no | `dspy.GEPA` with an LM key |

No adapter claims `enforce`. AutoGen tracing covers AgentChat 0.4+ `run()`
and `run_stream()` on agents and teams in source; the published
`evalgate-sdk` 3.14.1 traces only the legacy `initiate_chat` API.

## Harness²

The `harness2-runs` adapter imports a finished
[Harness²](https://github.com/google-research/harness2) namespace directory.
It reads task summaries, `judged.json`, `base_judged.json`, selection
records, and adoption records, and never runs Harness² or reads harness trees,
transcripts, or deliverables.

```bash theme={null}
# Plan only: writes .evalgate/integrations/harness2-<namespace>.jsonl
evalgate harness import-harness2 "$HARNESS2_RUNS_DIR/jb-oc-agg2-s7"

# Ask EvalGate to validate without writing
evalgate harness import-harness2 "$HARNESS2_RUNS_DIR/jb-oc-agg2-s7" --send

# Write events and metadata-only harness outcomes
evalgate harness import-harness2 "$HARNESS2_RUNS_DIR/jb-oc-agg2-s7" --apply
```

| Harness² artifact | Event | Stance |
| - | - | - |
| `summary.json` mode result and cost ledger | `harness2.task_run.completed.v1` | observed |
| `judged.json` / `base_judged.json` row | `harness2.candidate.judged.v1` | evaluated |
| Selection among the k candidates | `harness2.probe_comparison.selected.v1` | inferred |
| Selection experience (cue, lesson, evidence) | `harness2.lesson.recorded.v1` | inferred |
| `adoption.json` | `harness2.harness.adopted.v1` | inferred |
| `--request-promotion` | `harness2.harness.promotion_requested.v1` | promoted |

A judge failure is recorded as an `error` outcome, never as a zero score.
Cloud project, location, and endpoint identifiers from `config.json` are
replaced by SHA-256 digests. Adoption is Harness²'s own decision; EvalGate
promotion requires `--request-promotion` and a human approval. With
`--apply`, judged task results also go to `/api/harness/outcomes` as
metadata-only harness outcomes (score and cost; Harness² does not record
token counts outside its substrate logs).

`--send` and `--apply` fail closed. `--apply` first validates every event and
writes nothing if any would be rejected; if a write then reports a conflict,
the command stops and uploads no harness outcomes. Either way it lists each
failed event and exits non-zero. Re-running after a fix is safe because
accepted events replay.

The mapping is derived from Harness² source at commit `1cfe254`.

## GEPA and DSPy

`EvalGateMetric` combines EvalGate evaluators and assertion helpers into one
score plus feedback text, and the adapters hand it to the optimizer:

```python theme={null}
import gepa.optimize_anything as oa
from evalgate_sdk.assertions import contains_keywords
from evalgate_sdk.integrations.gepa import EvalGateMetric, GEPACandidateRecorder, gepa_evaluator

metric = EvalGateMetric({
    "keywords": lambda outputs, **_: contains_keywords(outputs, ["refund", "policy"]),
})
recorder = GEPACandidateRecorder()  # local JSONL; nothing is sent yet
result = oa.optimize_anything(
    seed_candidate="Answer the customer.",
    evaluator=gepa_evaluator(lambda candidate, example: run_agent(candidate, example), metric),
    config=oa.GEPAConfig(callbacks=[recorder]),
)
recorder.send(api_key=EVALGATE_API_KEY, base_url="https://www.evalgate.com", apply=False)  # validate
```

* `recorder.send(...)` returns `ok` and a `failures` list; with
  `apply=True` it validates every event first and writes nothing if any would
  be rejected.
* `gepa_evaluator` returns `(score, side_info)`, so EvalGate feedback reaches
  GEPA's reflection step.
* `dspy_feedback_metric(metric)` returns `dspy.Prediction(score=..., feedback=...)`,
  which `dspy.GEPA` reads as score-with-feedback and `dspy.Evaluate` sums.
* `GEPACandidateRecorder` records candidate evaluations, acceptances,
  rejections, merges, and the best candidate with parent lineage. Pass it to
  `GEPAConfig(callbacks=[...])` or `dspy.GEPA(gepa_kwargs={"callbacks": [...]})`.
  Candidate text is hashed unless `capture_content=True`.
  `recorder.request_promotion(idx)` records a pending promotion claim.

Install the optional extras with `pip install "evalgate-sdk[gepa]"` or
`"evalgate-sdk[dspy]"` once released.

## OpenInference

OTLP spans that follow
[OpenInference semantic conventions](https://github.com/Arize-ai/openinference/blob/main/spec/semantic_conventions.md)
keep these fields on ingest through `/api/otel/v1/traces`:

* span kind, `input.value` / `output.value`, and flattened
  `llm.input_messages.*` / `llm.output_messages.*` when no value is set;
* model, provider (`llm.provider` or `llm.system`), prompt, completion, and
  total tokens, `llm.cost.total`, and the JSON `llm.invocation_parameters`;
* `session.id`. `user.id`, whether sent as a span or a resource attribute,
  is stored only as a SHA-256 digest.

Values hidden by the producer (`OPENINFERENCE_HIDE_*` settings write
`__REDACTED__`) are recorded under `evalgate.openinference.hidden` and never
stored as span content.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.