> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent framework integrations

> Connect LangChain, CrewAI, AutoGen, Vercel AI SDK, LiteLLM, LlamaIndex, MCP, and OpenTelemetry-compatible agent runtimes to EvalGate.

# Agent framework integrations

EvalGate accepts quality evidence at three boundaries: an SDK wrapper around a
runtime call, an OpenTelemetry/OpenInference trace export, or an authenticated
MCP/API call. Pick the narrowest boundary your application already supports.

> [!WARNING]
>
> Hosted trace, MCP, and API calls require an EvalGate account, an organization,
> and a scoped API key. Your EvalGate key does not pay for model inference;
> model-backed workflows use a provider credential owned and billed by your
> organization.

## Official coding-agent Skills

Browse the [official EvalGate Skills repository](https://github.com/evalgate/skills)
for the portable evaluation decision framework and setup, gate, trace,
repository, and MCP workflows. Preview the collection with
`npx skills add evalgate/skills --list`, then install the complete collection
with `npx skills add evalgate/skills` or a single self-contained Skill with
`npx skills add evalgate/skills --skill evaluate-ai-change`. Skills do not
configure MCP servers.

The collection is experimental. Skills supply instructions; they do not issue
credentials or replace release-policy enforcement. See the
[coding-agent guide](https://www.evalgate.com/docs/guides/write-evals-with-your-agent)
and [agent discovery guide](https://www.evalgate.com/docs/guides/agent-discovery).

## Support matrix

| Runtime | Supported boundary | Package or endpoint | Current limitation |
| - | - | - | - |
| LangChain | TypeScript structural tracing wrapper and offline recipe | `traceLangChainAgent` from `@evalgate/sdk` | Wraps structural `invoke` or `call` methods; install and configure LangChain separately |
| CrewAI | TypeScript structural tracing wrapper | `traceCrewAI` from `@evalgate/sdk` | Wraps `kickoff`; install and configure CrewAI separately |
| AutoGen | TypeScript structural tracing wrapper | `traceAutoGen` from `@evalgate/sdk` | Wraps `initiate_chat`; install and configure AutoGen separately |
| Vercel AI SDK | Language Model v3-compatible model wrapper | `@evalgate/sdk/integrations/vercel-ai` | Trace collection is fail-open and does not replace provider policy |
| LiteLLM | OpenAI-compatible Model Gateway backend | Customer-configured LiteLLM endpoint | Gateway routing path, not a dedicated framework tracing wrapper |
| LlamaIndex | OpenTelemetry/OpenInference trace export | OTLP/HTTP | No first-class EvalGate wrapper is advertised today |
| MCP clients | Streamable HTTP | `/api/mcp` and `/api/mcp/docs` | Product tools use API-key scopes; the public docs server has no tenant context |
| DSPy and other Python runtimes | OpenTelemetry/OpenInference or explicit Python SDK trace calls | OTLP/HTTP and `evalgate-sdk` | No dedicated DSPy adapter is advertised today |

The framework packages remain your dependencies. EvalGate's structural wrappers
do not install them or take over their model-provider configuration.

Each adapter's readiness, limitations, and the evidence still needed for live
qualification are listed in [Integration adapters](https://evalgate.com/docs/guides/integration-adapters.md),
which also covers the unreleased Harness², GEPA, and DSPy candidates.

## CrewAI

Wrap the crew object after creating an authenticated client and workflow tracer:

```ts theme={null} theme={null}
import {
  AIEvalClient,
  createWorkflowTracer,
  traceCrewAI,
} from "@evalgate/sdk";

const evalgate = AIEvalClient.init();
const tracer = createWorkflowTracer(evalgate);
const tracedCrew = traceCrewAI(crew, tracer, { crewName: "ResearchCrew" });

const result = await tracedCrew.kickoff({ topic: "release evidence" });
```

The wrapper records workflow and agent-span completion or failure around the
existing `kickoff` call. It does not change the crew's tools or provider keys.

## AutoGen

Use the corresponding conversation wrapper when your runtime exposes
`initiate_chat`:

```ts theme={null} theme={null}
import {
  AIEvalClient,
  createWorkflowTracer,
  traceAutoGen,
} from "@evalgate/sdk";

const evalgate = AIEvalClient.init();
const tracer = createWorkflowTracer(evalgate);
const tracedConversation = traceAutoGen(conversation, tracer, {
  conversationName: "ReleaseReview",
});

await tracedConversation.initiate_chat(reviewer, { message: "Review this change" });
```

## Vercel AI SDK

The Vercel integration wraps a Language Model v3-compatible object without
adding the `ai` package as an EvalGate runtime dependency:

```ts theme={null} theme={null}
import { traceVercelAIModel } from "@evalgate/sdk/integrations/vercel-ai";
import { generateText } from "ai";

const tracedModel = traceVercelAIModel(model, evalgate, {
  captureInput: true,
  captureOutput: true,
  environment: "staging",
});

const result = await generateText({
  model: tracedModel,
  prompt: "Summarize the release evidence.",
});
```

Trace-delivery failures do not mask the model result. If trace persistence is a
release requirement, verify the trace separately before promoting a baseline.

## MCP product and documentation servers

Use the public server cards to inspect the tools before connecting:

* Product card: `https://www.evalgate.com/.well-known/mcp/server-card.json`
* Documentation card: `https://www.evalgate.com/.well-known/mcp/docs-server-card.json`
* Product transport: `https://www.evalgate.com/api/mcp`
* Documentation transport: `https://www.evalgate.com/api/mcp/docs`

The documentation server is public, read-only, and limited to the reviewed
published-doc catalog. Product tools are filtered by the scopes on the
organization API key. See [MCP integration](https://evalgate.com/docs/platform/mcp-integration.md)
and [agent authentication](https://evalgate.com/docs/api/agent-authentication.md).

## DSPy and other OpenTelemetry-compatible runtimes

EvalGate does not currently ship a first-class DSPy wrapper. Use an existing
OpenTelemetry or OpenInference instrumentation path to export trace evidence, or
call the Python SDK explicitly at the application boundary. Keep model calls and
tool side effects in your application; EvalGate receives the trace and evaluation
evidence you choose to send.

See [TypeScript SDK: OTLP/OpenInference](https://evalgate.com/docs/sdk/typescript.md)
and [trace setup](https://evalgate.com/docs/guides/tracing-setup.md). Do not label a generic trace export as a
dedicated framework integration in architecture or procurement documents.

## Verify the integration

Before relying on a framework path for a release gate:

1. Send one successful and one failing run from a non-production environment.
2. Confirm workflow, agent, model, tool, and error fields preserve the evidence
   needed by your assertions.
3. Turn one reviewed failure into a permanent evaluation case.
4. Run the gate against the exact commit and baseline used for the release.
5. Confirm provider, budget, parser, and missing-trace failures remain explicit.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.