> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Openai integration

# Trace OpenAI API calls with EvalGate

> Wrap the OpenAI client to automatically capture prompts, responses, token usage, and latency for every chat completion request you make.

The EvalGate SDK integrates with the OpenAI client through a thin wrapper that intercepts `chat.completions.create` calls and records each one as a trace — no manual instrumentation required. Standard, streaming, and tool-calling chat completions are captured automatically; other OpenAI endpoints (such as embeddings) are not wrapped. This guide shows you how to set up the wrapper and use it for multi-turn conversations, model A/B testing, and error tracking.

## Install dependencies

```bash TypeScript theme={null} theme={null}
npm install openai @evalgate/sdk
```

```bash Python theme={null} theme={null}
pip install openai evalgate-sdk
```

## Wrap the OpenAI client

Replace your existing OpenAI client initialization with the traced version. All subsequent calls through this client are automatically recorded:

```typescript TypeScript theme={null} theme={null}
import OpenAI from 'openai'
import { AIEvalClient, WorkflowTracer, traceOpenAI, traceWorkflowStep } from '@evalgate/sdk'

const client = new AIEvalClient({
  apiKey: process.env.EVALGATE_API_KEY
})

const tracer = new WorkflowTracer(client)

// Wrap the OpenAI client — all calls are now automatically traced
const openai = traceOpenAI(new OpenAI(), client)
```

```python Python theme={null} theme={null}
import os
from openai import AsyncOpenAI
from evalgate_sdk import AIEvalClient, WorkflowTracer
from evalgate_sdk.integrations.openai import trace_openai

client = AIEvalClient(api_key=os.environ["EVALGATE_API_KEY"])
tracer = WorkflowTracer(client)

# Wrap the OpenAI client — all calls are now automatically traced
openai = trace_openai(AsyncOpenAI(), client)
```

> [!NOTE]
>
> Make sure `EVALGATE_API_KEY` and `EVALGATE_ORGANIZATION_ID` are set in your `.env` file before initializing the client.

## Chat completions

Call the OpenAI API exactly as you normally would. EvalGate captures the full interaction in the background:

```typescript TypeScript theme={null} theme={null}
// No extra instrumentation needed — the wrapped client handles it
const response = await openai.chat.completions.create({
  model: 'gpt-5.5',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is the capital of France?' }
  ],
  temperature: 0.7
})

console.log(response.choices[0].message.content)
```

```python Python theme={null} theme={null}
# No extra instrumentation needed — the wrapped client handles it
response = await openai.chat.completions.create(
    model="gpt-5.5",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"}
    ],
    temperature=0.7
)

print(response.choices[0].message.content)
```

Every chat completion automatically records: the input messages and response message, provider token usage, request duration, and model name and parameters.

## Streaming responses

Streaming works exactly the same way. The SDK buffers the full response as it streams so the complete output appears in the trace:

```typescript TypeScript theme={null} theme={null}
const stream = await openai.chat.completions.create({
  model: 'gpt-5.5',
  messages: messages,
  stream: true
})

let fullResponse = ''
for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content || ''
  fullResponse += content
  process.stdout.write(content)
}

// The full assembled response is automatically captured in the trace
```

```python Python theme={null} theme={null}
stream = await openai.chat.completions.create(
    model="gpt-5.5",
    messages=messages,
    stream=True
)

full_response = ""
async for chunk in stream:
    content = chunk.choices[0].delta.content or ""
    full_response += content
    print(content, end="", flush=True)

# The full assembled response is automatically captured in the trace
```

## Function calling

Function calls are traced like any other completion. The tool definitions, selected function, and arguments all appear in the trace:

```typescript TypeScript theme={null} theme={null}
const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get current weather for a location',
      parameters: {
        type: 'object',
        properties: {
          location: { type: 'string' }
        },
        required: ['location']
      }
    }
  }
]

const response = await openai.chat.completions.create({
  model: 'gpt-5.5',
  messages: messages,
  tools: tools,
  tool_choice: 'auto'
})

// Function calls are automatically tracked in the trace
```

## Multi-turn conversations

Use `WorkflowTracer` to group multiple turns of a conversation into a single workflow trace with per-turn spans:

```typescript TypeScript theme={null} theme={null}
import { traceWorkflowStep } from '@evalgate/sdk'

await tracer.startWorkflow('multi-turn-conversation', undefined, {
  sessionId: 'session_456'
})

const messages = []

// Turn 1
messages.push({ role: 'user', content: 'Hello!' })
const response1 = await traceWorkflowStep(tracer, 'turn-1', () =>
  openai.chat.completions.create({ model: 'gpt-5.5', messages })
)
messages.push(response1.choices[0].message)

// Turn 2
messages.push({ role: 'user', content: 'Tell me a joke' })
const response2 = await traceWorkflowStep(tracer, 'turn-2', () =>
  openai.chat.completions.create({ model: 'gpt-5.5', messages })
)
messages.push(response2.choices[0].message)

await tracer.endWorkflow({ status: 'success' })
```

## A/B testing models

Trace model variants with metadata so you can compare quality, latency, and cost across experiments in the dashboard:

```typescript TypeScript theme={null} theme={null}
async function abTestModels(prompt: string) {
  const variant = Math.random() < 0.5 ? 'gpt-5.5' : 'gpt-5-mini'

  await tracer.startWorkflow('model-ab-test', undefined, {
    variant,
    experimentId: 'gpt55-vs-gpt5mini'
  })

  const result = await traceWorkflowStep(tracer, 'llm-call', () =>
    openai.chat.completions.create({
      model: variant,
      messages: [{ role: 'user', content: prompt }]
    })
  )

  await tracer.endWorkflow({ status: 'success' })
  return result
}

// Analyze results in the dashboard to compare:
// - Quality scores
// - Latency
// - Cost per completion
// - User satisfaction signals
```

## What gets tracked automatically

| Field | Description |
| - | - |
| **Full prompt / response** | Every input message, including system prompts, and the reply |
| **Token usage** | The `usage` object OpenAI returns for the request |
| **Latency** | Total request duration |
| **Model** | Model name, temperature, and other safe parameters |
| **Status** | Success or error, including cancelled streams |

## Troubleshooting

**Traces not capturing token usage?**

Token usage comes from the `usage` object OpenAI returns. For streaming requests, OpenAI only sends usage when you pass `stream_options: { include_usage: true }`. Also make sure you are on the latest `@evalgate/sdk` or `evalgate-sdk`.

**Noticing high latency in traces?**

Make sure you are using `async/await` consistently. Synchronous operations block the event loop and inflate latency measurements.

**Missing streaming response data?**

The SDK automatically buffers streaming responses. If data appears missing, confirm the stream iterator is fully consumed (the `for await` loop completes) before the process exits.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.