Trace OpenAI API calls with EvalGate

Wrap the OpenAI client to automatically capture prompts, responses, token usage, and latency for every chat completion request you make.

The EvalGate SDK integrates with the OpenAI client through a thin wrapper that intercepts chat.completions.create calls and records each one as a trace — no manual instrumentation required. Standard, streaming, and tool-calling chat completions are captured automatically; other OpenAI endpoints (such as embeddings) are not wrapped. This guide shows you how to set up the wrapper and use it for multi-turn conversations, model A/B testing, and error tracking.

Install dependencies

npm install openai @evalgate/sdk

pip install openai evalgate-sdk

Wrap the OpenAI client

Replace your existing OpenAI client initialization with the traced version. All subsequent calls through this client are automatically recorded:

import OpenAI from 'openai'
import { AIEvalClient, WorkflowTracer, traceOpenAI, traceWorkflowStep } from '@evalgate/sdk'

const client = new AIEvalClient({
  apiKey: process.env.EVALGATE_API_KEY
})

const tracer = new WorkflowTracer(client)

// Wrap the OpenAI client — all calls are now automatically traced
const openai = traceOpenAI(new OpenAI(), client)

import os
from openai import AsyncOpenAI
from evalgate_sdk import AIEvalClient, WorkflowTracer
from evalgate_sdk.integrations.openai import trace_openai

client = AIEvalClient(api_key=os.environ["EVALGATE_API_KEY"])
tracer = WorkflowTracer(client)

# Wrap the OpenAI client — all calls are now automatically traced
openai = trace_openai(AsyncOpenAI(), client)

Make sure EVALGATE_API_KEY and EVALGATE_ORGANIZATION_ID are set in your .env file before initializing the client.

Chat completions

Call the OpenAI API exactly as you normally would. EvalGate captures the full interaction in the background:

// No extra instrumentation needed — the wrapped client handles it
const response = await openai.chat.completions.create({
  model: 'gpt-5.5',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: 'What is the capital of France?' }
  ],
  temperature: 0.7
})

console.log(response.choices[0].message.content)

# No extra instrumentation needed — the wrapped client handles it
response = await openai.chat.completions.create(
    model="gpt-5.5",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is the capital of France?"}
    ],
    temperature=0.7
)

print(response.choices[0].message.content)

Every chat completion automatically records: the input messages and response message, provider token usage, request duration, and model name and parameters.

Streaming responses

Streaming works exactly the same way. The SDK buffers the full response as it streams so the complete output appears in the trace:

const stream = await openai.chat.completions.create({
  model: 'gpt-5.5',
  messages: messages,
  stream: true
})

let fullResponse = ''
for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content || ''
  fullResponse += content
  process.stdout.write(content)
}

// The full assembled response is automatically captured in the trace

stream = await openai.chat.completions.create(
    model="gpt-5.5",
    messages=messages,
    stream=True
)

full_response = ""
async for chunk in stream:
    content = chunk.choices[0].delta.content or ""
    full_response += content
    print(content, end="", flush=True)

# The full assembled response is automatically captured in the trace

Function calling

Function calls are traced like any other completion. The tool definitions, selected function, and arguments all appear in the trace:

TypeScript

const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Get current weather for a location',
      parameters: {
        type: 'object',
        properties: {
          location: { type: 'string' }
        },
        required: ['location']
      }
    }
  }
]

const response = await openai.chat.completions.create({
  model: 'gpt-5.5',
  messages: messages,
  tools: tools,
  tool_choice: 'auto'
})

// Function calls are automatically tracked in the trace

Multi-turn conversations

Use WorkflowTracer to group multiple turns of a conversation into a single workflow trace with per-turn spans:

TypeScript

import { traceWorkflowStep } from '@evalgate/sdk'

await tracer.startWorkflow('multi-turn-conversation', undefined, {
  sessionId: 'session_456'
})

const messages = []

// Turn 1
messages.push({ role: 'user', content: 'Hello!' })
const response1 = await traceWorkflowStep(tracer, 'turn-1', () =>
  openai.chat.completions.create({ model: 'gpt-5.5', messages })
)
messages.push(response1.choices[0].message)

// Turn 2
messages.push({ role: 'user', content: 'Tell me a joke' })
const response2 = await traceWorkflowStep(tracer, 'turn-2', () =>
  openai.chat.completions.create({ model: 'gpt-5.5', messages })
)
messages.push(response2.choices[0].message)

await tracer.endWorkflow({ status: 'success' })

A/B testing models

Trace model variants with metadata so you can compare quality, latency, and cost across experiments in the dashboard:

TypeScript

async function abTestModels(prompt: string) {
  const variant = Math.random() < 0.5 ? 'gpt-5.5' : 'gpt-5-mini'

  await tracer.startWorkflow('model-ab-test', undefined, {
    variant,
    experimentId: 'gpt55-vs-gpt5mini'
  })

  const result = await traceWorkflowStep(tracer, 'llm-call', () =>
    openai.chat.completions.create({
      model: variant,
      messages: [{ role: 'user', content: prompt }]
    })
  )

  await tracer.endWorkflow({ status: 'success' })
  return result
}

// Analyze results in the dashboard to compare:
// - Quality scores
// - Latency
// - Cost per completion
// - User satisfaction signals

What gets tracked automatically

Field	Description
Full prompt / response	Every input message, including system prompts, and the reply
Token usage	The `usage` object OpenAI returns for the request
Latency	Total request duration
Model	Model name, temperature, and other safe parameters
Status	Success or error, including cancelled streams

Troubleshooting

Traces not capturing token usage? Token usage comes from the usage object OpenAI returns. For streaming requests, OpenAI only sends usage when you pass stream_options: { include_usage: true }. Also make sure you are on the latest @evalgate/sdk or evalgate-sdk. Noticing high latency in traces? Make sure you are using async/await consistently. Synchronous operations block the event loop and inflate latency measurements. Missing streaming response data? The SDK automatically buffers streaming responses. If data appears missing, confirm the stream iterator is fully consumed (the for await loop completes) before the process exits.

Get Started

Core Concepts

Guides

SDK Reference

Platform

Openai integration

Trace OpenAI API calls with EvalGate

Install dependencies

Wrap the OpenAI client

Chat completions

Streaming responses

Function calling

Multi-turn conversations

A/B testing models

What gets tracked automatically

Troubleshooting

​Trace OpenAI API calls with EvalGate

​Install dependencies

​Wrap the OpenAI client

​Chat completions

​Streaming responses

​Function calling

​Multi-turn conversations

​A/B testing models

​What gets tracked automatically

​Troubleshooting

Trace OpenAI API calls with EvalGate

Install dependencies

Wrap the OpenAI client

Chat completions

Streaming responses

Function calling

Multi-turn conversations

A/B testing models

What gets tracked automatically

Troubleshooting