Skip to main content

​
Bring one reviewed case first

Keep your existing tool while you run the local gate against one behavior. An experiment compares candidates; a trace records one interaction; a gate decides whether the required checks for a change cleared. Importing a trace or an experiment is not accepting a release baseline. There is no advertised one-command Promptfoo or Braintrust importer. The LangSmith command is file conversion, not hosted account migration:
Read the installed command’s options before converting a copy of your export. Keep the original file and source IDs. Missing timestamps, provider identity, cost or human approval remain unknown; never fabricate them to satisfy a schema. An imported score is not a calibrated EvalGate judge result.

​
Compare before switching

  1. Freeze one small exported set and review its expected behavior.
  2. Port one deterministic case using the five-minute loop. Inspect both a passing and failing output in your existing tool and in EvalGate.
  3. Record differences in input rendering, normalization, timeout, retries, score scale and missing-evidence handling. Do not average away disagreement.
  4. Accept a baseline only after those cases pass and a deliberate break blocks.
  5. Run both systems in parallel before changing a required CI check. Retain the original tool for history and unsupported features.
Go and Ruby generated REST clients are not evaluation runners. TypeScript and Python execute their own runtime’s cases; Java and .NET runners are not available. Generic CI execution does not imply native Jenkins, Buildkite, GitLab or CircleCI repository integration. The official evalgate/examples repository runs the full five-minute loop against published SDK3.15.0 in hosted CI, with network access denied after installation. It includes the reviewed cases, deliberate break and restore steps. This proves the local gate; it does not import another tool’s hosted history or qualify a production review workflow.