Skip to main content

​
Connect a model provider key

EvalGate is bring your own key (BYOK). It does not bundle model inference credits. LLM judges, Playground runs, online monitors, Copilot, and other model-backed workflows use a provider or gateway credential owned by your workspace, and the provider bills that model usage separately.

​
Step 1 — Open AI providers

Open Settings → AI providers at /settings/connections. The page combines provider connections, their health/model inventory/routing, and the separate Provider keys section. The old standalone Provider Keys and Model Gateway settings destinations have been consolidated here.

​
Step 2 — Add a credential

For a direct provider key:
  1. Scroll to Provider keys.
  2. Add the provider credential and give it a recognizable label.
  3. Save it, then return to the connection list above.
For a provider or gateway connection, use Add connection and follow the guided setup. Direct providers can prefill their standard endpoint; compatible gateways ask for the endpoint and model information they need.
EvalGate stores provider credentials encrypted. Never paste a key into a screenshot, commit, issue, or chat. If a key leaks, revoke it at the provider and replace it.

​
Step 3 — Check the connection

Select the connection and stay on Overview.
  • Check connection refreshes the recorded health result.
  • Send a test request sends one short request through that connection; your provider bills it.
  • The result reports current health, latency, and discovered capabilities without turning an old successful check into current health.
Only workspace admins and owners can run or change connection-level actions.

​
Step 4 — Refresh the model inventory

Open the selected connection’s Models view. Refresh/synchronize the model list when the provider catalog changes, then search the inventory for the model you intend to use. Use Routing when you need routing profiles or an explanation of which model would be chosen.

​
Troubleshooting

  • Connection check fails — verify the credential, endpoint, provider access, and workspace egress policy, then run Check connection again.
  • A model is missing — refresh the selected connection’s model inventory and confirm the provider account can access that model.
  • A test request is blocked — check workspace provider policy and cost controls; do not bypass them with an unregistered endpoint.
  • A key is inactive or expired — replace or reactivate it in the Provider keys section.

​
Up next

Now you can create an evaluation that uses a model-backed scorer or target.