> ## Documentation Index
> Fetch the complete documentation index at: https://evalgate.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Model providers and BYOK

> Connect your own model provider or gateway credential, understand billing, and keep EvalGate and provider keys separate.

# Bring your own provider key (BYOK)

> [!WARNING]
>
> Provider-key setup is temporarily unavailable in the hosted application while
> credential storage is restored and verified. Credential-free local regression
> gates remain available. A local setup test does not establish hosted provider
> access.

EvalGate uses a **bring your own provider key (BYOK)** model for model-backed
workflows. EvalGate does not bundle model inference credits. You connect a
provider or gateway credential owned by your organization, and that provider
bills your organization directly for model usage.

> [!NOTE]
>
> Local regression gates and deterministic assertions do not require an
> EvalGate account or a provider key. A provider credential is required only
> when a workflow asks EvalGate to call a model, such as an LLM judge,
> synthesis, or governed eval execution.

## Know which key you are using

| Credential | Purpose | Where it is used | Who bills it |
| - | - | - | - |
| **EvalGate API key** | Authenticates your SDK, CLI, or REST request to EvalGate | `Authorization: Bearer ...` or `EVALGATE_API_KEY` | EvalGate plan and platform limits apply |
| **Provider or gateway key** | Authorizes model inference for model-backed workflows | Stored in **Settings → AI providers** | Your provider or gateway bills model usage directly |

An EvalGate API key does not include model access and cannot substitute for a
provider key. Do not send a provider key in the EvalGate `Authorization`
header.

## Choose a connection path

> [card] [Direct provider key](#direct-provider-keys)
>
> Connect a supported provider credential for judge evaluations and other
> model-backed product workflows.

> [card] [Model Gateway](#model-gateway)
>
> Register a direct endpoint, OpenAI-compatible gateway, OpenRouter, LiteLLM,
> or a customer-hosted gateway with governed routing and call evidence.

### Direct provider keys

Use the **Provider keys** section of **Settings → AI providers** when the
product workflow supports a direct provider integration.

**1. Add the provider credential**

Choose the provider, give the credential an environment-specific name, and
paste the key issued by that provider.

**2. Confirm the model-backed workflow**

Return to setup or the evaluation that uses an AI reviewer (LLM judge) and
verify that the provider is available. A saved key alone does not prove that a model is healthy or
allowed by organization policy.

**3. Rotate or revoke deliberately**

Use separate credentials for development, staging, and production. Rotate a
suspect credential and revoke it at the provider as well as in EvalGate.

### Keep provider keys in your Infisical vault

Instead of pasting a key into EvalGate, you can link it from your own
[Infisical](https://infisical.com) project. EvalGate stores only where the key
lives. When you replace the key in Infisical, EvalGate uses the new value
within about a minute, with no redeploy and nothing to change in EvalGate.

> [!NOTE]
>
> Linking works on Infisical's free plan. Scheduled, automatic key rotation is
> a separate Infisical feature on its paid plans; EvalGate picks up whatever
> value Infisical currently holds either way. The Infisical connection secret
> uses the same hosted credential storage as provider keys, so it is subject
> to the availability notice above.

**1. Prepare Infisical**

Create a project used only for EvalGate, add your provider keys (for
example `OPENAI_API_KEY`), and create a machine identity with Universal
Auth and read access to that project. Prefer a client secret without an
expiry; an expired secret stops EvalGate from reading your keys.

**2. Connect**

Open **Connections → Secret managers**, choose your Infisical region (US or
EU), and enter the project ID, environment, client ID, and client secret.
EvalGate signs in and reads the environment before saving anything.

**3. Link a provider**

Choose the provider, folder, and secret name, then link it. EvalGate reads
the secret once to confirm it is reachable. **Check now** reads it again
and, where the provider offers a free metadata request, confirms the
provider accepts it.

What to expect:

* A linked key takes priority over a key saved directly in EvalGate for the
  same provider. Unlinking switches back to the saved key immediately.
* If EvalGate cannot read the linked key (Infisical unavailable, identity
  revoked, secret removed), model calls for that provider stop with a clear
  error. EvalGate never falls back to a different key.
* If you revoke an old key right after updating Infisical, EvalGate re-reads
  the vault once when the provider refuses the old key and retries with the
  new value. It never re-sends a key the provider has refused.
* EvalGate records a one-way fingerprint and the Infisical version number of
  each value it reads, so you can see when a new value took effect. It never
  stores or logs the key.
* Supported providers: OpenAI, Anthropic, Google Gemini, xAI, and Moonshot
  direct keys. Model Gateway connections, self-hosted Infisical, and more than
  one Infisical project per workspace are not supported yet.

### Model Gateway

Use the gateway **Connections** on **Settings → AI providers** when you need custom endpoints, explicit model
inventory, routing profiles, health checks, budget and policy decisions, or a
model-call ledger.

The onboarding flow requires a provider or gateway URL and credential, then
performs these steps in order:

1. Save the encrypted credential and validate the registered egress target.
2. Run a provider health check and synchronize the available models.
3. Select a default model or routing profile.
4. Complete a real test call and inspect its policy, cost, and provenance
   evidence.

> [!WARNING]
>
> Do not assume that adding a direct provider key completes Model Gateway
> onboarding. Follow the setup state for the workflow you intend to run and
> complete its health, model-sync, and test-call checks.

## Security and data flow

BYOK means you control the upstream provider account and credential. It does
not mean the credential bypasses EvalGate.

* Organization provider credentials are encrypted at rest and are not returned
  in plaintext by the API after creation.
* EvalGate decrypts the credential server-side only when an allowed
  model-backed request needs it.
* The prompt, output, and metadata required for a model call are sent to the
  provider or gateway selected by your organization.
* Provider errors, application logs, and API responses redact recognized
  credentials. Rotate any credential you suspect was exposed.
* Provider retention, training, residency, and zero-data-retention terms still
  apply to requests sent to that provider.

For deployment-level encryption-key rotation, revocation, and crypto-erasure,
see [Provider credential key management](https://evalgate.com/docs/platform/provider-credential-key-management.md).

## Billing and cost evidence

Your EvalGate subscription covers the EvalGate platform according to your
plan. Provider inference is separate: the connected provider or gateway bills
your account for tokens, requests, or other model usage.

The Model Gateway records known provider costs and the source of each price. If
pricing cannot be verified, EvalGate reports the cost as **unknown** rather than
assuming it is zero. Configure organization budgets and provider-side limits
before enabling automated or high-volume workflows.

## Operational checklist

* Use a least-privilege provider credential dedicated to EvalGate.
* Keep development, staging, and production credentials separate.
* Confirm the provider, model, region, and data policy before the first call.
* Run a health check, model sync, and governed test call after onboarding or
  rotation.
* Set provider-side spend limits in addition to EvalGate budget policy.
* Inspect the Model Gateway ledger when a call is blocked, retried, or has
  unknown cost.

## Troubleshooting

| Symptom | What to check |
| - | - |
| `401` from EvalGate | The EvalGate API key, its scopes, and organization access |
| Provider reports unauthorized | The provider or gateway credential; rotate it if needed |
| "Could not read the key from your Infisical vault" | **Connections → Secret managers**: the identity's client secret, its read access to the project, and the secret's folder and name |
| No models available | Gateway health, model synchronization, provider permissions, and region |
| Judge or synthesis is blocked | Provider allowlist, routing profile, budget, calibration, and policy decision |
| Cost shows unknown | Verified model pricing and provider cost-reporting support |

See [Authentication](https://evalgate.com/docs/authentication.md) for EvalGate API keys and
[LLM judge orchestration](https://evalgate.com/docs/concepts/llm-judge.md) for judge calibration and policy
guidance.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.