Skip to main content

​
Enterprise governance and compliance controls

Configure provider policies, PII protection, retention, and access controls for your org from the System Controls card in Settings → Workspace.
EvalGate enforces governance as system behavior, not as a checklist. Provider restrictions, PII scrubbing, drift detection, and audit logging are part of the runtime path — they run on every evaluation and judge execution, not just when you remember to check them. Configure the editable controls from the System Controls card in Settings → Workspace (/settings/organization).
Control which AI providers and models your organization can use for judge evaluations. These settings apply to all evaluation runs across your org.What you can configure:
  • Allow external LLM providers — one switch that blocks or allows outbound model calls at the org boundary
  • Provider allowlist — restrict outbound calls to an approved list of providers; an empty list allows none
  • Cost alert budgets — daily, weekly, and monthly USD budgets (costAlerts on PUT /api/system/controls)
Provider and model restrictions are enforced before outbound calls are made. Disallowed providers are blocked at the platform boundary — the request never reaches the external API.
EvalGate scrubs inputs before they reach the embedding pipeline or external judge calls, so sensitive data in your test cases or traces does not leave your org boundary in cleartext.How it works:
  • Inputs pass through the scrubber before embedding or being sent to an external judge
  • External judge calls respect your org’s PII policy — if scrubbing is not permitted for a given request, the provider call is blocked entirely
  • Scrubbing is auditable: the audit log records that scrubbing occurred and what policy applied, without storing the raw sensitive values themselves
To enable PII protection:In Settings → Workspace → System Controls → PII policy, choose a scrub mode (Redact, Tokenize, Hash, or Drop) and decide whether to allow PII in embeddings and raw PII storage. Changes apply to subsequent runs.
Enabling PII protection does not retroactively scrub data already stored in traces or evaluation records. Apply PII policies before ingesting sensitive data.
The drift gate enforces cross-surface parity — it detects when evaluation behavior shifts across environments, model versions, or prompt changes and treats those shifts as violations, not informational alerts.What drift enforcement covers:
  • Cross-surface parity — the drift gate compares results across surfaces (e.g. staging vs. production, model v1 vs. v2) and flags deviations that exceed your thresholds
  • Runtime monitors — continuous tracking of score distribution and variance shifts catches gradual degradation that single-run comparisons miss
  • Hardcoded threshold violations — if you set a minimum pass rate or quality score floor, breaches are treated as violations and surface in the dashboard and CI gates
  • Versioned, auditable baselines — every baseline is versioned so you can trace exactly which run and configuration a comparison was made against
Use drift enforcement in combination with CI gates to automatically block deployments when a model or prompt change causes a statistically significant quality regression.
Every action that modifies data in EvalGate is org-scoped and recorded in the audit log. This applies to all users and API keys in your organization.What is audited:
  • All mutations (evaluation creation, run creation, test case changes, configuration updates) are attributed to an org member or API key with a timestamp
  • Judge executions record the full execution context: model config, retry count, timing, and whether PII redaction was applied
  • Controls are queryable through the API so you can export audit data for compliance reporting
Access control:
  • RBAC is part of the control layer — role assignments determine which members can create evaluations, run judges, export data, and modify org settings
  • Multi-tenant isolation ensures data from one org is never accessible to another, even under the same EvalGate instance
Manage member roles from Settings → Team.
Retention policy in EvalGate is not just metadata tagging. You can configure org-scoped destructive retention sweeps that permanently remove expired evaluation data and logs under policy.How retention works:
  • Set a retention period for evaluation runs, traces, and audit records
  • When a sweep runs, expired data is actually deleted — not soft-deleted or flagged
  • Each sweep is recorded in the audit log with scope, timestamp, and affected record counts so you have full visibility into what was removed and when
Configure eval data retention, the audit evidence window, Copilot message retention, and remote runner log retention in Settings → Workspace → System Controls.
Destructive retention sweeps are irreversible. Review your retention settings carefully and confirm the policy applies only to the data categories you intend before activating a sweep.

​
Where to configure governance

Governance controls live under Settings. Updating System Controls requires an Admin role or higher; drift thresholds come from the org policy and are shown as drift alerts, not edited here.

Provider and PII policy

Settings → Workspace → System Controls

Runtime policy packs

Settings → Rules

Activity history

Settings → Activity history

Member roles

Settings → Team