API rate limits and evaluation usage
EvalGate bills completed evaluation results. API rate limits are separate operational safeguards and do not change the price.
Evaluation usage plans
The hard result-budget and provider-estimate contracts are a source candidate and require the matching deployed release. The prices below do not establish that rollout; review the active limit in Billing. Choose the monthly allowance that fits your evaluation volume:
Organizations above 5 million monthly results, rolling EvalGate out across
multiple organizations, or requiring procurement and centralized invoicing can
use a Strategic Enterprise agreement with a custom annual commitment and
negotiated volume schedule.
Every plan continues beyond its included allowance at $1 per 1,000 additional
results. A hard budget starts at 10,000 results per UTC calendar month, across plans. An administrator must explicitly approve increases. Included allowances still apply; this is a result limit, not a provider-dollar limit. Users, projects, datasets, and runs are unlimited on every plan.
Model inference is bring-your-own-key (BYOK). Your model provider bills its own
tokens and requests directly; EvalGate does not add an inference markup.
What counts as one result
One result is one persisted test-case outcome from an evaluation run.- A passed case counts once.
- An evaluated failure counts once.
- Multiple scorers on the same case do not multiply usage.
- A retry of the same evaluation run reuses the run billing key and does not create a second usage event.
- An empty run, a human-review placeholder, or an infrastructure failure before a result is persisted counts as zero.
Developer does not require a payment method for its included allowance. A
payment method is required to continue beyond that allowance or to subscribe
to Team, Scale, or Business. Every tier uses the same $1 per 1,000
overage rate.
API rate limits
Rate limits protect the service from runaway request loops. They are independent of monthly evaluation-result billing.
Authenticated requests are keyed by organization and user, or by API key when
available. Billing plans can select the corresponding session tier. EvalGate
uses Upstash Redis for shared admission. When Redis is unavailable, production
sensitive operations fail closed by default; permitted fallback uses bounded
state within each process.
Anonymous utility requests use Vercel’s platform-provided client address only
when running on Vercel. Self-hosted deployments, and requests without a valid
trusted address, share one anonymous bucket across utility routes. Caller-sent
forwarding headers cannot select a separate bucket. On self-hosted deployments,
this means the 100 requests/hour allowance is shared by anonymous visitors;
authenticated requests retain their organization, user, or API-key identity.
Authenticated requests also consume short-term token buckets:
Each bucket replenishes steadily to its listed capacity over 60 seconds.
Sensitive operations consume both their sensitive bucket and the shared total
bucket. Daily allowances stay unchanged. Successful responses continue to
report the daily quota; a burst refusal reports the binding short-term policy
and its
Retry-After delay.
Single and batch collector ingestion share additional organization-wide work
buckets: 1,000 traces and 20,000 spans, replenished over 60 seconds. A batch may
contain up to 100 traces. Admission uses the actual parsed trace and span counts
before any trace is written. A work-limit refusal returns 429, Retry-After,
and X-Collector-RateLimit-Policy, -Limit, -Remaining, and -Reset headers.
Reduce a batch that exceeds a bucket’s entire capacity before retrying it.
Command search admits up to four concurrent searches per process and two per
organization within that process, with no waiting queue. Each search dispatches
at most two object queries at once. Queries use transaction-local PostgreSQL
statement timeouts of at most one second, shortened when less remains in the
four-second query-dispatch budget. A timed-out object kind is reported as
unavailable; successful kinds retain their ranked results. Full admission returns
429 with Retry-After: 1. These process limits are operational bounds, not a
deployment-wide concurrency or availability guarantee.
Rate limit response headers
Every response that reaches rate-limit admission includes quota hints, including successful calls and admitted400, 401, and 500 responses. Public responses
that permit shared caching omit per-caller quota hints. Infrastructure health
checks and rejections before admission may also omit them. To inspect a safe,
public response, run curl -i https://www.evalgate.com/api/demo/custom-eval.
RateLimit-Policy and RateLimit follow the current IETF HTTPAPI structured
field draft. Numeric RateLimit-* fields support clients using the earlier
HTTPAPI convention; X-RateLimit-* fields retain their existing semantics.
Do not interpret RateLimit-Reset as a Unix timestamp. Headers are also listed
in Access-Control-Expose-Headers for responses with permitted CORS access.
These quota hints are operational controls, not an SLA or a
guarantee that a later request will be served.
When a request exceeds its limit, the API returns HTTP 429:
Retry-After, the t parameter in
RateLimit, or X-RateLimit-Reset before retrying. Retry-After takes
precedence when it is present.