Run a Playground
The Playground is where you compare candidate prompt/model changes against a baseline before promoting anything. For an account-free introduction, open the public refund check. It starts with the computed result for one synthetic example: expectedreview,
answer deny, and 0/1 checks passed. You can check the answer review against
the same rule. This does not call a model or issue a refund.
Other public examples include
recorded samples and a check for your own output. The workspace comparison
workflow below uses your evaluation cases.
Step 1 — Open a Playground
Open/playgrounds, then choose the evaluation you want to experiment on.
The evaluation-scoped Playground lives at
/evaluations/{id}/playground.
Step 2 — Define the baseline and candidates
Keep one baseline variant that represents the current behavior, then add one or more candidates. Freeze the prompt/model/configuration identity used by each run so the comparison remains interpretable.Step 3 — Run the cases
Choose the cases and scorer, then start the run. Results retain variant, case, scorer/run evidence, model-call references, cost, and terminal status where available. A cancelled, partial, or otherwise incomplete run is not a verified winner.Step 4 — Compare outcomes
Compare quality together with the protected requirements, reliability, latency, and cost that matter for the decision. Do not promote a candidate merely because one average metric improved.Troubleshooting
- Provider/connection failure — open Settings → AI providers, select the connection, and check its current health/model inventory.
- Cost or provider policy block — resolve the named workspace policy rather than bypassing it.
- All variants look identical — verify the scorer and cases can actually distinguish the behavior you changed.
- A run is stuck or cancelled — inspect the run state and rerun only after the execution problem is understood.