Skip to main content

​
Add test cases

A test case is one input plus the expected behavior or output used by the evaluation. The evaluation detail page keeps ordinary case editing on Overview.

​
Add one case in the evaluation

  1. Open the evaluation and stay on Overview.
  2. In Test Cases, choose Add Test Case.
  3. Enter the input.
  4. Optionally add a name and expected output.
  5. Save the case.
Newly created or promoted cases can carry lifecycle state. If a case is waiting for activation, review it and choose Activate before expecting it to participate in later runs. You can edit or delete cases from the same Test Cases section.

​
Promote a real failure

A production failure is usually more valuable than a synthetic happy-path case. From Evidence → Logs, open the relevant trace and use the governed Create candidate action. The candidate keeps source lineage instead of pretending a copied prompt appeared from nowhere. Review and promote that candidate into the target evaluation before treating it as active regression coverage.

​
Import many cases

Bulk CSV/JSONL import belongs in Dataset Hub at /datasets, where imports produce an explicit report and a versioned dataset. Publish/review the dataset version, then bind that version to the evaluation. Do not look for Paste cases or direct JSONL upload buttons in the evaluation’s Test Cases section; those are not the current evaluation-detail workflow.

​
What makes a good case

  • Specific. Use behavior that has failed or would matter if it failed.
  • Reviewable. Record the expected behavior or an appropriate scorer/rubric.
  • Traceable. Preserve source trace, dataset, or review lineage where it exists.
  • Active deliberately. A saved candidate is not automatically a release gate.

​
Up next

Run the evaluation.