ApiArena

Start run

POST
/arena/suites/{suite_id}/runs

Calls each active test case (or those in case_ids) repeats times. Test calls are real calls and are billed as such on both legs. The cost is estimated up front, with every call at its full max_duration_s, and the run is refused if your balance does not cover the estimate.

Authorization

BearerAuth
AuthorizationBearer <token>

API key authentication. Get your key from the dashboard.

In: header

Path Parameters

suite_id*string

Test suite ID

Formatuuid

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Response Body

application/json

application/json

application/json

application/json

application/json

application/json

application/json

curl -X POST "https://example.com/arena/suites/497f6eca-6276-4993-bfeb-53cbbbba6f08/runs" \  -H "Content-Type: application/json" \  -d '{}'
{
  "arena_run": {
    "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
    "suite_id": "3e9c08eb-436b-4d4b-9cfe-ca21d8bdb99b",
    "assistant_id": "b3a0271c-5834-40ff-bc5d-41ab5e39dfef",
    "status": "queued",
    "trigger": "string",
    "repeats": 0,
    "estimated_max_cost_eur": 0,
    "config_hash": "string",
    "created_at": "2019-08-24T14:15:22Z",
    "started_at": "2019-08-24T14:15:22Z",
    "finished_at": "2019-08-24T14:15:22Z",
    "estimate": {
      "max_cost_eur": 0,
      "max_credits": 0,
      "attempts": 0,
      "complete": true
    },
    "attempts": [
      {
        "id": "497f6eca-6276-4993-bfeb-53cbbbba6f08",
        "case_id": "c74269e5-1f97-4e20-9164-ffbe3494d8d6",
        "attempt_no": 0,
        "status": "queued",
        "void_reason": "no_answer",
        "last_error": "string",
        "started_at": "2019-08-24T14:15:22Z",
        "ended_at": "2019-08-24T14:15:22Z",
        "latency": {
          "property1": 0,
          "property2": 0
        },
        "tester_call_id": "string",
        "target_call_id": "string",
        "recordings": {
          "tester": "string",
          "events": "string",
          "assistant": "string",
          "received": "string"
        },
        "verdicts": [
          {
            "criterion_id": "string",
            "outcome": "pass",
            "source": "fact",
            "evidence_quote": "string",
            "turn_index": 0,
            "reasoning": "string",
            "judge_prompt_version": "string",
            "override_outcome": "string"
          }
        ],
        "metrics": [
          {
            "metric_id": "string",
            "metric_version": "string",
            "value": 0,
            "unit": "string"
          }
        ]
      }
    ]
  }
}
{
  "error": "Invalid API key",
  "message": "string"
}
{
  "error": "Invalid API key",
  "message": "string"
}
{
  "error": "Invalid API key",
  "message": "string"
}
{
  "error": "Invalid API key",
  "message": "string"
}
{
  "error": "Invalid API key",
  "message": "string"
}
{
  "error": "Invalid API key",
  "message": "string"
}

Save test case POST

Saving with an existing key creates a new version; a version used in a run never changes. Every tool of the assistant needs a choice in tool_mode: live, mock or fault. There is no default. facts, each tool response and each criterion are at most 8 KB as JSON, the whole case at most 64 KB. source_call_ids must be calls of your organization.

Cancel run POST

Attempts still in the queue stop; a call already in progress is finished.