Platform

Batches

Send many chat completion requests at once. They run in the background, cost 50% of the list price, and finish within 24 hours.

POST /v1/batches
GET  /v1/batches
GET  /v1/batches/{id}
POST /v1/batches/{id}/cancel
GET  /v1/batches/{id}/results

Create a batch

The requests go in the JSON body. There is no file upload step.

curl https://api.utmost.site/v1/batches \
  -H "Authorization: Bearer $UTMOST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {"custom_id": "doc-1", "body": {"model": "auto", "messages": [{"role": "user", "content": "Summarize: ..."}]}},
      {"custom_id": "doc-2", "body": {"model": "utmost/llama-4-scout", "messages": [{"role": "user", "content": "Summarize: ..."}]}}
    ],
    "metadata": {"job": "nightly-summaries"}
  }'
Field Notes
requests 1 to 50,000 items.
requests[].custom_id 1 to 128 characters, unique within the batch. Results are matched by it.
requests[].body A chat completion request. stream must be absent or false. stream_options is ignored.
completion_window Optional. Only "24h".
metadata Optional. String values up to 512 characters.

The whole request body is limited to 8 MB, which caps large batches before the 50,000-item limit. Split big jobs into several batches.

Each body is validated when you create the batch. One invalid body rejects the whole batch with invalid_request, and param points at it. A duplicate custom_id does the same.

Creating a batch needs a positive balance and room under your spend caps. It counts as one request against your rate limit.

The response is 201 with the batch:

{
  "id": "batch_6HsowVhheD3l7E8JGI4V",
  "object": "batch",
  "endpoint": "/v1/chat/completions",
  "status": "in_progress",
  "created_at": 1795066929,
  "completed_at": null,
  "expires_at": 1795153329,
  "results_expire_at": null,
  "request_counts": { "total": 2, "completed": 0, "failed": 0 },
  "cost_usd": 0,
  "metadata": { "job": "nightly-summaries" },
  "results_url": null
}

Status

status Meaning
in_progress Requests are queued or running.
completed Every request succeeded or failed.
cancelled You cancelled it.
expired 24 hours passed. Unfinished requests failed with expired.

expires_at is the 24-hour deadline. cost_usd grows as requests finish. results_url is set once the batch is finished.

curl https://api.utmost.site/v1/batches/batch_6HsowVhheD3l7E8JGI4V -H "Authorization: Bearer $UTMOST_API_KEY"

GET /v1/batches returns your organization's 20 most recent batches. There is no pagination.

How requests run

  • Each request runs through the normal gateway: routing, refusal retry, fallback, hard-limit checks and metering. A batch request can use an auto alias or a pinned model.
  • x-utmost-variant is not available in batches. Pin an -abl model id instead.
  • Requests run in order of batch creation, then submission order, and are not rate limited.
  • Requests that fail with model_unavailable, upstream_error, upstream_timeout, safety_unavailable or internal_error are retried, up to 3 attempts in total. Other errors fail the request at once.
  • The balance, spend caps and key are checked again for each request. If credit runs out or a cap is reached part-way, the remaining requests fail with insufficient_credits or spend_cap_reached. If the key is revoked, they fail with key_revoked.

Results

curl https://api.utmost.site/v1/batches/batch_6HsowVhheD3l7E8JGI4V/results \
  -H "Authorization: Bearer $UTMOST_API_KEY"

JSON Lines (application/jsonl), one line per request, in submission order:

{"custom_id":"doc-1","response":{"status_code":200,"body":{"id":"chatcmpl-req_...","object":"chat.completion","model":"utmost/qwen3-30b-a3b","choices":[...],"usage":{...,"cost_usd":0.000004}}},"error":null}
{"custom_id":"doc-2","response":null,"error":{"code":"cancelled","message":"Batch cancelled."}}

Each line has response or error, not both. error.code is an API error code, or one of:

Code Meaning
cancelled The batch was cancelled before this request ran.
expired The 24-hour window ended before this request ran.
not_processed The request never ran.

Results are available once the batch is no longer in_progress. Before that, the endpoint returns invalid_request.

Cancel

curl -X POST https://api.utmost.site/v1/batches/batch_6HsowVhheD3l7E8JGI4V/cancel \
  -H "Authorization: Bearer $UTMOST_API_KEY"

Requests that have not started fail with cancelled. Requests already running finish and are billed. Finished requests keep their results. Cancelling a batch that is not in progress returns it unchanged.

Pricing

Each request is billed at 50% of the served model's list price, including cached-token pricing. You pay only for requests that succeed. See Billing.

Retention

Batch inputs are stored only so they can run. Each request's input is deleted once that request succeeds or fails. Inputs of requests that never ran (cancelled or expired) are kept until the results expire. Results are kept for 7 days after the batch finishes (results_expire_at). After that the results endpoint returns not_found. The batch object itself, with its counts and cost, stays listed.

Webhook

When a batch finishes, whether it completed, was cancelled or expired, we send batch.completed:

{
  "type": "batch.completed",
  "timestamp": "2026-11-19T05:42:27.000Z",
  "data": { "batch_id": "batch_6HsowVhheD3l7E8JGI4V", "status": "completed", "completed": 2, "failed": 0, "total": 2, "cost_usd": 0.000009 }
}

status is completed, cancelled or expired. See Webhooks.