Concepts
Routing
Send one of three aliases as model and the router picks a model for each request.
| Alias | Easy | Medium | Hard |
|---|---|---|---|
auto |
small | medium | large |
auto-fast |
small | small | medium |
auto-best |
large | large | large |
The table maps the estimated difficulty of the request to a tier. Within the tier, the router uses the cheapest model (input price plus output price) that fits the request.
Routing adds no surcharge. You pay the price of the model that served the request. The routing classifier's own calls are not billed.
Estimating difficulty
The router uses one of two methods.
Classifier. When a routing classifier model is configured, the router sends it the last user message (up to 4,000 characters), the system prompt (up to 1,000 characters), and the number of attached tools. It returns a difficulty and whether the request needs a refusal-free model. The call has a 2.5-second budget. If it fails or times out, the router uses the heuristic.
Heuristic. A deterministic score:
| Signal | Score |
|---|---|
| Estimated input over 30,000 tokens | +2 |
| Estimated input over 6,000 tokens | +1 |
| Any tools attached | +1 |
| Code in the last user message | +1 |
| Words such as prove, derive, debug, refactor, threat model, audit, analyze, compare, plan | +1 |
reasoning_effort: "high" / "medium" |
+2 / +1 |
| Last user message under 200 characters, with no code and no tools | −1 |
A total of 0 or less is easy, 1 to 2 is medium, 3 or more is hard.
The heuristic cannot tell whether a request needs a refusal-free model. With the heuristic, the router always starts on the standard variant and relies on the refusal retry below.
Choosing the variant
- If the request has an
x-utmost-variantheader, that variant is used. - Otherwise, if the classifier says the request needs a refusal-free model and your organization is Verified or above, the router uses the abliterated variant.
- Otherwise it uses the standard variant.
If no abliterated model can serve the request in case 2, the router falls back to standard models.
Fitting the request
A model fits when the estimated input plus max_completion_tokens (or max_tokens, or 1,024 if neither is set) is within its context window, and, if tools are attached, it supports tools. Input is estimated at about 4 characters per token, plus about 750 tokens per image.
If no model in the chosen tier fits, the router tries larger tiers, then smaller ones. If none fits, the request fails with model_unavailable (503).
Refusal retry
When a standard model refuses, the router retries the request once on the abliterated variant of the same tier.
It applies only when all of these hold:
- the request used an auto alias, not a pinned model;
- the request is not streamed;
- your organization is Verified or above;
- you did not send
x-utmost-variant; - the answer has no tool calls.
A refusal is detected by its opening phrases ("I'm sorry", "I can't help with", "As an AI", and similar), the convention used to score refusal rates in the abliteration literature. Only the first choice is checked.
The refused attempt is logged with the suffix _r on its request id and is not billed. You pay only for the retried answer. If the retry cannot run (no capacity, or the safety classifier is down), you get the original refusal.
A streamed request cannot be retried: by the time a refusal is visible, it has been sent to you. To get the retry, use a non-streaming request, or send x-utmost-variant: abliterated.
Fallback
Within the chosen tier and variant, the router tries models in price order, and each model's deployments in priority order, until one accepts the request. A routed request is never moved to a different tier or variant by fallback.
Pinned requests can fail over between deployments of the same model, but are never sent to a different model.
Fallback happens only before the first byte of the response. Once a stream has started, there is no fallback. When the backend rejects the request itself (HTTP 400, 404, 413 or 422), there is no fallback either; you get invalid_request with the backend's message.
Seeing what happened
| Header | Example | Present |
|---|---|---|
x-utmost-model |
utmost/qwen3-30b-a3b |
Always |
x-utmost-route |
auto->small/standard |
Auto aliases only |
x-utmost-route |
auto->small/abliterated (refusal-retry) |
After a refusal retry |
x-utmost-fallback |
utmost/mistral-small-3.2-24b |
Only after a fallback; names the model tried first |
The model field in the response body is the served model, not the alias.
Forcing a variant: x-utmost-variant
curl https://api.utmost.site/v1/chat/completions \
-H "Authorization: Bearer $UTMOST_API_KEY" \
-H "Content-Type: application/json" \
-H "x-utmost-variant: abliterated" \
-d '{"model": "auto-best", "messages": [{"role": "user", "content": "..."}]}'- Values:
standardorabliterated. Anything else returnsinvalid_request. abliteratedneeds a Verified organization, or the request fails withtier_required.- With the header set, there is no refusal retry and no fallback to the other variant.
- The header has no effect on pinned models. Pin an
-ablid instead. - It works on
/v1/chat/completions,/v1/responses,/v1/messagesand/v1/workflows/security-review(where it applies to every pass). Batches do not accept it.
Allowlists and routing
If the calling key has a model allowlist:
- When the alias is on the list (or matches a
*entry), the router may choose any model. - When it is not, the router considers only models on the list, including for the refusal retry. If none can serve the request, it fails with
model_not_allowed.
Feedback
Tell us when routing got it wrong. Ratings feed router tuning. We store the rating on the request log; we never store content.
curl https://api.utmost.site/v1/feedback \
-H "Authorization: Bearer $UTMOST_API_KEY" \
-H "Content-Type: application/json" \
-d '{"request_id": "req_vhX379RVYQlmrhnWrPHvHUfU", "rating": "bad", "reason": "wrong_model"}'{ "object": "feedback", "request_id": "req_vhX379RVYQlmrhnWrPHvHUfU", "rating": "bad", "reason": "wrong_model" }| Field | Values |
|---|---|
request_id |
The x-request-id of a request from your organization |
rating |
good or bad |
reason |
Optional: wrong_answer, refused, too_slow, wrong_model, other |
An unknown request_id returns not_found. Sending feedback again for the same request replaces the earlier rating.