Concepts

Structured output

Set response_format on chat completions, or text.format on Responses.

JSON mode

{
  "model": "auto",
  "response_format": { "type": "json_object" },
  "messages": [
    { "role": "system", "content": "Reply with a JSON object with keys `verdict` and `reason`." },
    { "role": "user", "content": "Is `eval(input())` safe?" }
  ]
}

The model returns valid JSON, but its shape is not enforced. Say in the prompt which keys you want.

JSON Schema

{
  "model": "auto",
  "messages": [{ "role": "user", "content": "Classify this log line: 'Failed password for root from 203.0.113.9'" }],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "classification",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "category": { "type": "string", "enum": ["auth_failure", "auth_success", "other"] },
          "source_ip": { "type": "string" }
        },
        "required": ["category", "source_ip"],
        "additionalProperties": false
      }
    }
  }
}

The content of the reply is a JSON string that matches the schema:

{ "category": "auth_failure", "source_ip": "203.0.113.9" }
Field Notes
name Required
schema A JSON Schema object
strict Asks the backend to enforce the schema exactly
description Optional

Support and limits

  • Every model lists capabilities.json_mode: true. Schema enforcement is done by the model backend. How strictly a schema is enforced, and which JSON Schema keywords are supported, varies by model. Validate the output before you rely on it.
  • Utmost does not validate model output against your schema. A reply that does not parse is returned as is, not as an error.
  • finish_reason: "length" means the JSON was cut off. Raise max_completion_tokens.

Responses API form

In /v1/responses, the schema fields sit directly in text.format:

{
  "model": "auto",
  "input": "Classify: 'Failed password for root from 203.0.113.9'",
  "text": { "format": { "type": "json_schema", "name": "classification", "strict": true, "schema": { "type": "object", "properties": { "category": { "type": "string" } }, "required": ["category"], "additionalProperties": false } } }
}

Python

import json, os
from openai import OpenAI

client = OpenAI(api_key=os.environ["UTMOST_API_KEY"], base_url="https://api.utmost.site/v1")

r = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "Classify: 'Failed password for root from 203.0.113.9'"}],
    response_format={"type": "json_schema", "json_schema": {"name": "classification", "strict": True, "schema": {
        "type": "object",
        "properties": {"category": {"type": "string"}, "source_ip": {"type": "string"}},
        "required": ["category", "source_ip"],
        "additionalProperties": False,
    }}},
)
print(json.loads(r.choices[0].message.content))