Streaming, tools and structured output

The chat completion reference lists stream, tools and response_format as supported fields. This page shows what each one actually looks like on the wire.

Everything below is OpenAI-compatible, so an existing SDK works unchanged once you point it at the base URL.

Streaming

Set stream: true and the response arrives as server-sent events instead of one JSON body. Each data: line carries a JSON delta; the stream ends with a literal data: [DONE].

import os, requests, json

response = requests.post(
    "https://api.deutschlandgpt.de/v2/chat/completions",
    headers={"Authorization": f"Bearer {os.environ['DGPT_API_KEY']}"},
    json={
        "model": "gpt-4o",
        "messages": [{"role": "user", "content": "Count to five."}],
        "stream": True,
        "stream_options": {"include_usage": True},
    },
    stream=True,
)

for line in response.iter_lines():
    if not line or not line.startswith(b"data: "):
        continue
    payload = line[len(b"data: "):]
    if payload == b"[DONE]":
        break
    delta = json.loads(payload)["choices"][0]["delta"]
    print(delta.get("content", ""), end="", flush=True)

stream_options.include_usage adds one final chunk with the token counts before [DONE]. Without it a streamed response never reports its own usage, which is the usual reason a streaming integration cannot account for its spend.

Tools (function calling)

Describe the functions the model may call in tools. Each is a JSON Schema; the model never executes anything, it only asks you to.

{
  "model": "gpt-4o",
  "messages": [{ "role": "user", "content": "What is the weather in Berlin?" }],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city — used by the model to decide when to call it",
        "parameters": {
          "type": "object",
          "properties": { "city": { "type": "string" } },
          "required": ["city"]
        }
      }
    }
  ]
}

A function name is at most 64 characters, from a-z, A-Z, 0-9, underscores and dashes.

When the model wants a call, the response carries tool_calls instead of content. You run the function yourself and send the result back as a tool message, then call the endpoint again with the extended history:

{
  "model": "gpt-4o",
  "messages": [
    { "role": "user", "content": "What is the weather in Berlin?" },
    {
      "role": "assistant",
      "tool_calls": [
        {
          "id": "call_abc123",
          "type": "function",
          "function": { "name": "get_weather", "arguments": "{\"city\":\"Berlin\"}" }
        }
      ]
    },
    { "role": "tool", "tool_call_id": "call_abc123", "content": "18 °C, light rain" }
  ]
}

arguments is a JSON string, not an object. Parse it before use. The tool_call_id on your reply must match the id of the call it answers.

When you send a call back in the history, its arguments should be a JSON object. An empty string, or leaving the field out, means a call without arguments. Anything else, such as truncated JSON, is passed to the model as a call without arguments.

By default the model may request several calls in one turn. Set parallel_tool_calls: false to force one at a time.

Structured output

To get JSON matching a schema rather than prose, pass a response_format of type json_schema:

{
  "model": "gpt-4o",
  "messages": [{ "role": "user", "content": "Extract the invoice total and currency." }],
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "invoice",
      "strict": true,
      "schema": {
        "type": "object",
        "properties": {
          "total": { "type": "number" },
          "currency": { "type": "string" }
        },
        "required": ["total", "currency"],
        "additionalProperties": false
      }
    }
  }
}

The answer still arrives in choices[0].message.content, as a JSON string you parse yourself.

strict: true is what makes the model adhere to the schema rather than treat it as a suggestion. It requires additionalProperties: false and every property listed in required.

Reasoning models

reasoning_effort sets how much thinking a reasoning model does before answering: none (the default, extended thinking off), minimal, low, medium, high or xhigh. Lower is faster and cheaper. It has no effect on models that do not reason. Some reasoning models cannot switch thinking off; for those, a lower value is raised to the lowest level the model supports.

What bounds your throughput

There is no request-per-minute rate limit and no 429. What actually stops a request is money and permission:

See API keys for per-key limits and Billing for credits.