> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming de respostas de LLM

> Habilite o streaming em tempo real de respostas de LLM. Entregue conteúdo incremental para uma UX melhor com Server-Sent Events, hooks do React e padrões de tratamento de erros.

**Casos de uso**

* Interfaces de chat que exibem as respostas à medida que chegam, antes de a geração terminar.
* Geração de conteúdo longo (relatórios, código), em que esperar pela saída completa prejudica a UX.
* Fluxos de agentes que exibem etapas de raciocínio ou chamadas de ferramentas em tempo real.
* Redução da latência percebida em modelos lentos ou saídas grandes.

***

## Início rápido

Habilite o streaming de respostas em tempo real para uma experiência do usuário melhor.

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -N -X POST https://router.toqan.ai/eu/v3/router/responses \
  -H "Authorization: Bearer $TOQAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "input": "Write a story about space exploration",
    "stream": true
  }'
```

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const stream = await client.responses.create({
  model: "openai/gpt-5.4",
  input: "Write a story about space exploration",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

stream = client.responses.create(
    model="openai/gpt-5.4",
    input="Write a story about space exploration",
    stream=True,
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const stream = await client.chat.completions.create({
  model: "openai/gpt-5.4",
  messages: [
    { role: "user", content: "Write a story about space exploration" },
  ],
  stream: true,
});

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content || "";
  if (content) {
    process.stdout.write(content);
  }
}
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

stream = client.chat.completions.create(
    model="openai/gpt-5.4",
    messages=[{"role": "user", "content": "Write a story about space exploration"}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

## Configuração

| Parâmetro | Tipo | Obrigatório | Descrição |
| - | - | - | - |
| `stream` | boolean | Sim | Habilita respostas em streaming |
| `stream_options.include_usage` | boolean | Não | Somente **Chat Completions**. Informa o uso de tokens de toda a solicitação em um [chunk final](#usage-in-the-final-chunk) |

**Todos os modelos são compatíveis com streaming**: nenhuma configuração adicional é necessária.

## Formato da resposta

**Chunks do streaming:**

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "type": "response.output_text.delta",
  "delta": "Hello"
}
```

```json JSON (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "id": "chatcmpl-123",
  "object": "chat.completion.chunk",
  "created": 1677652288,
  "model": "openai/gpt-5.4",
  "choices": [
    {
      "index": 0,
      "delta": {
        "content": "Hello"
      },
      "finish_reason": null
    }
  ]
}
```

**Chunk final:**

O último evento de um stream Responses é `response.completed`. Ele traz o objeto da resposta, incluindo `usage` e `telemetry`. O evento `response.output_text.done` marca apenas o fim do texto; por isso, continue lendo depois dele até que `response.completed` chegue.

```json JSON theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "type": "response.completed",
  "sequence_number": 24,
  "response": {
    "id": "resp_01KP6E957P5VF...",
    "status": "completed",
    "telemetry": {
      "trace_id": "9f2c1b3a4d5e6f708192a3b4c5d6e7f8",
      "span_id": "1a2b3c4d5e6f7081"
    },
    "usage": {
      "input_tokens": 123,
      "output_tokens": 36,
      "total_tokens": 159
    }
  }
}
```

```json JSON (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "choices": [
    {
      "index": 0,
      "delta": {},
      "finish_reason": "stop"
    }
  ]
}
```

### Uso no chunk final

`stream_options: { "include_usage": true }` segue o formato da OpenAI e é aceito. Quando o stream informa o uso de tokens de toda a solicitação, ele faz isso em um chunk final após o chunk de conteúdo, enviado antes do marcador `[DONE]`.

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -N -X POST https://router.toqan.ai/eu/v3/router/chat/completions \
  -H "Authorization: Bearer $TOQAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "messages": [{ "role": "user", "content": "Hello!" }],
    "stream": true,
    "stream_options": { "include_usage": true }
  }'
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const stream = await client.chat.completions.create({
  model: "openai/gpt-5.4",
  messages: [{ role: "user", content: "Hello!" }],
  stream: true,
  stream_options: { include_usage: true },
});

let text = "";
let totalTokens: number | null = null;

for await (const chunk of stream) {
  const content = chunk.choices[0]?.delta?.content;
  if (content) text += content;
  if (chunk.usage) totalTokens = chunk.usage.total_tokens;
}

console.log(text || "");
console.log(`total_tokens: ${totalTokens}`);
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

stream = client.chat.completions.create(
    model="openai/gpt-5.4",
    messages=[{"role": "user", "content": "Hello!"}],
    stream=True,
    stream_options={"include_usage": True},
)

text = ""
total_tokens = None

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        text += chunk.choices[0].delta.content
    if chunk.usage:
        total_tokens = chunk.usage.total_tokens

print(text)
print(f"total_tokens: {total_tokens}")
```

Leia `usage` do chunk que o contiver. A maioria das solicitações o envia em um chunk com um array `choices` vazio, mas uma solicitação que usa [ferramentas de servidor](/docs/pt-BR/ai-gateway/features/server-tools.md) o envia no chunk que também contém `finish_reason`; por isso, verificar `usage` em todos os chunks é a abordagem confiável:

```json JSON (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
{
  "id": "chatcmpl-123",
  "object": "chat.completion.chunk",
  "created": 1677652288,
  "model": "openai/gpt-5.4",
  "choices": [],
  "usage": {
    "prompt_tokens": 12,
    "completion_tokens": 8,
    "total_tokens": 20
  }
}
```

## Exemplos de código

```bash cURL theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -N -X POST https://router.toqan.ai/eu/v3/router/responses \
  -H "Authorization: Bearer $TOQAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "input": "Write a detailed explanation of quantum computing",
    "stream": true
  }'
```

```bash cURL (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
curl -N -X POST https://router.toqan.ai/eu/v3/router/chat/completions \
  -H "Authorization: Bearer $TOQAN_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "openai/gpt-5.4",
    "messages": [
      {
        "role": "user",
        "content": "Write a detailed explanation of quantum computing"
      }
    ],
    "stream": true
  }'
```

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const stream = await client.responses.create({
  model: "openai/gpt-5.4",
  input: "Write a detailed explanation of quantum computing",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

stream = client.responses.create(
    model="openai/gpt-5.4",
    input="Write a detailed explanation of quantum computing",
    stream=True,
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const stream = await client.chat.completions.create({
  model: "openai/gpt-5.4",
  messages: [
    {
      role: "user",
      content: "Write a detailed explanation of quantum computing",
    },
  ],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

stream = client.chat.completions.create(
    model="openai/gpt-5.4",
    messages=[
        {
            "role": "user",
            "content": "Write a detailed explanation of quantum computing",
        }
    ],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end="")
```

## Padrões de processamento de streams

> [!NOTE]
>
> Os exemplos desta seção usam o endpoint de Chat Completions. Os mesmos padrões se aplicam à Responses API: substitua `chat.completions.create(...)` por `responses.create(...)`, altere o endpoint para `/v3/router/responses` e trate eventos `response.output_text.delta` em vez de `choices[0].delta.content`.

### Processamento básico

Acumule os deltas em uma string completa e detecte a conclusão por meio de `finish_reason`.

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
const processStream = async (stream) => {
  let fullResponse = "";

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content || "";
    if (content) {
      fullResponse += content;
      console.log(content); // Real-time output
    }

    // Check for completion
    if (chunk.choices[0]?.finish_reason) {
      console.log(`\nStream finished: ${chunk.choices[0].finish_reason}`);
      break;
    }
  }

  return fullResponse;
};
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
def process_stream(stream) -> str:
    full_response = ""

    for chunk in stream:
        content = chunk.choices[0].delta.content or "" if chunk.choices else ""
        if content:
            full_response += content
            print(content, end="", flush=True)

        if chunk.choices and chunk.choices[0].finish_reason:
            print(f"\nStream finished: {chunk.choices[0].finish_reason}")
            break

    return full_response
```

### Com tratamento de erros

Proteja-se contra quedas de rede e erros inesperados envolvendo o loop do stream em um try/except. O exemplo em TypeScript também reinicia um timeout a cada chunk.

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
const updateUI = (content: string) => { process.stdout.write(content); }; // replace with your UI update logic

const robustStreamProcessing = async (stream) => {
  try {
    let response = "";
    const timeout = setTimeout(() => {
      throw new Error("Stream timeout");
    }, 30000);

    for await (const chunk of stream) {
      clearTimeout(timeout);

      if (chunk.choices[0]?.delta?.content) {
        response += chunk.choices[0].delta.content;
        // Update UI with new content
        updateUI(chunk.choices[0].delta.content);
      }

      if (chunk.choices[0]?.finish_reason) {
        break;
      }
    }

    return response;
  } catch (error) {
    console.error("Streaming error:", error);
    throw error;
  }
};
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import sys

def update_ui(content: str) -> None:
    sys.stdout.write(content)  # replace with actual UI update logic

def robust_stream_processing(stream) -> str:
    try:
        response = ""
        for chunk in stream:
            if chunk.choices and chunk.choices[0].delta.content:
                response += chunk.choices[0].delta.content
                update_ui(chunk.choices[0].delta.content)

            if chunk.choices and chunk.choices[0].finish_reason:
                break

        return response
    except Exception as error:
        print(f"Streaming error: {error}", file=sys.stderr)
        raise
```

## Chamada de funções com streaming

Faça streaming das chamadas de ferramentas à medida que são geradas:

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const tools = [
  {
    type: "function" as const,
    function: {
      name: "get_weather",
      description: "Get current weather",
      parameters: {
        type: "object",
        properties: { location: { type: "string" } },
        required: ["location"],
      },
    },
  },
];

const stream = await client.chat.completions.create({
  model: "openai/gpt-5.4",
  messages: [{ role: "user", content: "What's the weather in Paris?" }],
  tools,
  stream: true,
});

for await (const chunk of stream) {
  if (!chunk.choices.length) continue;
  const delta = chunk.choices[0].delta;
  if (delta.tool_calls?.[0]?.function?.arguments) {
    process.stdout.write(delta.tool_calls[0].function.arguments);
  } else if (delta.content) {
    process.stdout.write(delta.content);
  }
}
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get current weather",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {"type": "string"}
                },
                "required": ["location"]
            }
        }
    }
]

stream = client.chat.completions.create(
    model="openai/gpt-5.4",
    messages=[{"role": "user", "content": "What's the weather in Paris?"}],
    tools=tools,
    stream=True
)

for chunk in stream:
    if not chunk.choices:
        continue
    # Handle tool calls
    if chunk.choices[0].delta.tool_calls:
        tool_call = chunk.choices[0].delta.tool_calls[0]
        if tool_call.function.arguments:
            print(tool_call.function.arguments, end="")

    # Handle regular content
    elif chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
```

## Exemplos de integração com a interface

### Hook do React para streaming

Encapsule o estado do streaming em um hook para que os componentes recebam `response` e `isStreaming` sem precisar gerenciar o loop de eventos.

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";
import { useState, useCallback } from "react";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const useStreamingChat = () => {
  const [response, setResponse] = useState("");
  const [isStreaming, setIsStreaming] = useState(false);

  const streamChat = useCallback(async (message) => {
    setIsStreaming(true);
    setResponse("");

    try {
      const stream = await client.chat.completions.create({
        model: "openai/gpt-5.4",
        messages: [{ role: "user", content: message }],
        stream: true,
      });

      for await (const chunk of stream) {
        const content = chunk.choices[0]?.delta?.content || "";
        if (content) {
          setResponse((prev) => prev + content);
        }

        if (chunk.choices[0]?.finish_reason) {
          setIsStreaming(false);
          break;
        }
      }
    } catch (error) {
      console.error("Streaming failed:", error);
      setIsStreaming(false);
    }
  }, []);

  return { response, isStreaming, streamChat };
};
```

**Server-Sent Events (navegador):**

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
const streamWithSSE = async (message: string): Promise<void> => {
  const response = await fetch("/api/chat-stream", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ message }),
  });

  if (!response.ok || !response.body) {
    throw new Error(`Request failed: ${response.status}`);
  }

  const reader = response.body.getReader();
  const decoder = new TextDecoder();
  const output = document.getElementById("response")!;
  let buffer = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const lines = buffer.split("\n");
    buffer = lines.pop() ?? "";

    for (const line of lines) {
      if (line === "data: [DONE]") break;
      if (!line.startsWith("data: ")) continue;
      const data = JSON.parse(line.slice(6));
      const content = data.choices[0]?.delta?.content;
      if (content) output.innerHTML += content;
    }
  }
};
```

## Otimização de desempenho

### Buffer de chunks

Agrupar chunks pequenos antes de enviá-los à interface reduz os ciclos de renderização e suaviza a saída percebida.

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
class StreamBuffer {
  private buffer: string;
  private flushInterval: number;
  private lastFlush: number;

  constructor(flushInterval = 50) {
    this.buffer = "";
    this.flushInterval = flushInterval;
    this.lastFlush = Date.now();
  }

  add(content: string): void {
    this.buffer += content;

    // Flush periodically or when buffer is large
    if (
      Date.now() - this.lastFlush > this.flushInterval ||
      this.buffer.length > 100
    ) {
      this.flush();
    }
  }

  flush(): void {
    if (this.buffer) {
      this.onFlush(this.buffer);
      this.buffer = "";
      this.lastFlush = Date.now();
    }
  }

  onFlush(content: string): void {
    // Override this method
    console.log(content);
  }
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
import time
import sys

class StreamBuffer:
    def __init__(self, flush_interval: float = 0.05):
        self._buffer = ""
        self._flush_interval = flush_interval
        self._last_flush = time.time()

    def add(self, content: str) -> None:
        self._buffer += content
        if time.time() - self._last_flush > self._flush_interval or len(self._buffer) > 100:
            self.flush()

    def flush(self) -> None:
        if self._buffer:
            self.on_flush(self._buffer)
            self._buffer = ""
            self._last_flush = time.time()

    def on_flush(self, content: str) -> None:
        sys.stdout.write(content)  # override in subclass
```

### Gerenciamento de memória

Em conclusões longas, limite o acúmulo para evitar crescimento ilimitado da memória.

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
const processLargeStream = async (stream, maxMemory = 1000000) => {
  let totalLength = 0;
  const chunks = [];

  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content || "";

    if (content) {
      totalLength += content.length;
      chunks.push(content);

      // Prevent memory overflow
      if (totalLength > maxMemory) {
        console.warn("Stream too large, truncating");
        break;
      }
    }

    if (chunk.choices[0]?.finish_reason) {
      break;
    }
  }

  return chunks.join("");
};
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import sys

def process_large_stream(stream, max_memory: int = 1_000_000) -> str:
    total_length = 0
    chunks = []

    for chunk in stream:
        content = chunk.choices[0].delta.content or "" if chunk.choices else ""

        if content:
            total_length += len(content)
            chunks.append(content)

            if total_length > max_memory:
                print("Stream too large, truncating", file=sys.stderr)
                break

        if chunk.choices and chunk.choices[0].finish_reason:
            break

    return "".join(chunks)
```

## Boas práticas

### Gerenciamento de streams

* Defina timeouts razoáveis (de 30 a 60 segundos).
* Implemente limites de erro adequados.
* Trate interrupções de rede de forma elegante.
* Ofereça ao usuário opções de cancelamento.

### Considerações de UI/UX

* Exiba indicadores de digitação durante o streaming.
* Permita que os usuários interrompam a geração.
* Use buffer para chunks pequenos e obtenha uma exibição mais suave.
* Trate atualizações rápidas com eficiência.

### Exemplo de recuperação de erros

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const streamWithRetry = async (input: string, maxRetries = 3) => {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      const stream = await client.responses.create({
        model: "openai/gpt-5.4",
        input,
        stream: true,
      });

      let fullResponse = "";
      for await (const event of stream) {
        if (event.type === "response.output_text.delta") {
          fullResponse += event.delta;
          process.stdout.write(event.delta);
        }
      }
      return fullResponse;
    } catch (error) {
      if (attempt === maxRetries) throw error;

      console.log(`Stream attempt ${attempt} failed, retrying...`);
      await new Promise((resolve) => setTimeout(resolve, 1000 * attempt));
    }
  }
};
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os
import sys
import time

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

def stream_responses_with_retry(input_text: str, max_retries: int = 3) -> str:
    for attempt in range(1, max_retries + 1):
        try:
            stream = client.responses.create(
                model="openai/gpt-5.4",
                input=input_text,
                stream=True,
            )
            full_response = ""
            for event in stream:
                if event.type == "response.output_text.delta":
                    full_response += event.delta
                    sys.stdout.write(event.delta)
                    sys.stdout.flush()
            return full_response
        except Exception:
            if attempt == max_retries:
                raise
            print(f"Stream attempt {attempt} failed, retrying...")
            time.sleep(attempt)
    return ""
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const streamWithRetry = async (messages: { role: string; content: string }[], maxRetries = 3) => {
  for (let attempt = 1; attempt <= maxRetries; attempt++) {
    try {
      const stream = await client.chat.completions.create({
        model: "openai/gpt-5.4",
        messages,
        stream: true,
      });

      let fullResponse = "";
      for await (const chunk of stream) {
        const content = chunk.choices[0]?.delta?.content || "";
        if (content) {
          fullResponse += content;
          process.stdout.write(content);
        }
      }
      return fullResponse;
    } catch (error) {
      if (attempt === maxRetries) throw error;

      console.log(`Stream attempt ${attempt} failed, retrying...`);
      await new Promise((resolve) => setTimeout(resolve, 1000 * attempt));
    }
  }
};
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os
import sys
import time

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

def stream_chat_with_retry(messages: list, max_retries: int = 3) -> str:
    for attempt in range(1, max_retries + 1):
        try:
            stream = client.chat.completions.create(
                model="openai/gpt-5.4",
                messages=messages,
                stream=True,
            )
            full_response = ""
            for chunk in stream:
                content = chunk.choices[0].delta.content or "" if chunk.choices else ""
                if content:
                    full_response += content
                    sys.stdout.write(content)
                    sys.stdout.flush()
            return full_response
        except Exception:
            if attempt == max_retries:
                raise
            print(f"Stream attempt {attempt} failed, retrying...")
            time.sleep(attempt)
    return ""
```

## Solução de problemas

**O stream é interrompido inesperadamente**

* Verifique a estabilidade da rede.
* Confira as configurações de timeout.
* Monitore se há limitação de taxa.
* Verifique os limites específicos do modelo.

**Desempenho lento do streaming**

* Otimize o processamento dos chunks.
* Reduza a frequência de esvaziamento do buffer.
* Verifique a latência da rede.
* Reavalie a escolha do modelo.

**400 em uma solicitação grande sem streaming**

Em solicitações para a Anthropic, uma solicitação sem streaming cujo `max_tokens` implique
mais de 10 minutos de geração retorna `400` com `type: "invalid_request_error"`.
Defina `stream: true` para saídas longas ou reduza `max_tokens`.

**Problemas de memória**

* Implemente limites de tamanho de chunk.
* Use parsers de streaming.
* Descarte os chunks já processados.
* Monitore o uso de memória.

## Limitações

Uma solicitação de modelo com streaming é executada por até 30 minutos; um `limits.max_execution_time` explícito reduz esse limite. Os clientes SDK aplicam o próprio timeout de solicitação à chamada inteira, então aumente `timeoutMs` ou `timeout_ms` antes de iniciar um stream longo. Consulte [Timeouts e retries](/docs/pt-BR/reference/client-libraries.md#timeouts-and-retries).

| Limitação | Impacto | Alternativa |
| - | - | - |
| **Interrupção de rede** | O stream é interrompido | Reenvie a solicitação; um stream interrompido não pode ser retomado |
| **Sobrecarga de processamento** | Pequeno custo de desempenho | Otimize o tratamento dos chunks |
| **Variações entre modelos** | Tamanhos de chunk diferentes | Trate chunks de tamanho variável |
| **Limitação de taxa** | Throttling do stream | Implemente estratégias de backoff |

Um stream interrompido não pode ser retomado: nenhum dos endpoints oferece cursor de retomada ou replay de eventos. Reenvie a solicitação. Na **Responses API**, se o servidor concluiu a resposta antes da interrupção e ela ainda está armazenada, continue a partir dela com `previous_response_id`. Para trabalhos que precisam sobreviver a uma desconexão, use o [modo em segundo plano](/docs/pt-BR/ai-gateway/features/background-responses.md) e consulte o ID da resposta em vez de usar streaming.

## Recursos avançados

> [!NOTE]
>
> Os exemplos desta seção usam o endpoint de Chat Completions. Os mesmos padrões se aplicam à Responses API: substitua `chat.completions.create(...)` por `responses.create(...)`. Para cURL, use `/v3/router/responses`.

### Streaming com outros recursos do Gateway

Recursos do **AI Gateway** como cache, timeouts e nomes de deployment se combinam diretamente com o streaming: adicione-os ao mesmo objeto da solicitação.

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const advancedStream = await client.chat.completions.create({
  model: "openai/gpt-5.4",
  messages: [{ role: "user", content: "Explain machine learning" }],
  stream: true,
  name: "StreamingBot-v1",
  cache: { type: "exact_match", ttl: 3600 },
  timeout: { call_timeout: 30000 },
});
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

stream = client.chat.completions.create(
    model="openai/gpt-5.4",
    messages=[{"role": "user", "content": "Explain machine learning"}],
    stream=True,
    extra_body={
        "name": "StreamingBot-v1",
        "cache": {"type": "exact_match", "ttl": 3600},
        "timeout": {"call_timeout": 30000},
    },
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

### Streaming paralelo

Dispare vários streams simultaneamente usando `Promise.all` em TypeScript e `asyncio.gather` em Python para obter respostas independentes sem esperar que cada uma termine.

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const processQuery = async (query) => {
  const stream = await client.chat.completions.create({
    model: "openai/gpt-5.4",
    messages: [{ role: "user", content: query }],
    stream: true,
  });
  let fullResponse = "";
  for await (const chunk of stream) {
    const content = chunk.choices[0]?.delta?.content || "";
    if (content) fullResponse += content;
  }
  return fullResponse;
};

const parallelStreaming = async (queries) => Promise.all(queries.map(processQuery));
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import AsyncOpenAI
import asyncio
import os

client = AsyncOpenAI(
    api_key=os.environ.get("TOQAN_API_KEY"),
    base_url="https://router.toqan.ai/eu/v3/router",
)

async def process_query(query: str) -> str:
    stream = await client.chat.completions.create(
        model="openai/gpt-5.4",
        messages=[{"role": "user", "content": query}],
        stream=True,
    )
    full_response = ""
    async for chunk in stream:
        content = chunk.choices[0].delta.content or "" if chunk.choices else ""
        if content:
            full_response += content
    return full_response

async def parallel_streaming(queries: list[str]) -> list[str]:
    return await asyncio.gather(*[process_query(query) for query in queries])
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.