> ## Documentation Index
> Fetch the complete documentation index at: /docs/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI SDK drop-in integration

> Use the AI Gateway with OpenAI SDK. Replace OpenAI base URL with Toqan Router for routing, caching, monitoring, and multi-provider fallback capabilities.

> [card] [AI Gateway](#ai-gateway)
>
> Route your LLM calls through the AI Gateway with a single base URL change. Zero vendor lock-in: always run on the best model at the lowest cost for your use case.

> [card] [Observability](#observability)
>
> Instrument your code with OpenTelemetry to capture traces, logs, and metrics for every LLM call, agent step, and tool use.

## AI Gateway

### Overview

The OpenAI SDK provides powerful tools for building AI applications with GPT models. By connecting the SDK to Toqan Router's AI Gateway, you transform your OpenAI integration into a production-ready system with enterprise-grade capabilities, complete observability, and access to 500+ models across 30+ providers.

### Key Benefits

Toqan Router's AI Gateway enhances your OpenAI SDK with:

> [card] Complete Observability
>
> Track every API call, token usage, and model interaction with detailed traces and analytics

> [card] Built-in Reliability
>
> Automatic fallbacks, retries, and load balancing for production resilience

> [card] Cost Optimization
>
> Real-time cost tracking and spend management across all your AI operations

> [card] Multi-Provider Access
>
> Access 500+ LLMs and 30+ providers through a single, unified integration

### Prerequisites

Before integrating OpenAI SDK with Toqan Router, ensure you have:

* A Toqan Router account and API Key
* Python 3.8+ or Node.js 18+ with TypeScript support
* OpenAI SDK installed

> [!NOTE]
>
> To setup your API key, see API keys & Endpoints.

> [!NOTE]
>
> To use libraries with private models, see [Onboarding Private Models](/docs/ai-gateway/using-the-router.md#onboarding-private-models).

> [!TIP]
>
> For a step-by-step migration of an existing OpenAI SDK application, see [Migrate to Toqan Router](/docs/ai-gateway/get-started/migrate.md).

### Installation

```bash Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
pip install openai
```

```bash TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
npm install openai
```

### Configuration

While using the OpenAI SDK, set the **Base URL** to the [AI Gateway](/docs/ai-gateway/using-the-router.md) to feed calls through our API without changing any other part of your code.

Using the **Toqan Router AI Gateway**, you benefit from Platform [Traces](/docs/ai-studio/observability/traces.md) and [Cost and Usage Monitoring](/docs/ai-studio/observability/quickstart.md), keeping full compatibility and a unified API with all models while using the OpenAI SDK.

> **base\_url**: `https://router.toqan.ai/eu/v3/router`

### Text Generation

Basic text generation with the OpenAI SDK through Toqan Router:

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const response = await openai.responses.create({
  model: "openai/gpt-5.6-sol",
  instructions: "You are a helpful assistant.",
  input: "Hello!",
});

console.log(response.output_text);
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

response = client.responses.create(
    model="openai/gpt-5.6-sol",
    instructions="You are a helpful assistant.",
    input="Hello!",
)

print(response.output_text)
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const completion = await openai.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "Hello!" },
  ],
});

console.log(completion.choices[0].message.content);
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

completion = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello!"},
    ],
)

print(completion.choices[0].message.content)
```

### Streaming Responses

Stream responses for real-time output:

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const stream = await openai.responses.create({
  model: "openai/gpt-5.6-sol",
  input: "Write a short story about robots.",
  stream: true,
});

for await (const event of stream) {
  if (event.type === "response.output_text.delta") {
    process.stdout.write(event.delta);
  }
}
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

stream = client.responses.create(
    model="openai/gpt-5.6-sol",
    input="Write a short story about robots.",
    stream=True,
)

for event in stream:
    if event.type == "response.output_text.delta":
        print(event.delta, end="", flush=True)
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const stream = await openai.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [{ role: "user", content: "Write a short story about robots." }],
  stream: true,
});

for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content || "");
}
```

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

stream = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[{"role": "user", "content": "Write a short story about robots."}],
    stream=True,
)

for chunk in stream:
    if chunk.choices and chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="", flush=True)
```

### Model Selection

With Toqan Router, you can use any supported model from 30+ providers:

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const openai = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});

const claudeResponse = await openai.responses.create({
  model: "anthropic/claude-sonnet-5",
  input: "Explain machine learning",
});

const geminiResponse = await openai.responses.create({
  model: "google/gemini-2.5-flash",
  input: "Explain machine learning",
});

const groqResponse = await openai.responses.create({
  model: "groq/qwen/qwen3.8-27b",
  input: "Explain machine learning",
});
```

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

claude_response = client.responses.create(
    model="anthropic/claude-sonnet-5",
    input="Explain machine learning",
)

gemini_response = client.responses.create(
    model="google/gemini-2.5-flash",
    input="Explain machine learning",
)

groq_response = client.responses.create(
    model="groq/qwen/qwen3.8-27b",
    input="Explain machine learning",
)
```

## Observability

### Getting Started

Integrate OpenAI SDK with Toqan Router's observability to gain complete insights into model performance, token usage, API latency, and conversation flows using OpenTelemetry.

### Prerequisites

Before you begin, ensure you have:

* A Toqan Router account and API Key
* Python 3.8+ or Node.js 18+
* OpenAI SDK installed

### Install Dependencies

```bash Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
# Core OpenTelemetry packages
pip install opentelemetry-sdk opentelemetry-instrumentation opentelemetry-exporter-otlp

# OpenAI SDK
pip install openai

# OpenAI Auto-Instrumentation
pip install opentelemetry-instrumentation-openai
```

```bash TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
# OpenTelemetry SDK (includes all core packages)
npm install @opentelemetry/sdk-node @opentelemetry/api @opentelemetry/exporter-trace-otlp-http

# OpenAI SDK
npm install openai

# OpenAI Auto-Instrumentation
npm install @opentelemetry/instrumentation-openai
```

### Configure Toqan Router

Set up your environment variables to connect to Toqan Router's OpenTelemetry collector:

**Unix/Linux/macOS:**

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export OTEL_EXPORTER_OTLP_ENDPOINT="https://router.toqan.ai/eu/v2/otel"
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer $TOQAN_API_KEY"
export OTEL_RESOURCE_ATTRIBUTES="service.name=openai-app,service.version=1.0.0"
export OTEL_EXPORTER_OTLP_TRACES_PROTOCOL="http/json"
```

**Windows (PowerShell):**

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "https://router.toqan.ai/eu/v2/otel"
$env:OTEL_EXPORTER_OTLP_HEADERS = "Authorization=Bearer <TOQAN_API_KEY>"
$env:OTEL_RESOURCE_ATTRIBUTES = "service.name=openai-app,service.version=1.0.0"
$env:OTEL_EXPORTER_OTLP_TRACES_PROTOCOL = "http/json"
```

**Using .env file:**

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
OTEL_EXPORTER_OTLP_ENDPOINT=https://router.toqan.ai/eu/v2/otel
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer <TOQAN_API_KEY>
OTEL_RESOURCE_ATTRIBUTES=service.name=openai-app,service.version=1.0.0
OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json
```

### Setup

Configure OpenTelemetry once at application startup:

```python Python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.instrumentation.openai import OpenAIInstrumentor
import os

# Configure OpenTelemetry
tracer_provider = TracerProvider()
tracer_provider.add_span_processor(BatchSpanProcessor(OTLPSpanExporter()))
trace.set_tracer_provider(tracer_provider)

# Instrument OpenAI
OpenAIInstrumentor().instrument(tracer_provider=tracer_provider)

# Create OpenAI client
client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY")
)
```

```typescript TypeScript theme={"theme":{"light":"github-light","dark":"github-dark"}}
import { NodeSDK } from "@opentelemetry/sdk-node";
import { OTLPTraceExporter } from "@opentelemetry/exporter-trace-otlp-http";
import { OpenAIInstrumentation } from "@opentelemetry/instrumentation-openai";
import OpenAI from "openai";

// Initialize SDK
const sdk = new NodeSDK({
  traceExporter: new OTLPTraceExporter(),
  instrumentations: [new OpenAIInstrumentation()],
});

sdk.start();

// Graceful shutdown
process.on("SIGTERM", () => {
  sdk.shutdown()
    .then(() => console.log("Tracing terminated"))
    .catch((error) => console.log("Error terminating tracing", error))
    .finally(() => process.exit(0));
});

// Create OpenAI client
const client = new OpenAI({
  baseURL: "https://router.toqan.ai/eu/v3/router",
  apiKey: process.env.TOQAN_API_KEY,
});
```

### Examples

> [!NOTE]
>
> The examples below use the Chat Completions endpoint. OpenTelemetry instrumentation works identically with the Responses API: replace `client.chat.completions.create(...)` with `client.responses.create(...)`.

#### Basic Example

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

response = client.chat.completions.create(
    model="openai/gpt-5.6-sol",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "What is quantum computing in one sentence?"}
    ]
)

print(response.choices[0].message.content)
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const response = await client.chat.completions.create({
  model: "openai/gpt-5.6-sol",
  messages: [
    { role: "system", content: "You are a helpful assistant." },
    { role: "user", content: "What is quantum computing in one sentence?" },
  ],
});

console.log(response.choices[0].message.content);
```

#### Streaming Example

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
from opentelemetry import trace
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

tracer = trace.get_tracer(__name__)
with tracer.start_as_current_span("streaming-completion") as span:
    stream = client.chat.completions.create(
        model="openai/gpt-5.6-sol",
        messages=[{"role": "user", "content": "Write a haiku about code."}],
        stream=True
    )

    full_response = ""
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            content = chunk.choices[0].delta.content
            full_response += content
            print(content, end="", flush=True)

    span.set_attribute("response.length", len(full_response))
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";
import { trace } from "@opentelemetry/api";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const tracer = trace.getTracer("openai-app");

await tracer.startActiveSpan("streaming-completion", async (span) => {
  try {
    const stream = await client.chat.completions.create({
      model: "openai/gpt-5.6-sol",
      messages: [{ role: "user", content: "Write a haiku about code." }],
      stream: true,
    });

    let fullResponse = "";
    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content;
      if (content) {
        fullResponse += content;
        process.stdout.write(content);
      }
    }

    span.setAttribute("response.length", fullResponse.length);
  } finally {
    span.end();
  }
});
```

#### Custom Spans Example

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
from opentelemetry import trace
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

tracer = trace.get_tracer(__name__)

def analyze_document(document: str):
    with tracer.start_as_current_span("document-analysis") as span:
        span.set_attribute("document.length", len(document))

        # Prepare prompt
        prompt = f"Analyze this text: {document}"
        with tracer.start_as_current_span("prepare-prompt") as prep_span:
            prep_span.set_attribute("prompt.length", len(prompt))

        # Model inference
        with tracer.start_as_current_span("model-inference") as inf_span:
            inf_span.set_attribute("model", "gpt-5.6-sol")

            response = client.chat.completions.create(
                model="openai/gpt-5.6-sol",
                messages=[{"role": "user", "content": prompt}]
            )

            inf_span.set_attribute("tokens.total", response.usage.total_tokens)

        # Process result
        with tracer.start_as_current_span("process-result") as proc_span:
            result = response.choices[0].message.content
            proc_span.set_attribute("result.length", len(result))

        return result

analysis = analyze_document("Machine learning is a subset of AI.")
print(analysis)
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";
import { trace } from "@opentelemetry/api";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const tracer = trace.getTracer("openai-app");

async function analyzeDocument(document: string) {
  return await tracer.startActiveSpan("document-analysis", async (span) => {
    try {
      span.setAttribute("document.length", document.length);

      // Prepare prompt
      const prompt = `Analyze this text: ${document}`;
      await tracer.startActiveSpan("prepare-prompt", async (prepSpan) => {
        try {
          prepSpan.setAttribute("prompt.length", prompt.length);
        } finally {
          prepSpan.end();
        }
      });

      // Model inference
      let response: Awaited<ReturnType<typeof client.chat.completions.create>>;
      await tracer.startActiveSpan("model-inference", async (infSpan) => {
        try {
          infSpan.setAttribute("model", "gpt-5.6-sol");

          response = await client.chat.completions.create({
            model: "openai/gpt-5.6-sol",
            messages: [{ role: "user", content: prompt }],
          });

          infSpan.setAttribute("tokens.total", response.usage?.total_tokens || 0);
        } finally {
          infSpan.end();
        }
      });

      // Process result
      await tracer.startActiveSpan("process-result", async (procSpan) => {
        try {
          const result = response!.choices[0].message.content || "";
          procSpan.setAttribute("result.length", result.length);
        } finally {
          procSpan.end();
        }
      });

      return response!.choices[0].message.content;
    } finally {
      span.end();
    }
  });
}

analyzeDocument("Machine learning is a subset of AI.")
  .then((analysis) => console.log(analysis));
```

#### Advanced Workflows Example

```python Python (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
from openai import OpenAI
from opentelemetry import trace
import os

client = OpenAI(
    base_url="https://router.toqan.ai/eu/v3/router",
    api_key=os.environ.get("TOQAN_API_KEY"),
)

tracer = trace.get_tracer(__name__)

def content_generation_pipeline(topic: str):
    with tracer.start_as_current_span("content-pipeline") as pipeline_span:
        pipeline_span.set_attribute("pipeline.topic", topic)
        pipeline_span.set_attribute("pipeline.stages", 3)

        # Stage 1: Research
        with tracer.start_as_current_span("stage-research") as research_span:
            research_span.set_attribute("stage.name", "research")

            research = client.chat.completions.create(
                model="openai/gpt-5.6-sol",
                messages=[{"role": "user", "content": f"List 3 key facts about {topic}."}]
            )

            facts = research.choices[0].message.content
            research_span.set_attribute("facts.count", 3)
            research_span.set_attribute("tokens.used", research.usage.total_tokens)

        # Stage 2: Writing
        with tracer.start_as_current_span("stage-writing") as writing_span:
            writing_span.set_attribute("stage.name", "writing")

            writing = client.chat.completions.create(
                model="openai/gpt-5.6-sol",
                messages=[{"role": "user", "content": f"Write a brief introduction using these facts: {facts}"}]
            )

            content = writing.choices[0].message.content
            writing_span.set_attribute("content.length", len(content))

        # Stage 3: Review
        with tracer.start_as_current_span("stage-review") as review_span:
            review_span.set_attribute("stage.name", "review")

            review = client.chat.completions.create(
                model="openai/gpt-5.6-sol",
                messages=[{"role": "user", "content": f"Rate this content quality 1-10: {content}"}]
            )

            rating = review.choices[0].message.content
            review_span.set_attribute("quality.rating", rating)

        pipeline_span.set_attribute("pipeline.success", True)
        return {"content": content, "rating": rating}

result = content_generation_pipeline("neural networks")
print(f"Content: {result['content']}")
print(f"Rating: {result['rating']}")
```

```typescript TypeScript (Chat Completions) theme={"theme":{"light":"github-light","dark":"github-dark"}}
import OpenAI from "openai";
import { trace } from "@opentelemetry/api";

const client = new OpenAI({
  apiKey: process.env.TOQAN_API_KEY,
  baseURL: "https://router.toqan.ai/eu/v3/router",
});

const tracer = trace.getTracer("openai-app");

async function contentGenerationPipeline(topic: string) {
  return await tracer.startActiveSpan("content-pipeline", async (pipelineSpan) => {
    try {
      pipelineSpan.setAttribute("pipeline.topic", topic);
      pipelineSpan.setAttribute("pipeline.stages", 3);

      let facts = "";
      let content = "";
      let rating = "";

      // Stage 1: Research
      await tracer.startActiveSpan("stage-research", async (researchSpan) => {
        try {
          researchSpan.setAttribute("stage.name", "research");

          const research = await client.chat.completions.create({
            model: "openai/gpt-5.6-sol",
            messages: [{ role: "user", content: `List 3 key facts about ${topic}.` }],
          });

          facts = research.choices[0].message.content || "";
          researchSpan.setAttribute("facts.count", 3);
          researchSpan.setAttribute("tokens.used", research.usage?.total_tokens || 0);
        } finally {
          researchSpan.end();
        }
      });

      // Stage 2: Writing
      await tracer.startActiveSpan("stage-writing", async (writingSpan) => {
        try {
          writingSpan.setAttribute("stage.name", "writing");

          const writing = await client.chat.completions.create({
            model: "openai/gpt-5.6-sol",
            messages: [{ role: "user", content: `Write a brief introduction using these facts: ${facts}` }],
          });

          content = writing.choices[0].message.content || "";
          writingSpan.setAttribute("content.length", content.length);
        } finally {
          writingSpan.end();
        }
      });

      // Stage 3: Review
      await tracer.startActiveSpan("stage-review", async (reviewSpan) => {
        try {
          reviewSpan.setAttribute("stage.name", "review");

          const review = await client.chat.completions.create({
            model: "openai/gpt-5.6-sol",
            messages: [{ role: "user", content: `Rate this content quality 1-10: ${content}` }],
          });

          rating = review.choices[0].message.content || "";
          reviewSpan.setAttribute("quality.rating", rating);
        } finally {
          reviewSpan.end();
        }
      });

      pipelineSpan.setAttribute("pipeline.success", true);
      return { content, rating };
    } finally {
      pipelineSpan.end();
    }
  });
}

contentGenerationPipeline("neural networks")
  .then((result) => {
    console.log(`Content: ${result.content}`);
    console.log(`Rating: ${result.rating}`);
  });
```

### View Traces

View your traces in the [AI Studio](/docs/ai-studio/observability/quickstart.md) in the **Traces** tab.

> [!NOTE]
>
> Visit your [AI Studio](https://router.toqan.ai/eu) to view real-time analytics and traces.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.