Streaming

Receive tokens as they're generated using Server-Sent Events (SSE). Both SDKs support streaming with the same stream=true flag.

OpenAI

UltraFast + streaming

Pair any ultrafast model with stream=true for 1,800 TPS sustained. First token in <50ms; full response typically under 1s for short prompts.

Python

with client.chat.completions.stream(    model="openai/gpt-oss-120b",  # ⚡ UltraFast — 500 TPS    messages=[{"role": "user", "content": "Tell me a story"}],) as stream:    for text in stream.text_stream:        print(text, end="", flush=True)

Anthropic

Python

with client.messages.stream(    model="claude-3-5-sonnet-20241022",    max_tokens=1024,    messages=[{"role": "user", "content": "Tell me a story"}],) as stream:    for text in stream.text_stream:        print(text, end="", flush=True)

UltraFast + streaming

UltraFast + streaming is the ideal combination for real-time chat UIs, voice interfaces, and interactive code editors. GPT OSS 20B and Claude 3.5 Haiku can stream at 1,000+ tokens/second on the LPU tier.