Demo
Background
I’ve been building small, fun tools on top of large language models lately — things like a cyber fortune teller and a celebrity biography generator. LLM responses can be pretty long, and dumping the whole thing on the user in one shot makes for a poor experience. SSE is the natural fit here: stream the tokens as they arrive, and the page fills in progressively.
What Is SSE?
SSE (Server-Sent Events) is a one-way server push technique:
server → client
- Long-lived connection (the HTTP connection stays open)
- Data is pushed incrementally, a good match for streaming responses (e.g. tokens coming back from an LLM)
How it differs from WebSocket:
| Feature | SSE | WebSocket |
|---|---|---|
| Direction | One-way (server to client) | Bidirectional |
| Best for | Streaming data | Real-time chat, multi-user collaboration |
| Compatibility | All modern browsers | Requires support on both server and client |
| Connections | Capped by the browser | Effectively unlimited |
How SSE Fits into an LLM App
The use case: conversational models like ChatGPT, DeepSeek Chat, or Gemini transmit their answer token by token instead of returning one big JSON blob.
The flow
- The client initiates the request
- Opens an SSE connection with EventSource (browser) or fetch + ReadableStream (Node.js)
- The server streams data back
- The server keeps pushing tokens into the SSE connection
- The client parses chunks as they arrive
- Browser or front-end code listens for SSE events and renders the message progressively
SSE Code Snippets
Server side, in Node.js:
import OpenAI from 'openai';
const openai = new OpenAI({ baseURL: 'https://api.deepseek.com', apiKey: process.env.DEEPSEEK_KEY,});
const stream = await openai.chat.completions.create({ model: "deepseek-chat", messages: [{ role: "user", content: "Hello, world!" }], stream: true,});Client side, in JavaScript:
const stream = await fetch('http://localhost:3000/sse', { method: 'GET',}).then(res => res.body);
const reader = stream?.getReader();const decoder = new TextDecoder();while (true) { const { done, value } = await reader!.read(); if (done) break; const text = decoder.decode(value); console.log(text);}A Small Wrapper
Based on the SSE data an LLM returns, here is a thin wrapper that handles streaming transfer and page rendering, including support for reasoning models.
Server Side
The function
// @/utils/server.ts
/** * Get an LLM response stream * @param completion the response stream returned by service.chat.completions.create with create({ stream: true }) * @param model model name * @param type type * @returns the stream */export const getStreamData = (completion: Stream<ChatCompletionChunk>) => { let count = 0; let thinkingCount = 0; const stream = new ReadableStream({ async start(controller) { try { console.log('Starting stream processing...'); for await (const chunk of completion) { const { choices } = chunk; const delta = choices[0]?.delta as Delta ?? {};
if (!delta) continue;
const { reasoning_content = null, content = null } = delta; // open the thinking tag if (count === 0 && reasoning_content) { controller.enqueue(new TextEncoder().encode('<thinking>')); // wait 100ms to avoid batching await new Promise(resolve => setTimeout(resolve, 100)); } if (reasoning_content) { count++; thinkingCount++; controller.enqueue(new TextEncoder().encode(reasoning_content)); } if (count - thinkingCount === 0 && thinkingCount !== 0 && content) { controller.enqueue(new TextEncoder().encode('</thinking>')); // wait 100ms to avoid batching await new Promise(resolve => setTimeout(resolve, 100)); } if (content) { count++; controller.enqueue(new TextEncoder().encode(content)); } } } catch (error) { console.error('Stream processing error:', error); controller.error(error); } finally { if (count === 0) { controller.enqueue(new TextEncoder().encode('[服务器繁忙,请稍后再试。]')); } else { controller.enqueue(new TextEncoder().encode('[DONE]')); } controller.close(); } }, }); return stream;};Usage
import OpenAI from 'openai';import { getStreamData } from '@/utils/server';
const openai = new OpenAI({ baseURL: 'https://api.deepseek.com', apiKey: process.env.DEEPSEEK_KEY,});
const completion = await openai.chat.completions.create({ model: "deepseek-chat", messages: [{ role: "user", content: "Hello, world!" }], stream: true,});const stream = await getStreamData(completion);
// return the streamreturn new Response(stream, { headers: { 'Content-Type': 'text/event-stream', },});Client Side
The function
// @/utils/client.ts
/** * Parse a stream * @param stream the ReadableStream to parse * @param options options * @param options.output output type, defaults to text; if output is json, onResult receives the parsed JSON object as result * @param options.onStart called when the stream starts * @param options.onEnd called when the stream ends; thinking is the reasoning content, result is the final result * @param options.onchange called on every update; thinking is the reasoning content, result is the latest result * @param options.onThinkingStart fired when the reasoning phase starts (reasoning models only) * @param options.onThinkingEnd fired when the reasoning phase ends (reasoning models only) */export const parseReadableStream = async (stream: ReadableStream<Uint8Array<ArrayBufferLike>>, options: { output?: 'text' | 'json'; onStart?: () => void; onEnd?: (thinking: string, result: string | Record<string, unknown>) => void; onchange?: (thinking: string, result: string) => void; onThinkingStart?: () => void; onThinkingEnd?: () => void;}) => { const { output = 'text', onStart = () => {}, onEnd = () => {}, onThinkingStart = () => {}, onThinkingEnd = () => {}, onchange = () => {} } = options; const reader = stream?.getReader(); const decoder = new TextDecoder(); let isReasoning = false; let thinking = ''; let content = ''; onStart(); while (true) { const { done, value } = await reader!.read(); if (done) break; const text = decoder.decode(value); if (text.includes('<thinking>')) { isReasoning = true; onThinkingStart(); continue; } if (text.includes('</thinking>')) { isReasoning = false; onThinkingEnd(); continue; } if (text.includes('[DONE]')) { let result = content; if (output === 'json') { // extract the content between the json fences const jsonContent = result.match(/```json\s*([\s\S]*?)\s*```/)?.[1] || ''; try { result = JSON.parse(jsonContent); } catch (error) { console.error(error); } } onEnd(thinking, result); break; } if (isReasoning) { thinking += text; } else { content += text; } onchange(thinking, content); }};Usage
import { parseReadableStream } from '@/utils/client';
const stream = await fetch('http://localhost:3000/sse', { method: 'GET',}).then(res => res.body);
try { parseReadableStream(stream, { output: 'json', onStart: () => { console.log('started'); }, onEnd: (thinking, result) => { console.log('finished', thinking, result); }, onchange: (thinking, result) => { // when output is json, result is the already-parsed object console.log('update', thinking, result); }, onThinkingStart: () => { console.log('thinking started'); }, onThinkingEnd: () => { console.log('thinking finished'); }, });} catch (error) { console.error(error);}