964 words
5 minutes
A Lightweight SSE Wrapper for Streaming LLM Responses

Demo#

Background#

I’ve been building small, fun tools on top of large language models lately — things like a cyber fortune teller and a celebrity biography generator. LLM responses can be pretty long, and dumping the whole thing on the user in one shot makes for a poor experience. SSE is the natural fit here: stream the tokens as they arrive, and the page fills in progressively.

What Is SSE?#

SSE (Server-Sent Events) is a one-way server push technique:

server → client

  • Long-lived connection (the HTTP connection stays open)
  • Data is pushed incrementally, a good match for streaming responses (e.g. tokens coming back from an LLM)

How it differs from WebSocket:

FeatureSSEWebSocket
DirectionOne-way (server to client)Bidirectional
Best forStreaming dataReal-time chat, multi-user collaboration
CompatibilityAll modern browsersRequires support on both server and client
ConnectionsCapped by the browserEffectively unlimited

How SSE Fits into an LLM App#

The use case: conversational models like ChatGPT, DeepSeek Chat, or Gemini transmit their answer token by token instead of returning one big JSON blob.

The flow#

  1. The client initiates the request
    • Opens an SSE connection with EventSource (browser) or fetch + ReadableStream (Node.js)
  2. The server streams data back
    • The server keeps pushing tokens into the SSE connection
  3. The client parses chunks as they arrive
    • Browser or front-end code listens for SSE events and renders the message progressively

SSE Code Snippets#

Server side, in Node.js:

import OpenAI from 'openai';
const openai = new OpenAI({
baseURL: 'https://api.deepseek.com',
apiKey: process.env.DEEPSEEK_KEY,
});
const stream = await openai.chat.completions.create({
model: "deepseek-chat",
messages: [{ role: "user", content: "Hello, world!" }],
stream: true,
});

Client side, in JavaScript:

const stream = await fetch('http://localhost:3000/sse', {
method: 'GET',
}).then(res => res.body);
const reader = stream?.getReader();
const decoder = new TextDecoder();
while (true) {
const { done, value } = await reader!.read();
if (done) break;
const text = decoder.decode(value);
console.log(text);
}

A Small Wrapper#

Based on the SSE data an LLM returns, here is a thin wrapper that handles streaming transfer and page rendering, including support for reasoning models.

Server Side#

The function#

// @/utils/server.ts
/**
* Get an LLM response stream
* @param completion the response stream returned by service.chat.completions.create with create({ stream: true })
* @param model model name
* @param type type
* @returns the stream
*/
export const getStreamData = (completion: Stream<ChatCompletionChunk>) => {
let count = 0;
let thinkingCount = 0;
const stream = new ReadableStream({
async start(controller) {
try {
console.log('Starting stream processing...');
for await (const chunk of completion) {
const { choices } = chunk;
const delta = choices[0]?.delta as Delta ?? {};
if (!delta) continue;
const { reasoning_content = null, content = null } = delta;
// open the thinking tag
if (count === 0 && reasoning_content) {
controller.enqueue(new TextEncoder().encode('<thinking>'));
// wait 100ms to avoid batching
await new Promise(resolve => setTimeout(resolve, 100));
}
if (reasoning_content) {
count++;
thinkingCount++;
controller.enqueue(new TextEncoder().encode(reasoning_content));
}
if (count - thinkingCount === 0 && thinkingCount !== 0 && content) {
controller.enqueue(new TextEncoder().encode('</thinking>'));
// wait 100ms to avoid batching
await new Promise(resolve => setTimeout(resolve, 100));
}
if (content) {
count++;
controller.enqueue(new TextEncoder().encode(content));
}
}
} catch (error) {
console.error('Stream processing error:', error);
controller.error(error);
} finally {
if (count === 0) {
controller.enqueue(new TextEncoder().encode('[服务器繁忙,请稍后再试。]'));
} else {
controller.enqueue(new TextEncoder().encode('[DONE]'));
}
controller.close();
}
},
});
return stream;
};

Usage#

import OpenAI from 'openai';
import { getStreamData } from '@/utils/server';
const openai = new OpenAI({
baseURL: 'https://api.deepseek.com',
apiKey: process.env.DEEPSEEK_KEY,
});
const completion = await openai.chat.completions.create({
model: "deepseek-chat",
messages: [{ role: "user", content: "Hello, world!" }],
stream: true,
});
const stream = await getStreamData(completion);
// return the stream
return new Response(stream, {
headers: {
'Content-Type': 'text/event-stream',
},
});

Client Side#

The function#

// @/utils/client.ts
/**
* Parse a stream
* @param stream the ReadableStream to parse
* @param options options
* @param options.output output type, defaults to text; if output is json, onResult receives the parsed JSON object as result
* @param options.onStart called when the stream starts
* @param options.onEnd called when the stream ends; thinking is the reasoning content, result is the final result
* @param options.onchange called on every update; thinking is the reasoning content, result is the latest result
* @param options.onThinkingStart fired when the reasoning phase starts (reasoning models only)
* @param options.onThinkingEnd fired when the reasoning phase ends (reasoning models only)
*/
export const parseReadableStream = async (stream: ReadableStream<Uint8Array<ArrayBufferLike>>, options: {
output?: 'text' | 'json';
onStart?: () => void;
onEnd?: (thinking: string, result: string | Record<string, unknown>) => void;
onchange?: (thinking: string, result: string) => void;
onThinkingStart?: () => void;
onThinkingEnd?: () => void;
}) => {
const { output = 'text', onStart = () => {}, onEnd = () => {}, onThinkingStart = () => {}, onThinkingEnd = () => {}, onchange = () => {} } = options;
const reader = stream?.getReader();
const decoder = new TextDecoder();
let isReasoning = false;
let thinking = '';
let content = '';
onStart();
while (true) {
const { done, value } = await reader!.read();
if (done) break;
const text = decoder.decode(value);
if (text.includes('<thinking>')) {
isReasoning = true;
onThinkingStart();
continue;
}
if (text.includes('</thinking>')) {
isReasoning = false;
onThinkingEnd();
continue;
}
if (text.includes('[DONE]')) {
let result = content;
if (output === 'json') {
// extract the content between the json fences
const jsonContent = result.match(/```json\s*([\s\S]*?)\s*```/)?.[1] || '';
try {
result = JSON.parse(jsonContent);
} catch (error) {
console.error(error);
}
}
onEnd(thinking, result);
break;
}
if (isReasoning) {
thinking += text;
} else {
content += text;
}
onchange(thinking, content);
}
};

Usage#

import { parseReadableStream } from '@/utils/client';
const stream = await fetch('http://localhost:3000/sse', {
method: 'GET',
}).then(res => res.body);
try {
parseReadableStream(stream, {
output: 'json',
onStart: () => {
console.log('started');
},
onEnd: (thinking, result) => {
console.log('finished', thinking, result);
},
onchange: (thinking, result) => {
// when output is json, result is the already-parsed object
console.log('update', thinking, result);
},
onThinkingStart: () => {
console.log('thinking started');
},
onThinkingEnd: () => {
console.log('thinking finished');
},
});
} catch (error) {
console.error(error);
}
A Lightweight SSE Wrapper for Streaming LLM Responses
https://www.mihouo.com/en/posts/front/sse-for-streaming-data-processing-in-large-language-models/
Author
Jia
Published at
2025-04-01
License
CC BY-NC-SA 4.0