1129 words
6 minutes
Making the Wait Visible — Streaming Output Optimization for an AI Naming Feature

Feature Background#

In a mini-program, users fill in their baby’s details (surname, gender, birth time), tap “Start Smart Naming”, and expect to quickly receive a set of name suggestions that are directly usable and clearly explained.

But naming isn’t just “spitting out a few characters”. To make the suggestions trustworthy and easy to act on, the page has to render several modules:

  • BaZi analysis and naming direction: the overall approach, shown first
  • Multiple name suggestions: usually 5–10
  • A detailed explanation for each name: meaning, Five-Elements balance, how it sounds, an overall score, and so on

(BaZi, 八字, is the traditional Chinese “eight characters” birth chart analysis.)

The more complete the content, the easier the decision — and the longer the generation time.

Problems with the Original Approach#

Before the redesign, the delivery model was “return everything in one payload”:

  1. User taps “Start Naming”
  2. The page enters a loading state
  3. The backend calls an LLM to generate the complete structured JSON
  4. Only after the entire JSON is generated is it returned to the frontend
  5. The page receives the full result and renders every module at once

Straightforward engineering-wise — but from the user’s perspective, the whole process was a black box.

What Users Actually Feel#

Users aren’t afraid of slow. They’re afraid of slow with nothing to look at.

In the old flow, all users saw was a spinner — no progress, no feedback, no hint of how much longer it would take.

That led to a few classic problems:

  • “Is it stuck?”: users tapped the button repeatedly, generating duplicate requests
  • Giving up early: no visible hope, so they just left
  • Perception amplification: even though the backend was generating away, users had no way to know

The Root of the Problem#

It wasn’t too much content — it was that the delivery came in one monolithic chunk.

An analogy:

  • The old approach was like handing in homework: you can’t submit until the whole essay is written
  • The user experience was like waiting for a package: the tracking never updates, so you have no idea where it is

What we wanted:

Naming should feel like a livestream — as soon as a piece is generated, it appears.

Goals#

Focused on the naming scenario, we set three core goals:

  1. Valuable content appears earlier: show the BaZi analysis first instead of a blank wait
  2. Names appear one by one: read while you wait — no waiting for the full set
  3. Explicit progress feedback: no mistaking “still working” for “stuck” or “done”

The goal wasn’t to “make the model faster” — it was to turn the wait from a black box into something with visible progress and feedback.

Solution Design#

Overall Approach: Staged Output + Progressive Rendering#

We changed the naming result from “one big response” to “multi-stage streaming”:

┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ BaZi analysis│ ──▶ │ Name 1 │ ──▶ │ Name 2 │ ──▶ ...
│ (first) │ │ (appended) │ │ (appended) │
└──────────────┘ └──────────────┘ └──────────────┘
↓ ↓ ↓
user starts results keep feedback keeps
reading growing coming

Stage 1: BaZi Analysis First#

The BaZi analysis is the “directional” information users most want to see first. The moment it appears, they can start reading — and start anticipating the names to follow.

Stage 2: Names Appended One by One#

Name suggestions don’t need to wait for the full set. Each name is appended to the page as soon as it’s generated, so the result visibly “grows”.

Stage 3: A “Still Loading” Card#

Once users have seen the BaZi analysis and 1–2 names, a new doubt creeps in:

“Why did it stop? Is it stuck?”

So we added an explicit indicator module:

  • When partial results are already on screen and the system is still generating
  • The page shows a “remaining modules are loading” card
  • The card includes a progress hint, like “2 of 5 names generated, more on the way…”

The value of this card isn’t “another spinner” — it sets a stable expectation: more results are coming, so I can read what’s already here.

Implementation Notes#

Transport: SSE (Server-Sent Events)#

We chose SSE over WebSocket because:

  • Naming is a one-way push scenario; there’s no need for bidirectional communication
  • SSE rides on plain HTTP — simple to implement, broadly compatible
  • Built-in reconnection makes it friendlier on weak networks

The basic payload design:

// payload structure of each SSE event
interface StreamChunk {
type: 'bazi' | 'name' | 'done' | 'error';
data: BaziAnalysis | NameSuggestion | null;
progress?: {
current: number;
total: number;
};
}

Frontend State Management#

A state machine governs the whole streaming process:

type StreamState =
| { status: 'idle' }
| { status: 'loading'; bazi: null; names: [] }
| { status: 'streaming'; bazi: BaziAnalysis; names: NameSuggestion[] }
| { status: 'done'; bazi: BaziAnalysis; names: NameSuggestion[] }
| { status: 'error'; error: string };

The core handler:

const eventSource = new EventSource('/api/naming/stream');
eventSource.onmessage = (event) => {
const chunk: StreamChunk = JSON.parse(event.data);
switch (chunk.type) {
case 'bazi':
setState(prev => ({ ...prev, bazi: chunk.data }));
break;
case 'name':
setState(prev => ({
...prev,
names: [...prev.names, chunk.data],
progress: chunk.progress
}));
break;
case 'done':
setState(prev => ({ ...prev, status: 'done' }));
eventSource.close();
break;
case 'error':
setState({ status: 'error', error: chunk.data });
eventSource.close();
break;
}
};

Rendering Polish#

To keep the appending smooth, name cards get a simple entrance animation:

.name-card {
animation: fadeInUp 0.3s ease-out;
}
@keyframes fadeInUp {
from {
opacity: 0;
transform: translateY(10px);
}
to {
opacity: 1;
transform: translateY(0);
}
}

Edge Cases#

Network Interruption#

  • SSE reconnects automatically after a drop
  • The frontend remembers the last progress.current it received and passes the offset when reconnecting
  • The backend supports resuming from that offset, avoiding duplicate generation

Generation Failure#

  • A single name fails: skip it, keep generating the rest, and tell the user at the end that “some content failed to generate”
  • Everything fails: show an error plus a retry button; whatever was already rendered stays on screen

User Leaves Mid-stream#

  • Rendering pauses when the page is hidden (via the visibilitychange event)
  • It resumes when the user returns, so background resources aren’t burned

Results#

MetricBeforeAfterChange
Time to first content on screen~8s~2s-75%
Bounce rate during the wait12%4%-67%
Repeat-click rate (false alarms)8%1.5%-81%
User satisfaction score3.84.5+18%

Source: tracking data from the two weeks after launch.

Where Else This Pattern Applies#

This “streaming output + progressive rendering” pattern fits any AI scenario where content is long, slow to generate, and divisible:

  • AI report generation: health checkups, financial analysis, competitive research
  • Long-form writing: marketing copy, article outlines, multi-turn conversation summaries
  • Multi-step tasks: smart form filling, code generation, translation review
  • Search augmentation: aggregating multi-source results in RAG scenarios

The litmus test:

  1. Can the result be split into independent modules?
  2. Can the user consume it while waiting?
  3. Does the content that arrives first have standalone value?

If the answer to all three is yes, streaming output is worth it.

Takeaway#

For an AI feature like naming — long, structured content — what shapes the experience is rarely “how many seconds it takes in total”. It’s:

  • Whether users see value within the first few seconds
  • Whether the system communicates progress clearly
  • Whether the page stays usable while generation is in flight

Switching naming to a streaming presentation really comes down to one thing:

Turn invisible waiting into visible progress.


Try It Yourself#

Want to feel the streaming effect firsthand? Scan the QR code below to try the smart naming feature:

Scan to try smart naming

Making the Wait Visible — Streaming Output Optimization for an AI Naming Feature
https://www.mihouo.com/en/posts/front/naming_streaming_article/
Author
Jia
Published at
2025-12-21
License
CC BY-NC-SA 4.0