Streaming UIs make a slow LLM feel fast, but only if you do them well. Done badly, the page reflows on every token, the scroll position jumps, and the markdown renderer keeps half-rendering things that complete two seconds later. Here's the rendering pipeline we've settled on.
Chunk, do not character-stream
Rendering on every token is a waste of frames. We batch incoming tokens into ~16ms chunks and render once per animation frame. The UI feels as fast as character-by-character and the browser is much happier.
The markdown reflow problem
Our pipeline maintains two states: the "committed" markdown (everything before the last paragraph break) and the "tentative" tail. Only the tail gets re-parsed; the committed part renders once and is left alone.
Preventing layout shift
The single biggest perceived-quality lever: reserve vertical space for the streaming content from the moment the response starts. Empty placeholder, fixed height, growing into it.
When code blocks or tables appear, they get reserved height too: a skeleton placeholder that matches the eventual size as closely as we can predict.
Auto-scroll, without trapping the user
If the user has scrolled up to read something, we stop auto-scrolling. If they scroll back to the bottom, we resume. This is one of those tiny details users notice instantly when it is wrong.
Patterns we use
- Typewriter: token-by-token reveal with a subtle cursor. Good for short, conversational responses.
- Paragraph fade: paragraphs fade in once complete. Good for long-form content where the reader catches up at paragraph breaks.
- Streaming markdown: live render with the committed/tentative split. Default for anything structured.