UX Engineering for AI Interfaces: Token Streaming, Optimistic UI, and Error States

The Eight-Second Loading Spinner Disaster

When I built my first AI-powered web feature, I treated it like any other REST endpoint. The user typed a question, clicked Submit, and my frontend triggered a standard axios.post() call. I showed a little spinning wheel in the center of the card while the backend waited for the model to finish generating.

It was a disaster. Generating a 400-word answer took eight full seconds. Users stared at the spinner, assumed the application had frozen or crashed, and clicked refresh three times. Each refresh triggered another expensive API call on our backend.

Traditional web interfaces deal with millisecond response times. AI interfaces deal with multi-second generation times. If you make users wait for the whole paragraph to generate before showing anything, your app feels sluggish and broken. But if your frontend streams the very first word in 200 milliseconds, users perceive the app as blazing fast, even if the complete answer takes seven seconds to finish.

Building a great AI interface is an engineering problem: consuming Server-Sent Events (SSE), stabilizing layout shifts, optimistic state management, and letting users cancel runaway streams.

1. Consuming Streams with Server-Sent Events (SSE)

Do not use heavy WebSockets for simple text generation. WebSockets require stateful connections and complex server scaling. Text generation is unidirectional: the client sends one prompt, and the server streams chunks back over standard HTTP/2 or HTTP/3.

Here is a clean TypeScript stream consumer using the native browser fetch API and ReadableStreamDefaultReader:

// src/lib/stream-consumer.ts
export interface StreamCallbacks {
  onChunk: (text: string) => void;
  onError: (error: Error) => void;
  onComplete: () => void;
}

export async function consumeAiStream(
  endpoint: string,
  prompt: string,
  callbacks: StreamCallbacks,
  signal?: AbortSignal
): Promise<void> {
  try {
    const response = await fetch(endpoint, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ prompt }),
      signal
    });

    if (!response.ok) {
      if (response.status === 429) {
        throw new Error('Rate limit reached. Please wait a few seconds before trying again.');
      }
      throw new Error(`Server returned HTTP error ${response.status}`);
    }

    if (!response.body) {
      throw new Error('Streaming response body is not supported by your network proxy.');
    }

    const reader = response.body.getReader();
    const decoder = new TextDecoder('utf-8');
    let done = false;

    while (!done) {
      const { value, done: streamDone } = await reader.read();
      done = streamDone;
      if (value) {
        const chunkText = decoder.decode(value, { stream: !done });
        callbacks.onChunk(chunkText);
      }
    }

    callbacks.onComplete();
  } catch (error: any) {
    if (error.name === 'AbortError') {
      console.log('Stream stopped by user.');
      return;
    }
    callbacks.onError(error instanceof Error ? error : new Error(String(error)));
  }
}

You can check JSON payload structures and response schemas with our free JSON Formatter.

2. Fixing Cumulative Layout Shift (CLS) While Streaming

Nothing annoys users more than trying to read streaming text that violently jumps up and down. When markdown tokens arrive word by word, unformatted asterisks flicker into bold tags, code blocks suddenly expand, and the page height jumps frantically.

Here are three practical rules to eliminate streaming layout jitter:

• Do Not Fight the User's Scroll: Only autoscroll the page if the user is already at the bottom. If the user scrolls up to read paragraph one, disable autoscroll immediately.
• Stable Font Metrics: Use line-height: 1.6 and font-variant-numeric: tabular-nums so new lines do not alter the vertical rhythm.
• A Pulsing Inline Cursor: Add a subtle blinking cursor to show the model is still actively generating.

/* styles/streaming.css */
.ai-message-bubble {
  min-height: 44px;
  line-height: 1.625;
  contain: content;
  word-break: break-word;
}

.ai-cursor-pulse {
  display: inline-block;
  width: 5px;
  height: 15px;
  margin-left: 3px;
  vertical-align: middle;
  background-color: #3b82f6;
  animation: cursor-blink 0.8s ease-in-out infinite;
}

@keyframes cursor-blink {
  0%, 100% { opacity: 1; }
  50% { opacity: 0; }
}

3. Optimistic Updates and the AbortController Stop Button

When a user hits Enter, do not wait for the server to reply. Add the user message to your UI state immediately. At the exact same time, swap your Send button for an active Stop button connected to an AbortController:

// src/controllers/ChatController.ts
export class ChatController {
  private abortController: AbortController | null = null;
  private isGenerating = false;

  public async handleSend(userPrompt: string) {
    if (!userPrompt.trim() || this.isGenerating) return;

    // 1. Optimistic update: render user message immediately
    this.appendMessage({ role: 'user', content: userPrompt });
    
    // 2. Prepare placeholder for assistant reply
    const assistantId = this.appendMessage({ role: 'assistant', content: '', isStreaming: true });

    // 3. Create abort controller for cancellation
    this.abortController = new AbortController();
    this.isGenerating = true;

    await consumeAiStream(
      '/api/chat',
      userPrompt,
      {
        onChunk: (chunk) => {
          this.appendAssistantChunk(assistantId, chunk);
        },
        onError: (err) => {
          this.markErrorMessage(assistantId, err.message);
          this.isGenerating = false;
        },
        onComplete: () => {
          this.finalizeMessage(assistantId);
          this.isGenerating = false;
        }
      },
      this.abortController.signal
    );
  }

  public handleCancel() {
    if (this.abortController) {
      this.abortController.abort();
      this.isGenerating = false;
    }
  }

  private appendMessage(msg: any): string { return 'msg-' + Date.now(); }
  private appendAssistantChunk(id: string, chunk: string) {}
  private markErrorMessage(id: string, message: string) {}
  private finalizeMessage(id: string) {}
}

4. Handling Mid-Stream Failures and Partial Data

Internet connections drop, especially on mobile data. When a stream dies at token 300, bad UX throws away the whole message and shows a giant generic error banner. Good UX preserves whatever text was already received and offers a clean in-line "Retry generation" button right beneath the partial text.

Always add a 1-click Copy Code button to every code fence. Developers do not want to highlight 40 lines of code with their trackpad on a laptop. Give them a quick copy button that copies clean code without markdown backticks.

You can estimate token counts and monitor response size using our free Token Counter and test developer tools on our Free Developer Tools page.

Good AI engineering is not about flashy landing page gimmicks. It is about fast time-to-first-token, stable layout, predictable scrolling, and graceful error recovery. Implement these patterns in your frontend tonight.