Streaming AI Responses: SSE & WebSocket in React

Overview

LLMs generate tokens sequentially. Waiting for the full response (5–30 seconds) before rendering creates a terrible UX. Streaming sends each token as it's generated — the user sees text appearing within ~200ms of the request.

Two protocols:

For most AI features (chat, inline suggestions, summarization), SSE is the right choice — simpler, works over HTTP/2, auto-reconnects.

Architecture

  React Component                  Next.js Route Handler          LLM API
  ─────────────────                ─────────────────────          ─────────
  useStreamingChat()                                               
       │                                                           
       │ fetch('/api/chat', {       POST /api/chat               
       │   method: 'POST',    ────────────────────────▶          
       │   body: {messages}   ◀─ ─ ─ ─text/event-stream─ ─ ─    
       │ })                                                        
       │                            await anthropic               
       │                            .messages.stream()  ────────▶
       │                                                          │
       │ ◀──── data: {"token": "The"} ◀──── yield token ◀────── │
       │ ◀──── data: {"token": " answer"} ◀─ yield token ◀───── │
       │ ◀──── data: {"token": " is"} ◀──── yield token ◀────── │
       │ ◀──── data: [DONE]                                       
       │                                                           
   setTokens(                                                      
     (prev) => prev + token                                        
   )                                                               
       │                                                           
   Render text                                                     
   as it streams                                                    

━━━━━━━━━━━━━━━━━  FULL CHAT SYSTEM  ━━━━━━━━━━━━━━━━━

  ┌─────────────────────────────────────────────────┐
  │                  Next.js App                     │
  │                                                  │
  │  ┌────────────────────────────────────────────┐ │
  │  │  ChatInterface Component                   │ │
  │  │  - messages[]                              │ │
  │  │  - streaming token buffer                  │ │
  │  │  - abort controller (cancel)               │ │
  │  └────────────────────────────────────────────┘ │
  │                       │                          │
  │                       │ fetch (SSE)              │
  │                       ▼                          │
  │  ┌────────────────────────────────────────────┐ │
  │  │  /api/chat  (Route Handler)                │ │
  │  │  - auth check                              │ │
  │  │  - rate limit                              │ │
  │  │  - build messages array                    │ │
  │  │  - stream from Anthropic                   │ │
  │  │  - pipe tokens to response stream          │ │
  │  └────────────────────────────────────────────┘ │
  └─────────────────────────────────────────────────┘

Technical Implementation

Next.js API Route — SSE Streaming

// app/api/chat/route.ts
import Anthropic from '@anthropic-ai/sdk';
import { NextRequest } from 'next/server';

const anthropic = new Anthropic();

export async function POST(req: NextRequest) {
  const { messages, system } = await req.json();

  // Validate auth
  const session = await getServerSession(req);
  if (!session) return new Response('Unauthorized', { status: 401 });

  // Create a ReadableStream that pipes Anthropic token events
  const stream = new ReadableStream({
    async start(controller) {
      const encoder = new TextEncoder();

      const send = (data: object) => {
        controller.enqueue(encoder.encode(`data: ${JSON.stringify(data)}\n\n`));
      };

      try {
        const anthropicStream = anthropic.messages.stream({
          model: 'claude-sonnet-4-6',
          max_tokens: 2048,
          system: system ?? 'You are a helpful assistant.',
          messages,
        });

        for await (const event of anthropicStream) {
          if (event.type === 'content_block_delta' && event.delta.type === 'text_delta') {
            send({ type: 'token', token: event.delta.text });
          }
          if (event.type === 'message_stop') {
            send({ type: 'done' });
          }
          if (event.type === 'message_delta') {
            send({
              type: 'usage',
              inputTokens: event.usage?.input_tokens,
              outputTokens: event.usage?.output_tokens,
            });
          }
        }
      } catch (err) {
        send({ type: 'error', message: (err as Error).message });
      } finally {
        controller.close();
      }
    },
  });

  return new Response(stream, {
    headers: {
      'Content-Type': 'text/event-stream',
      'Cache-Control': 'no-cache',
      'Connection': 'keep-alive',
      'X-Accel-Buffering': 'no',  // Disable nginx buffering
    },
  });
}

React Hook — useStreamingChat

// hooks/useStreamingChat.ts
import { useState, useCallback, useRef } from 'react';

interface Message {
  role: 'user' | 'assistant';
  content: string;
}

export function useStreamingChat() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [isStreaming, setIsStreaming] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const abortControllerRef = useRef<AbortController | null>(null);

  const sendMessage = useCallback(async (userMessage: string) => {
    const newMessages = [...messages, { role: 'user' as const, content: userMessage }];
    setMessages(newMessages);
    setIsStreaming(true);
    setError(null);

    // Add empty assistant message — we'll stream into it
    setMessages((prev) => [...prev, { role: 'assistant', content: '' }]);

    abortControllerRef.current = new AbortController();

    try {
      const resp = await fetch('/api/chat', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ messages: newMessages }),
        signal: abortControllerRef.current.signal,
      });

      if (!resp.ok) throw new Error(`HTTP ${resp.status}`);
      if (!resp.body) throw new Error('No response body');

      const reader = resp.body.getReader();
      const decoder = new TextDecoder();
      let buffer = '';

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        buffer += decoder.decode(value, { stream: true });
        const lines = buffer.split('\n\n');
        buffer = lines.pop() ?? '';

        for (const line of lines) {
          if (!line.startsWith('data: ')) continue;
          const data = JSON.parse(line.slice(6));

          if (data.type === 'token') {
            setMessages((prev) => {
              const updated = [...prev];
              updated[updated.length - 1] = {
                role: 'assistant',
                content: updated[updated.length - 1].content + data.token,
              };
              return updated;
            });
          }
        }
      }
    } catch (err) {
      if ((err as Error).name === 'AbortError') return;
      setError((err as Error).message);
      // Remove empty assistant message on error
      setMessages((prev) => prev.slice(0, -1));
    } finally {
      setIsStreaming(false);
    }
  }, [messages]);

  const cancel = useCallback(() => {
    abortControllerRef.current?.abort();
  }, []);

  return { messages, isStreaming, error, sendMessage, cancel };
}

Chat UI Component

// components/ChatInterface.tsx
'use client';

import { useStreamingChat } from '@/hooks/useStreamingChat';
import { useState } from 'react';

export function ChatInterface() {
  const { messages, isStreaming, error, sendMessage, cancel } = useStreamingChat();
  const [input, setInput] = useState('');

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    if (!input.trim() || isStreaming) return;
    sendMessage(input);
    setInput('');
  };

  return (
    <div className="flex flex-col h-[600px] border border-border rounded-lg overflow-hidden">
      {/* Messages */}
      <div className="flex-1 overflow-y-auto p-4 space-y-4">
        {messages.map((msg, i) => (
          <div
            key={i}
            className={`flex ${msg.role === 'user' ? 'justify-end' : 'justify-start'}`}
          >
            <div
              className={`max-w-[80%] rounded-lg px-4 py-2 text-sm ${
                msg.role === 'user'
                  ? 'bg-primary text-primary-foreground'
                  : 'bg-secondary text-foreground'
              }`}
            >
              {msg.content}
              {/* Blinking cursor on last assistant message while streaming */}
              {isStreaming && i === messages.length - 1 && msg.role === 'assistant' && (
                <span className="inline-block w-0.5 h-4 bg-current ml-0.5 animate-pulse" />
              )}
            </div>
          </div>
        ))}
        {error && (
          <div className="text-xs text-destructive text-center">{error}</div>
        )}
      </div>

      {/* Input */}
      <form onSubmit={handleSubmit} className="border-t border-border p-3 flex gap-2">
        <input
          value={input}
          onChange={(e) => setInput(e.target.value)}
          placeholder="Ask something..."
          className="flex-1 bg-secondary rounded-md px-3 py-2 text-sm outline-none focus:ring-1 focus:ring-primary"
          disabled={isStreaming}
        />
        {isStreaming ? (
          <button
            type="button"
            onClick={cancel}
            className="px-3 py-2 text-xs rounded-md bg-destructive/20 text-destructive hover:bg-destructive/30"
          >
            Stop
          </button>
        ) : (
          <button
            type="submit"
            disabled={!input.trim()}
            className="px-3 py-2 text-xs rounded-md bg-primary text-primary-foreground disabled:opacity-50"
          >
            Send
          </button>
        )}
      </form>
    </div>
  );
}

Module Federation — Embedding AI Features Across Microfrontends

// ai-assistant-remote/next.config.js — AI feature as a Module Federation remote
const NextFederationPlugin = require('@module-federation/nextjs-mf');

module.exports = {
  webpack(config) {
    config.plugins.push(
      new NextFederationPlugin({
        name: 'aiAssistant',
        filename: 'static/chunks/remoteEntry.js',
        exposes: {
          './ChatInterface': './components/ChatInterface',
          './InlineAssist': './components/InlineAssist',
          './SemanticSearch': './components/SemanticSearch',
        },
        shared: { react: { singleton: true } },
      })
    );
    return config;
  },
};

// Any domain app consumes the AI feature without rebuilding
const ChatInterface = dynamic(
  () => import('aiAssistant/ChatInterface').catch(() => import('./AIChatFallback')),
  { ssr: false }
);

Interview Preparation

Q: Why SSE over WebSocket for LLM streaming?

SSE works over standard HTTP (no upgrade handshake), auto-reconnects on disconnect, supported natively by browsers via EventSource. WebSocket is better when you need bidirectional real-time (presence, live collaboration). For one-directional token streaming, SSE is simpler and HTTP/2 multiplexes it well.

Q: How do you handle SSE reconnection if the stream drops mid-response?

Use the Last-Event-ID header. Each SSE event has an id field — browser includes it on reconnect. Server resumes from that token index. In practice for LLM streams, it's easier to restart the generation and skip to the last token count using the model's API (some support resumable streams).

Q: How do you show a "typing indicator" before the first token?

Set a isLoading state to true when the fetch starts. Show a pulsing placeholder. Switch to streaming state when the first token arrives. The distinction matters for UX — users see something within 200ms even if tokens don't arrive for 2 seconds.

Q: How do you prevent the component from unmounting mid-stream?

Abort controller. When the component unmounts (useEffect cleanup), call abortController.abort(). This cancels the fetch and stops token processing. Without this, you get "Can't update state on unmounted component" React warnings and memory leaks.

Q: How do you implement inline AI suggestions (like GitHub Copilot)?

Debounce the user's typing (500ms), then trigger a streaming call with the current editor context as prompt. Render suggestions in a ghost-text overlay (CSS ::after pseudo-element or absolutely positioned div). Accept on Tab, dismiss on Esc. Cancel any in-flight request when the user continues typing.

Learning Resources

• Vercel AI SDK — useChat, useCompletion hooks, SSE streaming built-in
• Anthropic Streaming Docs — SSE event types
• MDN: Server-Sent Events
• Next.js Route Handlers — streaming response support
• ReadableStream API — browser streaming