Streaming

A document arrives one line at a time, and each line mounts as soon as it is complete. Below, a Next.js route relays the model’s output with the AI SDK , and a page reads it into a growing array with TanStack Query .

The route

streamText returns the output as a text stream, and createTextStreamResponse relays it as is:

src/app/api/generate/route.ts


import { createTextStreamResponse, streamText } from "ai";
 
export async function POST(req: Request) {
  const { prompt } = await req.json();
 
  // A bare model id resolves through the Vercel AI Gateway (set AI_GATEWAY_API_KEY).
  const result = streamText({ model: "anthropic/claude-opus-5.5", system, prompt });
 
  return createTextStreamResponse({
    stream: result.textStream,
    headers: { "content-type": "application/jsonl; charset=utf-8" },
  });
}

Persisting as it streams

To keep a page, read the stream on the way through and store each entry as its line completes:


async function* lines() {
  for await (const value of streamJsonLines(result.textStream)) {
    if (!isComponentEntry(value)) continue;
    await db.insert(componentEntries).values({ data: value });
    yield `${JSON.stringify(value)}\n`;
  }
}
 
return createTextStreamResponse({
  stream: new ReadableStream<string>({
    async start(controller) {
      for await (const line of lines()) controller.enqueue(line);
      controller.close();
    },
  }),
  headers: { "content-type": "application/jsonl; charset=utf-8" },
});

isComponentEntry keeps non-entries out of the table: key and component must be strings, and children, if present, an array.

The page

TanStack Query’s streamedQuery appends each value of an AsyncIterable to the cached array and re-renders per chunk:


"use client";
import {
  experimental_streamedQuery as streamedQuery,
  useQuery,
} from "@tanstack/react-query";
import { type ComponentEntry, streamJsonLines } from "@uicast/core";
import { Evaluator } from "@uicast/expr";
import { EntriesRenderer, RendererProvider } from "@uicast/react";
 
const evaluator = new Evaluator({ functions: tools });
const fallbackComponents = { defaultSkeleton: () => <Skeleton className="h-6 w-24" /> };
 
function Generated({ prompt }: { prompt: string }) {
  const { data: entries = [], isFetching } = useQuery({
    queryKey: ["generate", prompt],
    // One model call per prompt: no refetch on window focus, no retry.
    staleTime: Infinity,
    retry: false,
    queryFn: streamedQuery({
      streamFn: async () => {
        const res = await fetch("/api/generate", {
          method: "POST",
          headers: { "content-type": "application/json" },
          body: JSON.stringify({ prompt }),
        });
        if (!res.ok || !res.body) throw new Error(`Generation failed (${res.status})`);
        return streamJsonLines<ComponentEntry>(res.body);
      },
    }),
  });
 
  return (
    <RendererProvider implementations={impls} evaluator={evaluator} fallbackComponents={fallbackComponents}>
      {isFetching && <p>Generating…</p>}
      <EntriesRenderer entries={entries} />
    </RendererProvider>
  );
}

entries holds the entries received so far. isFetching stays true until the stream ends.

The half-arrived tree

entries after one line, a parent naming a child that has not arrived:


{"key":"panel","component":"Card","props":{"literal":{"title":"Orders"}},"seed":[{"set":"scopes.root.open","expr":"countOpen()"}],"children":["stat"]}

While countOpen() runs, panel draws its implementation’s skeleton. Then panel renders with its “Orders” title, and the same skeleton holds the slot for stat. With no skeleton, fallbackComponents.defaultSkeleton does, or nothing. The next line lands:


{"key":"stat","component":"Stat","props":{"expr":"({ label: 'Open', value: scopes.root.open })"}}

stat replaces the skeleton, and countOpen() does not run again: a seed runs once. The tree is revealed, not rebuilt, and a document cut off at any line is still a valid document.

Where to go next

Renderer — mounting the entries.
Core — streamJsonLines’ signature and tolerance rules.
Getting started — the whole loop, end to end.