Generative UI

Agents don't have to describe your data in a wall of text. In DorkOS, an agent can render it instead, as a real widget right inside the chat.

The agent decides when a widget helps. You never have to ask for a chart or a table, though you can always nudge it ("show me that as a table") if you want one.

Behind the scenes, the agent sends a small JSON document (a simple, structured text format) and DorkOS turns it into a native UI element. Data widgets cover a stat card, a table, a list, a chart, a timeline, a checklist, a comparison matrix, a star rating, a key/value list, a badge, and a progress bar. There are playful ones too: an animated mood face, a game board, and a coin flip or dice roll. Every widget uses the same building blocks as the rest of DorkOS, so it looks and feels like it belongs, not like a mini website stuffed into the chat.

Widgets can talk back too. Click a button, tap a list item, or submit a form, and DorkOS sends your click to the agent so it can respond. A widget can also open in the canvas, a bigger space beside the chat, alongside the images, PDFs, files, and other documents the agent shows you there.

Widgets in Chat

To show a widget, an agent writes a fenced code block labeled dorkos-ui with one JSON document inside it:

```dorkos-ui
{ "version": 1, "root": { "type": "card", "title": "Weather", "children": [
  { "type": "stat", "label": "San Francisco", "value": "64°F", "delta": { "value": "+2°", "direction": "up" } }
] } }
```

DorkOS never shows you that raw code. It reads the JSON and renders a real card with a stat inside instead. While the agent is still typing, you see a loading skeleton in its place; the finished widget appears the moment the block completes.

Every runtime you can pick in DorkOS (Claude Code, Codex, or OpenCode) renders widgets the same way, so switching runtimes never changes what you see.

Widgets Talk Back

Buttons, list items, and form submits aren't just for show. Click "Refresh" on a weather widget, and DorkOS sends the agent a note saying you asked for a refresh, ready for it to respond in your next turn.

While DorkOS sends your click, the button shows a spinner and turns off, so you can't submit twice. If sending fails, most often because the agent is still busy on your last turn, a toast tells you to try again in a moment. There's no queue for these clicks, so you just retry once the agent is free.

You can also pop an interactive widget, like a tic-tac-toe board, out of the chat so it stays visible while you work elsewhere. On a computer it floats in a small window; on a phone it docks to a bottom sheet you can drag up to play, or drag down to tuck it into a small bar at the bottom of the screen; tap the bar to bring it back. It follows the newest board the agent posts, so a game keeps going even after you switch to another session; the chat keeps its own copy too, and closing it just dismisses the popped-out view.

The same floating window works for full interactive apps an MCP server provides, not just widgets. If a connected MCP server ships one of these apps, it renders inline in chat with its own pop-out button, so you can keep it visible the same way while you work in the rest of DorkOS.

Canvas

Widgets aren't limited to chat. An agent can open the same widget in the canvas instead, or alongside chat, giving it more room to breathe. The canvas can hold several documents at once, each in its own tab: widgets, images, PDFs, files, and more. When the agent opens something new, it adds a tab and switches to it rather than replacing what was already there.

For the deeper tour of everything the canvas can hold, see the Workbench guide.

Reference

The rest of this page is for developers and skill authors building or emitting widgets. If you just want to see generative UI in your own chats, you can stop here.

The Document Shape

Every runtime (Claude Code, Codex, OpenCode) receives the same <gen_ui> instructions block, which teaches it the schema below. A widget document has three fields:

{
  version: 1,          // schema version, currently always 1
  title?: string,       // optional: canvas tab title / accessibility label
  root: <node>           // the widget tree, starting from one node
}

A <node> is { "type": "<type>", ...props }. Every node type is part of a fixed catalog: an agent can't invent new ones, and an unrecognized type fails validation (surfacing the error card, not a crash).

Node Catalog

Layout

Text

Data

Delight

Each mood emotion has its own body language, not just a different face: a distinct idle animation and brow/mouth shape. Reduced-motion turns the animation off but keeps the pose, so the emotion still reads. board marks keep fixed, distinct colors per glyph (the classic X/O pair renders in a colorblind-safe blue/amber) so a game stays readable turn after turn, and the board self-heals: if the agent records a move but a re-render drops it, DorkOS draws the recorded move anyway rather than showing an empty cell.

Media

Action: see the ui-action channel below

List item and board cell icons are the places widgets reference something outside their own JSON: icon names (e.g. check-circle, clock, rocket) resolve against a curated Lucide icon allowlist and fall back to a neutral dot for anything unrecognized.

Writing a Widget

Any node can be the root, and containers (stack, card, form) nest recursively. A minimal table:

{
  "version": 1,
  "root": {
    "type": "table",
    "columns": [
      { "key": "id", "label": "Issue" },
      { "key": "status", "label": "Status" }
    ],
    "rows": [{ "id": "DOR-1", "status": "open" }]
  }
}

A couple of catalog-specific rules worth knowing before you emit one:

• image.src (and list item image) must start with https:// or data: (no bare paths, no http://).
• chart.data[].value must be >= 0.
• progress.value must be 0-100; rating.value clamps to 0-5.
• compare rows may be shorter than options: missing cells pad to a blank rather than failing.
• board.rows (and each row) longer than 12 are sliced to 12 rather than rejected; a bare string cell ("X") is shorthand for { glyph: "X" }, and an empty string is a blank cell.
• reveal.result accepts a plain number (dice rolls arrive numeric) and stores it as a string.

Reach for a widget when the reply has structured data worth scanning: metrics, tables, lists, charts. Don't wrap plain prose in one. mood, board, and reveal are the exception: reach for them for personality and games, not data. board paired with an agent action, re-emitted each turn with the latest state, is what makes a widget like tic-tac-toe playable.

Click an empty square and your mark draws itself instantly, before the agent even replies. Its next turn re-emits the same board widget with your move applied. It trusts the state string in your click's payload over its own memory, so the board you see and the board it thinks it's playing never drift apart.

The ui-action Channel

Buttons, list-item actions, and form submits are more than static UI. Clicking them can start a new turn with the agent. Every interactive control fires a WidgetAction, one of three kinds:

A ui-kind action's command can be any control_ui command, including { "action": "celebrate" } (confetti that erupts from the button you clicked, not the middle of the screen). An optional kind picks the style, and emoji picks the glyph for the emoji style:

• burst (the default): a multi-stage confetti pop; the same one a celebrating mood plays from its own face
• fireworks: a few seconds of aerial shells across the top of the screen
• cannons: side cannons firing from the screen edges toward the center
• stars: a golden star pop
• rain: a calm confetti drizzle from the top edge
• emoji: throws the glyph in emoji (default 🎉), e.g. { "action": "celebrate", "kind": "emoji", "emoji": "🏆" }

An unrecognized kind falls back to burst, and every style is skipped for people who prefer reduced motion. Use a celebration for a genuine milestone (tests passing, a deploy landing), not every small success.

agent actions are the interesting one. When you click an agent-kind button, pick a list item action, or submit a form, DorkOS POSTs the action to the session and the agent receives a <ui_action> block as your next turn, before your literal next message:

<ui_action>
The user interacted with a widget you rendered.
Widget: Weather
Action: refresh
Payload: (none)
</ui_action>

The payload carries whatever the action was authored with, plus (for form submits) every field's current value merged in under its name. Give actions stable, descriptive ids: that's the only handle the agent has for "which control fired."

Canvas Content Types

An agent can call control_ui with open_canvas (or update_canvas) and a widget content type to open a rendered widget in the canvas pane:

{ "type": "widget", "definition": { "version": 1, "root": { "...": "..." } }, "title": "Weather" }

The canvas is a tabbed, multi-document surface (each open_canvas call appends a tab and activates it, deduping by source so reopening the same file re-activates its existing tab instead of duplicating it), and widget is one of eleven content types it can render: widget, image, pdf, file (CodeMirror, for arbitrary text/code), markdown (the rich Blintz editor), model3d (glTF/GLB/STL/OBJ), csv, browser (an embedded, sandboxed page), mcp_app (a UI resource an MCP server hosts), url, and json. The two content-heavy ones worth knowing up front:

• image: { type: "image", src, title?, alt? }. Click the image to open it full size in a new tab (skipped for SVG data URIs, which are excluded from click-through as defense in depth against a scripting context).
• pdf: { type: "pdf", src, title? }. Renders through the browser's native PDF viewer; if the browser can't render it inline, a fallback link opens the PDF in a new tab.

For both, src accepts an https:///http:// URL, a data: URI (correctly typed: data:image/* for images, data:application/pdf for PDFs), or a local file path, which DorkOS resolves within the session's working directory. Any other scheme (javascript:, file:, blob:, and so on) is blocked outright.

For the file, markdown, 3D, CSV, and browser viewers, along with per-document edit protection, see the Workbench guide.

Skill Widget Templates

Skills that render the same kind of widget turn after turn don't have to hand-roll the JSON every time. A skill can ship a ui/ directory of *.widget.json templates (a named, described widget document whose string fields may contain {{placeholder}} tokens):

{
  "name": "issue-stat",
  "description": "A stat card summarizing a single tracker issue",
  "document": {
    "version": 1,
    "root": {
      "type": "stat",
      "label": "{{label}}",
      "value": "{{value}}",
      "hint": "{{hint}}"
    }
  }
}

The <gen_ui> instructions every runtime receives tell the agent to check a skill's ui/ directory for templates, fill their placeholders with real values, and emit the result as a dorkos-ui fence. The template itself is never renderable as-is.

Placeholders are only allowed in free-form string fields (text.text, card.title, stat.label, and similarly-typed fields) and in the string | number fields like stat.value. They're rejected in number-only fields (progress.value, chart.data[].value), enum/literal fields (badge.tone, stack.direction, button.variant, every node's type, and so on), and array-shaped fields (table.rows, chart.data, a node's children). Those need a concrete value baked into the template. A placeholder must also fill the entire string value ("{{value}}", not "Status: {{status}}") inside an https-refined field like image.src; mid-string placeholders only work in unconstrained text fields.

Skill validation catches a malformed ui/*.widget.json file as a structural error rather than silently ignoring it, so a broken template surfaces at authoring time, not at render time.

Next Steps