# GECX Chat SDK Agent Context > Headless chat SDK for building app-owned AI chat experiences. Framework-neutral core plus React, server, testing, rich-content, compatibility, scaffolding, recipe, and MCP tooling. ## Public Imports - `gecx-chat` - core client, auth, transport, storage, messages, tools, uploads, diagnostics, and types. - `gecx-chat/react` - React provider, hooks, message part renderer, and renderer registry. - `gecx-chat/server` - token endpoint helpers and server-side origin/redaction utilities. - `gecx-chat/testing` - mock client, mock transport, scenario catalog, fixtures, and tool fakes. - `gecx-chat/testing/vitest` - Vitest-only transport contract harness. - `gecx-chat/rich-content` - typed rich-content schemas, registry, and safety helpers. - `gecx-chat/a2ui` - experimental A2UI v0.9 frame validation, surface state, action policy, redaction helpers, and catalog runtime (validateCatalogDataModel, buildCatalogActionPolicy, createCatalogManifestLookup). - `gecx-chat/a2ui/react` - React renderer factory for A2UI generated UI surfaces (createA2UIRenderer with optional catalogManifestLookup + onCatalogTelemetry). - `gecx-chat/compat` - legacy widget migration bridge only when explicitly asked. - `gecx-chat/voice` - VoiceSession, VoiceProvider interface, audio framing helpers, mock providers (scripted + Web Audio), Gemini Live provider, voice transports. - `gecx-chat/react/voice` - React voice components: VoiceToggle, VoiceComposer, TranscriptDisplay, AudioMessage. - `gecx-chat/dashboards` - five pre-built analytics widgets (DeflectionTrend, CsatDistribution, AhtHistogram, AgentAssistRate, GmvBreakdown) + MetricsDashboard. Recharts is an optional peer dep. - `gecx-chat/eval` - programmatic eval runner, scorer registry, judge providers (Anthropic / OpenAI / Gemini), regression gate, reporter. - `gecx-chat/governance/commerce` - PCI/PII redaction policy and helpers for commerce chat. `commerceGovernance({ strict })`, `redactCommercePii`, `redactCommercePiiDeep`, `findCommercePii`, `containsLikelyPan`. Also re-exported from the main package entry. - `@gecx/lab` - `LabSurface` React component, prompt-card schema, and seed templates (support/commerce/handoff/blank) for the `/lab` agent-iteration playground. - `gecx-chat/cli` - CLI entry point for the `gecx` dispatcher. - `gecx-chat/schemas/*` - JSON Schemas for config, events, errors, messages, rich content, tools, agent-graph, computer-use, permissions, ui-catalog, eval, and walkthrough. ## Core Exports - `createChatClient(config?)` - create a headless chat client. Use for all framework-neutral integrations. - `createMemoryStorage()` - in-memory session storage. Use for tests and ephemeral development. - `createWebStorageAdapter(options)` - browser storage adapter. Use for local/session persistence. - `tokenEndpointAuth({ endpoint })` - browser auth via a server token endpoint. Use for most real apps. - `customAuth(provider)` - plug in custom auth behavior. Use when the app already owns auth refresh. - `googleTokenBrokerAuth(options)` - Google token broker auth. Use only from a validated server-side contract. - `createMockTransport(options)` - local mock transport. Use by default while building. - `createProxyTransport(options)` - browser-to-server proxy transport. Use to keep backend secrets server-side. - `createSessionApiTransport(options)` - live Google Session API transport. Use only with explicit live setup. - `createAgentisProxyTransport(options)` - proxy-first Agentis Platform transport. Pair with `createAgentisSessionHandler` and `createAgentisProxyHandler` (`gecx-chat/server`) so OAuth tokens stay server-side. `connectAgentis` is direct-browser compatibility only and requires explicit token-exposure opt-ins. See docs/guides/agentis-integration.md. - `defineClientTool(definition)` - schema-backed client tool. Use for safe app actions. - `createLocalMemoryAdapter({ storage })` - local-only memory adapter. Use for prototypes and consumer apps. - `createServerMemoryAdapter({ endpoint })` - REST-backed memory adapter. Use when memory must survive device changes. - `createHybridMemoryAdapter({ remote, cache })` - cache + remote memory adapter. Use as the production default. - `createHeuristicExtractor({ patterns })` - zero-LLM regex extractor. Use for basic memory extraction without extra cost. - `createA2UISurfaceManager()` - framework-neutral A2UI frame accumulator. Use only for generated UI frame integration. - `defaultActionPolicy()` - conservative A2UI action classifier. Compose with host policy before routing actions. - `agentGraph(definition)`, `defineRouter(...)`, `defineSpecialist(...)`, `createA2AAgentClient({ agentCardUrl })`, `createAgentGraphTransport({ graph, fallback })` - typed multi-specialist routing over Google's A2A protocol. Internal handoff is observable; external handoff still flows through `HandoffController`. - `SignalRunner`, `SignalEscalator`, `ruleAdapter`, `defaultRulePatterns`, `tfjsToxicityAdapter`, `modelToolAdapter`, `geminiAdapter`, `claudeAdapter`, `openaiAdapter` - sentiment / intent signal adapters and escalation. Configure via `signals: { adapters, escalation? }`. - `PermissionManager`, `createBrowserPermissionProvider`, `createMockPermissionProvider`, `createNoopPermissionProvider` - device permissions (microphone, camera, screen, geolocation). Wires into governance consent flags. - `computerUseTool({ endpoint, policy })` - default-off `computer_use` server tool. Server-side runtime in `gecx-chat/server` (`ComputerUseSession`, `MockProvider`, `BrowserbaseProvider`). - `createMockVoiceProvider`, `createWebAudioMockVoiceProvider`, `createGeminiLiveProvider`, `VoiceSession` - voice subsystem. Configure with `voice: 'auto'` for env-based provider picking. - `chat.trackPurchase({ orderId, revenue, currency, items?, paymentMethod? })` - first-class commerce outcome that emits `purchase_completed`. - `computeDeflectionRate`, `computeCsat`, `computeAverageHandleTime`, `computeAgentAssistRate`, `computeGmv` - pure aggregation primitives over `ProductAnalyticsEvent[]`. Window-aware, return `null` on zero-divide. - `runTransportContractTests({ name, factory })` from `gecx-chat/testing/vitest` - reusable harness for validating custom `ChatTransport` implementations. - `createWidgetEventBridge(options)` - compatibility bridge. Use only for migration tasks. - `createDebugBundle(input)` - redacted diagnostic bundle. Use for support and error investigation. - Message types: `ChatMessage`, `ChatMessagePart`, `TextPart`, `CitationPart`, `FilePart`, `ToolCallPart`, `ToolResultPart`, `HandoffStatusPart`, `RichPayloadPart`. - Transport types: `ChatTransport`, `SendRequest`, `TransportEvent`, `SessionContext`, `UploadBody`, `UploadProgressEvent`. - Tool types: `ClientTool`, `ClientToolContext`, `ClientToolDefinition`, `ToolApprovalRequest`, `ToolApprovalHandler`. ## React Exports - `ChatProvider` - provides a client and renderer registry. Use near the app chat surface. - `useChatClient()` - read the active client. Use inside provider-owned components. - `useChatSession(options)` - manage messages, `chat.input.value`, `send`, `regenerate`, `editLastMessage`, `attachFile`, `sendToolResponse`, streaming, handoff, voice, upload, and debug bundle state. - `useChatStore(selector?)` - subscribe to chat state. Use for dashboards and inspectors. - `useMemory(options?)` - read and control long-term memory. Use when `MemoryConfig` is configured. - `useSentiment(messages)`, `useIntent(messages)` - memoized selectors over signal parts. - `usePermission(capability)`, `usePermissionManager()` - device permission lifecycle. - `useClientToken({ endpoint, ttl? })`, `useRecaptcha()`, `useRecaptchaToken({ ttl? })`, `useImpressionRef(...)` - auxiliary helpers. - `MemoryList`, ``, ``, ``, ``, ``, ``, individual dashboard widgets - higher-level surfaces. - `MessagePart` - safe renderer for typed message parts. Use instead of custom branching when possible. - `createRendererRegistry(renderers?)` - register custom part renderers. Use for rich content and recipes. ## Server Exports - `createChatTokenHandler(options)` - create a POST token route. Use for `/api/gecx-chat-token`. - `assertAllowedOrigin(request, origins)` - origin guard. Use on token and proxy routes. - `redactServerSecrets(value)` - server-side redaction. Use before logs and debug responses. - `ComputerUseSession`, `MockProvider`, `BrowserbaseProvider`, `createComputerUseHandler({ provider, policy, audit })` - computer-use server-side runtime. ## Testing Exports - `createMockChatClient(options)` - client with mock transport and fixtures. Use in unit/integration tests. - `createMockTransport(options)` - mock transport with scenarios. Use in generated apps and demos. - `createMockVoiceProvider(options)` - timer-based scripted voice provider for vitest. Zero browser deps. - `createMockPermissionProvider(options)` - scripted permission provider for permission flow tests. - `runTransportContractTests({ name, factory })` from `gecx-chat/testing/vitest` - reusable harness for validating any custom `ChatTransport` against the connect/capability class/protocolVersion/stream/AbortSignal/close contract. - `mockScenarios` - compatibility scenario array. Existing tests may keep using this. - `scenarioCatalog.list()` - full scenario catalog. Use for new mock-first development. - `scenarioCatalog.get(id)` - retrieve a named scenario. - `scenarioCatalog.random({ seed? })` - deterministic random scenario for demos. - `recordMockScenario(events, metadata?)` - record transport events into a replayable scenario. - `replayMockScenario(recording)` - convert a recording into a mock scenario. - Tool fakes: `lookupOrderFake`, `lookupInvoiceFake`, `addToCartFake`, `applyCouponFake`, `createSupportTicketFake`, `openOrderDetailsFake`. ## Commands - `create-gecx-chat --framework --template --backend [--agent] [--json]` - `gecx scaffold --experience --framework --backend --out [--with-voice] [--json]` - `gecx add [--out ] [--json]` (and `gecx add ui:` for A2UI catalog components) - `gecx list [--category ] [--json]` - `gecx search [--json]` - `gecx agent-pack init [--out ] [--all | --target claude,codex,antigravity] [--mcp]` - installs AGENTS.md / CLAUDE.md / GEMINI.md and the headless-chat-vibe-coding skill into the requested agent folders. - `gecx doctor [--token-endpoint ] [--include-ui-catalog]` - integration health check. - `gecx ui:doctor` - verify installed A2UI catalog components. - `gecx eval [--baseline ] [--fail-on-regress] [--json] [--output ]` - scenario-replay eval. - `gecx lab init [--out ] [--template support|commerce|handoff|blank] [--framework next|vite-react|vanilla] [--overwrite]` - installs a `/lab` route with seed prompt cards. - `gecx walkthrough capture --task= [--url=] [--driver=stub|playwright]` - emits a PNG + JSON walkthrough artifact. - `gecx walkthrough verify --task=` - validates a captured walkthrough manifest. - `gecx validate [--path ]` - PCI / PII lint. - `pnpm dlx gecx-chat-mcp-server --install` - register the MCP server with detected agents. - `pnpm dev:studio` - run the start-studio onboarding wizard at http://localhost:3003. UI alternative to `create-gecx-chat`: pick a template, tools, handoff target, and analytics sink; download a runnable repo with a mock transport pre-seeded. Architecture: client-side zip via Web Worker + fflate, bundles the SDK as a tarball for offline install. Source: `apps/start-studio/`. - `pnpm --filter dashboard-reference dev` - reference analytics dashboard server. ## Agent Skills - `headless-chat-vibe-coding` - focused agent skill for building, debugging, and customizing `gecx-chat` experiences. Use it before loading broad docs when an agent supports local skills. ## MCP Resources And Tools - Package: `gecx-chat-mcp-server`. Install for any agent: `pnpm dlx gecx-chat-mcp-server --install [--all | --target claude,codex,antigravity]`. - URI scheme: `gecx-chat://`. - Static resources: `gecx-chat://docs/llms`, `gecx-chat://docs/llms-full`, `gecx-chat://schemas/{messages,tools,errors,walkthrough}`, `gecx-chat://examples/manifest`. - Templated resources: `gecx-chat://recipes/{recipeId}`, `gecx-chat://scenarios/{scenarioId}`, `gecx-chat://examples/{appName}/{filePath}`, `gecx-chat://walkthroughs/{taskId}`, `gecx-chat://errors/{code}`. - Project tools: `gecx_scaffold`, `gecx_add_tool`, `gecx_add_renderer`, `gecx_add_transport`, `gecx_configure`, `gecx_capture_walkthrough`. - Diagnostics: `gecx_diagnose`, `gecx_validate` (includes commerce PCI lint), `gecx_doctor`. - Discovery: `gecx_list_recipes`, `gecx_describe_api`, `gecx_list_parts`. - Preview: `gecx_preview`, `gecx_preview_stop`. - Prompts (parametric task templates): `build-branded-chat`, `add-client-tool`, `add-renderer-for-part`, `swap-transport`, `debug-error`, `scaffold-experience`, `add-voice`, `add-a2ui-surface`, `capture-walkthrough`, `review-headless-chat-integration`, `add-cart-renderer`, `add-checkout-flow`, `add-returns-flow`, `redact-pci-fields`. ## Common Tasks - Build a chat UI: read `docs/agent/llms-full.txt` section 1. - Add streaming affordances: section 2 and recipe `streaming-indicators`. - Add a client tool: section 3 and MCP `gecx_add_tool`. - Add rich content: sections 4-5 and recipes `product-carousel`, `citation-previews`, `suggestion-chips`. - Add token auth: section 6. - Add a custom backend: sections 7-8 and transport recipes. - Add file upload: section 9. - Add handoff: section 10. Mark resolutions explicitly with `HandoffController.complete()` / cockpit `completeSession()`. - Add storage/resume: section 11. - Debug an error: sections 12-13 and MCP `gecx_diagnose`. - Harden production: sections 15-17. - Build generative UI: section 20. Prefer the A2UI catalog (`gecx add ui:`, 20 vetted presets) before building a custom surface. - Add voice: one-line config `voice: 'auto'` + ``. For full control, configure an explicit `VoiceConfig` with a provider. `chat.voice` is a lazy getter — provider factory runs on first read. Microphone access flows through `permissions.ensure('microphone')`. - Add image input/output: use `VisionPart` (kind `input` | `output`). Always pass `altText`. - Voice retention: audio bytes are session-ephemeral by default. Opt-in via `voice_recording` consent flag + `retention.audio` policy. - Add long-term memory: configure `memory: { adapter: createLocalMemoryAdapter({ storage }) }`. Read modes: `inject`, `recall`, `hybrid`. Write paths: tool-call (default) or auto-extraction. - Add sentiment / intent signals: configure `signals: { adapters: [ruleAdapter({ patterns: defaultRulePatterns })], escalation: [...] }`. Use `useSentiment(messages)` and `useIntent(messages)` in React. - Add multi-specialist routing: define routers and specialists with `agentGraph(...)`, then wrap the host transport with `createAgentGraphTransport({ graph, fallback })`. `ChatSession` does not change. - Add computer-use: opt in via `governance.computerUse = { enabled: true, allowlist: [...], maxDurationMs, maxActionsPerSession, highRiskActions, killSwitch }` plus `computerUseTool({ endpoint, policy })`. Proxy-side: `createComputerUseHandler({ provider, policy, audit })`. - Add device permissions: pass `permissionProvider: createBrowserPermissionProvider()`. Use `` for the React surface. Errors are data on `request()`; `ensure()` throws. - Add pre-built dashboards: install recharts as a peer dep, import `` from `gecx-chat/dashboards`, point `httpSink` at your warehouse or `apps/dashboard-reference`. - Track commerce outcomes: call `chat.trackPurchase({ orderId, revenue, currency, items?, paymentMethod? })`. Emits `purchase_completed` for `computeGmv`. - Gate quality with evals: write `*.scenario.ts` or `*.scenario.yaml` files, run `gecx eval --baseline --fail-on-regress` in CI. - Onboard a new developer in <3 minutes: `pnpm dev:studio` (Start Studio wizard) instead of the manual quickstart. - Build a commerce chat (DTC retail): read `skills/headless-chat-vibe-coding/references/commerce.md`. Enable `commerceGovernance({ strict: true })` from `gecx-chat/governance/commerce`. Install commerce recipes via `pnpm exec gecx add cart-summary`, `pnpm exec gecx add order-status-card`, `pnpm exec gecx add commerce-funnel`. Approval-gate all state-changing tools; emit funnel events on impression / click / add / checkout / purchase. - Capture a walkthrough artifact (cross-agent "show your work"): `pnpm exec gecx walkthrough capture --task=`. Schema at `schemas/walkthrough.schema.json`. Antigravity emits natively; CC + Codex emit via the CLI. Eval lane consumes the same format. ## Non-Negotiable Rules - Keep SDK core framework-neutral. React imports belong only under `gecx-chat/react` or app code. - Mock mode is the default development backend. Do not call live Google APIs unless the user provides a validated live transport contract. - Keep service account keys, long-lived tokens, and privileged secrets out of browser code. - Use `/api/gecx-chat-token` for token endpoint examples and `/api/gecx-chat-proxy` for proxy examples. - Render `message.parts` with typed renderers. Do not execute rich payloads as code. - HTML rendering is disabled by default. Do not inject assistant output with `dangerouslySetInnerHTML`. - Use `part.id` keys, never array indexes. - Client tools require JSON Schema inputs and must fail closed when unknown, invalid, denied, or aborted. - Pass `ctx.signal` into cancellable tool work. - Computer-use is **default off**. Never re-enable it silently. Allowlist is hostname-exact; do not widen at the SDK layer to bypass the proxy's check. - Commerce surfaces must run with `commerceGovernance` strict mode. Never weaken or bypass `COMMERCE_PII_LEAK`. - Permissions: prefer `permissions.request()` (returns data) for UX flows and `permissions.ensure()` only when exception semantics are needed. Insecure context is distinct from denied. - Signal escalation rules must call into the existing `HandoffController.requestTransfer()`. Do not invent a parallel handoff path. - Agent graphs are a transport wrapper. Do not mutate `ChatSession` to special-case routed turns. - A2UI catalog manifests are the source of truth. Edits without `pnpm build:ui-catalog` will fail the CI parity check. - Add tests for behavior changes and run the narrow verification before claiming success. ## Verification ```bash pnpm lint pnpm typecheck pnpm test pnpm build:packages pnpm size pnpm e2e ``` Full recipes: `docs/agent/llms-full.txt`. Golden prompts: `docs/agent/agent-prompts.md`. Migration names: `docs/internal/migration-to-gecx-chat.md`. A2UI catalog: `docs/concepts/a2ui-catalog.md` (20 surface presets, install with `gecx add ui:`; contribution guide at `docs/guides/contributing-ui-catalog.md`). ## Computer-use (opt-in, default off) The `computer_use` server tool lets an agent drive a sandboxed browser. **Default off** for security. To enable: (1) set `governance.computerUse: { enabled: true, allowlist: [...] }` on `ChatClient`; (2) register `computerUseTool({ endpoint })` in `tools`; (3) mount `createComputerUseHandler` in your proxy with a `ComputerUseProvider` (Mock for tests, Browserbase for production); (4) render `` in response to the tool's `awaiting_approval` event. Errors: `COMPUTER_USE_NOT_ENABLED` / `_ALLOWLIST_VIOLATION` / `_DURATION_EXCEEDED` / `_ACTION_LIMIT_EXCEEDED` / `_USER_ABORT` / `_PROVIDER_UNAVAILABLE`. Audit events: `governance.computer_use.*`. See [docs/guides/computer-use.md](../guides/computer-use.md) and [docs/reference/computer-use-threat-model.md](../reference/computer-use-threat-model.md).