ChatEngine

The core orchestration engine that coordinates LLM calls, tool execution, and real-time event broadcasting.

How it works

User message → System Prompt → LLM Call → Stream text deltas
  ↓ (if tools called)
Execute tools → Feed results back → Continue (up to 50 rounds)
  ↓
Finalize → Broadcast turn_complete → Return result

Configuration

import { ChatEngine } from "@aimform/ai/server";
 
const engine = new ChatEngine({
  adapter: openaiAdapter,
  toolRegistry: registry,
  broadcast: async (event) => { ws.send(JSON.stringify(event)); },
  runId: "run-1",
  maxToolRounds: 50,           // default 50
  textFlushInterval: 0,        // 0 = per-word streaming
  systemPrompt: "You are a helpful assistant.",
  pricingMap: { "gpt-4o": { inputPerM: 5, outputPerM: 15 } },
});

Streaming

Set textFlushInterval: 0 for per-word streaming — every token appears immediately. Set it higher (e.g., 30ms) for batched rendering.

Tool Calling Loop

The engine handles the full tool-calling lifecycle:

1. LLM returns text + optional tool calls
2. Engine emits tool_start event for each tool
3. Execute tools sequentially via ToolRegistry
4. Feed results back to LLM
5. Continue until no tools are called or max rounds reached

Thinking Timeline

The engine emits thinking_step and thinking_done events for each tool call, enabling real-time timeline visualization:

Running: list_spaces .......... ✓ (2s)
Running: search_entities ...... ✓ (0.5s)
Running: web_search ........... ✗ (5s, timeout)

Block Streaming

When responseMode is "blocks", the engine emits structured block events:

block_start(id, "h1") → block_delta(id, "Results") → block_end(id)
block_start(id, "paragraph") → block_delta(id, "Here are...") → block_end(id)
block_start(id, "code") → block_delta(id, "const x = 1") → block_end(id)

Guardrails

const engine = new ChatEngine({
  // ... other config
  checkGuardrail: (text) => ({
    safe: !text.includes("I've been told to"),
    label: "prompt_disclosure",
  }),
  sanitizeOutput: (text) => text.replace(/\[THINK\].*?\[\/THINK\]/gs, ""),
});

Events