Custom LLM

Connect any completion provider to response generation and memory extraction.

Contract

A custom LLM adapter receives normalized messages plus optional system context and returns completion text.

Implementation

llm.ts

class MyLLM implements LLMAdapter {
  async complete(messages: Message[], system?: string) {
    const result = await client.generate({ messages, system });
    return result.text;
  }
}

Production concerns

Set request timeouts.

Retry only safe transient failures.

Track latency and token usage.

Avoid logging private prompts by default.