Advanced
Performance
Tune latency, model usage, retrieval size, and ingestion throughput.
Highest-impact controls
Bound recall with
limit and tokenBudget.Move ingestion to queue mode when it affects response latency.
Cache safe recall workloads with explicit invalidation expectations.
Batch embeddings where the adapter supports it.
Measure
Invoke latency and first-token time.
Ingestion lag and queue depth.
Embedding and completion usage.
Recall precision, empty-result rate, and prompt size.