Decision records

Status

Accepted (2026-10-11), amended with Dominic’s approval before merge

Context

Recruiters and engineers need brief, accurate answers about Dominic's work. A stronger model is useful for following scope restrictions, understanding his examples and resisting visitor instructions that contradict the rules. The complete document is small enough to include on every request, and the expected visitor volume is low. Thinking is always enabled on this model and shares max_tokens with the answer. A 1,024-token budget can end before any answer text, so it needs more room and a clear response when exhausted.

Decision

Use claude-opus-5-5 through the official Anthropic TypeScript SDK, with output_config: { effort: "low" } and max_tokens: 2000. This budget covers thinking and answer text together. Leave adaptive thinking at its model default. Do not send a thinking budget or explicitly disable thinking: those settings are rejected by this model.

Cache the fixed rules and complete knowledge document at one ephemeral breakpoint after the document. Typical answers with the document read from cache and short histories target roughly US$0.01 to US$0.02. This is not a per-request ceiling: client-supplied history is uncached, and cache writes, thinking and longer answers cost more. Decision 005 bounds history to 12,000 characters. Server logs record the actual input, output and cache token usage. Opus 5.5 needs at least 512 tokens in the cacheable prefix; shorter prefixes are not cached. Check cache read and write usage in production logs.

Enable beta server-side-fallback-2026-07-01 with fallbacks: "default" so the provider may retry a policy refusal with its default fallback configuration. A final refusal receives a short polite message. A max_tokens stop is a completed response with its stop reason logged, not a provider failure. If no answer text was sent, ask the visitor to narrow the question; otherwise append a short note that the answer was cut short. Close the stream cleanly. Stream only text deltas; never expose thinking or signatures.

The hard budget stop is the Anthropic workspace spend limit. Configure and verify an enforced ceiling before deployment. Visitor and global Redis limits reduce abuse and daily exposure, but do not replace the workspace limit.

Consequences

• Prompt caching makes repeated grounded answers inexpensive; the first cache write and requests after cache expiry cost more.
• The server-side fallback may serve a different model on a refused request.
• Low effort and short replies limit latency and output cost. The higher token budget leaves room for thinking but increases the maximum output cost.
• Token exhaustion remains visible to visitors without breaking their stream.
• SDK tests use synthetic local HTTP streams, not the paid API. The manual eval exercises real answers when credentials are available.
• Model behaviour and the workspace ceiling still need verification against the configured Anthropic account before public release.