Full context over retrieval for Layer 3

• Supersedes: the embedding/index paragraphs of ADR 0006 (the hosting decision itself stands)

Context

Layer 3 has been described since ADR 0006 as “a retrieval API with embeddings over /content” — POST /api/v1/ask, a vector index, chunking, top-k. That description was written before anyone measured the corpus it would retrieve from. The route name stands; only what sits behind it changes.

Measured on 2026-08-13, from the repository itself:

Slice	Bytes	Notes
All Markdown (find content -name '*.md')	14 013	8 files, EN + PL
— of which content/README.md	2 732	documents the corpus, is not corpus
— English prose only, excluding README	6 880	what one locale’s answer draws on
— Polish overlay only	4 401	translations of the same facts
content/stack.yaml	2 102	the technology registry
content/.schema/*.json	12 563	contracts, never part of a prompt

Words across all Markdown: 2 039.

So the payload a question would actually be answered from — one locale’s prose plus the stack registry — is on the order of 9 KB, roughly 2 500 tokens. Both locales together are about 4 000 tokens. (Token figures are estimates from character counts; the byte counts are exact and reproducible with the commands above.)

For scale: that is 0.4% of Gemini 2.5’s 1M-token window and 2% of Claude’s 200k. ADR 0006 quoted 80 KB for the same corpus — an estimate that was high by roughly 6x, and the estimate is what the retrieval design was reasoning from.

Measured after implementing it (the host logs this at startup): the rendered prompt is 7 725 characters / ~1 931 tokens for en and 7 906 / ~1 976 for pl. Slightly under the projection above, because the rendering drops markdown syntax and duplicated frontmatter.

Decision

Layer 3 passes the entire corpus in the prompt. There is no embedding model, no vector store, no chunking, and no retrieval step.

The corpus is projected once at startup into a prompt-shaped snapshot, held as a singleton beside IContentStore, and prepended to every question.

Why retrieval would be worse here, not merely unnecessary

• Chunking destroys more than it selects. At ~2 500 tokens, sensible chunks are ~500 tokens and k=5 returns substantially the whole corpus anyway — but reordered, with document boundaries and front-matter context stripped. The retrieval step can only lose information relative to passing the files whole. There is nothing for it to win.
• The likely questions are the ones top-k answers worst. A visitor asks “does he know Kafka”, “how much distributed-systems experience is there”, “what recurs across the projects”. These are aggregate and comparative; the answer is not located in any single chunk. Naive similarity search is weakest exactly where this corpus will be interrogated most.
• Aggregation is already solved deterministically. The get_stack MCP tool joins the technology registry against every project and experience entry and returns real counts. A vector search over the same content would produce a worse answer to the same question, non-deterministically.
• An index is a second source of truth. ADR 0001 makes /content the only one. An embedding index has to be rebuilt on every content change, can drift from the corpus, and turns a deploy into a data migration. Full context cannot drift: it is the corpus.

Cost, which is what usually forces retrieval

At roughly 2 500 input tokens per question and a small model (order of $0.30 per million input tokens — verify current pricing before relying on it), a question costs on the order of $0.001. A thousand questions a month is about $1.

Retrieval exists to stop the prompt growing without bound. This prompt is bounded by a corpus that a human wrote by hand, and any growth realistic for a personal site (a blog, ten more projects) leaves it inside the same order of magnitude.

Provider-side prompt caching is therefore not part of this decision. At this size the saving is cents a year and the added coupling is real.

The threshold for revisiting

Retrieval becomes the right answer when any of these is true:

• The corpus passes roughly 100 000 tokens — about 25x today.
• Inference cost passes roughly $20/month at steady state.
• Answer quality measurably degrades because relevant facts are lost in the middle of a long context.

None of these is close. When one becomes true, the seam to change is the prompt-assembly component; the endpoint, the safety controls and the UI are unaffected by which strategy fills the context.

Consequences

• The confidential rule becomes a security boundary, not a display rule. Today PortfolioView is the single place a confidential project’s body is withheld. The chat must assemble its prompt through that same projection — a model given the raw IContentStore would read withheld bodies and could repeat them. This is the one place in Layer 3 where a shortcut is a leak, and it is worth a test that asserts no confidential body ever reaches the prompt payload.
• Citations come free. Every view already carries the Source file each statement came from, so answers can attribute without any extra machinery.
• The engineering work moves from retrieval to abuse control. A public endpoint that calls a paid model is a cost-exhaustion target. Rate limiting, a bot check, a hard server-side spend cap and a kill switch are the real substance of Layer 3 — not the part that produces the answer.
• A dependency on an external model provider appears for the first time in this stack. The site degrades correctly without it: the static pages, the machine surfaces and the MCP server are all unaffected by the chat being down, because none of them route through it.
• The MCP server already covers machine access. An MCP client brings its own model and queries /content through typed tools. The chat endpoint exists for a human on the page who wants a text box — it is not a second machine interface, and it should not grow into one.

Alternatives rejected

Embeddings + a local vector store (sqlite-vec / Chroma), as ADR 0006 sketched. Rejected on the measurement above: it adds an index, a rebuild step and a failure mode in exchange for selecting a subset of a corpus that fits comfortably whole.

Build retrieval anyway, as a demonstration of competence. This was the strongest argument for it — the site is a portfolio, and RAG is a skill worth showing. Rejected because it demonstrates the opposite to the audience that matters: vector search over eight Markdown files reads as reaching for a pattern without checking whether it fits. This ADR, with the numbers in it, is the better artefact. If a retrieval implementation is wanted for its own sake, it belongs over a corpus that actually needs one.

Hybrid — retrieve, but fall back to full context when the corpus is small. Two code paths, one of which never executes at current size, and the untested one is the one that would run the day the corpus grew. The threshold above is a decision to make deliberately, not a branch to leave lying in the code.