Architecture · 7 min read · Aug 12, 2026
The prompt is small. The context is the system.
Prompt engineering was a reasonable name for a while. It described the era when the main lever was phrasing. That era is over — in a production system the wording is a small part of a much larger decision about what the model gets to see, and that decision is where reliability comes from.
What is actually in the window
On any real request, the instruction you wrote is a fraction of what the model receives. Around it sits retrieved documents, conversation history, tool definitions and their outputs, examples, and structural scaffolding. Every one of those is an engineering decision — what to include, in what order, at what fidelity, and what to leave out.
Which is why two teams with near-identical prompts get very different reliability. The difference is rarely the sentence. It is what surrounded the sentence.
More context is not better context
The intuitive move when a system underperforms is to add: more instructions, more examples, more retrieved documents, longer history. This frequently makes it worse, and the reasons are worth being precise about.
- Dilution. A critical instruction competes with everything else present. The more you add, the less any one item weighs.
- Position effects. Attention across a long input is uneven — material in the middle is used less reliably than material at either end.
- Contradiction. Accumulated instructions eventually disagree with each other, and the resolution is unpredictable.
- Stale history. Long conversations carry superseded facts that read as current, and the model has no way to know they were revised.
When a system starts behaving unpredictably, the first question is not what to add. It is what to remove.
The decisions that matter
Context engineering is mostly four choices, made deliberately rather than by accident of implementation.
| Decision | The failure it prevents |
|---|---|
| Selection — what to retrieve at all | Right answer never reaches the model |
| Compression — how much of each item | Budget consumed by low-value text |
| Ordering — where things sit | Critical material buried mid-context |
| Isolation — what each step sees | One step's noise contaminating the next |
Isolation is the one that most distinguishes systems that hold up under load. The instinct is to give every step the full accumulated context so nothing is lost. In practice, giving each step exactly what it needs and nothing more produces more reliable behaviour — errors and irrelevant material stop propagating forward, and each step's job becomes narrow enough to test on its own.
Structure beats phrasing
Where prompt wording still matters most is structure rather than eloquence. Clear separation between instruction and data, explicit output shape, and a stated fallback for the ambiguous case do more than any amount of rewording.
The fallback is the one that gets left out. A model given no instruction for what to do when the answer is not in the provided material will generally produce something anyway, because responding is what it does. Stating if the answer is not in the documents above, say so and stop converts a confident fabrication into a handled case — and it is often the single highest-value line in a system prompt.
Debug the context first
When output is wrong, the productive first step is to look at exactly what the model received on that request — not the template, the assembled reality, with retrieved chunks and history included.
Most of the time the fault is visible immediately: the necessary passage was never retrieved, a stale turn contradicted the current one, or an instruction landed in the middle of eight thousand tokens of documents. Teams that rewrite the prompt without looking at the assembled context are tuning the smallest variable in the system.
This is worth building for rather than doing by hand. A system that can show you the exact context for any given request — ideally stored alongside the output — turns most debugging sessions from speculation into reading. It is the same argument as logging an agent's actions: the cost is small and it converts an unbounded question into a bounded one.
Dealing with this in your own group?
We answer scoping questions before there's a contract in sight — including the ones about cost and data handling.