01What context engineering actually means

When you use a chat model, it doesn't have memory of your whole project or access to the open web by default. It only responds to the text placed in front of it for that one request — the context window. Context engineering is the practice of curating, structuring, and maintaining everything that occupies that window during inference so the most useful signal is present and the model behaves the way you want. Anthropic frames it as the natural next step after prompt engineering: prompt engineering is mostly about writing the instructions, while context engineering manages the full set of tokens present when the model runs — not just the prompt.

• The window can hold many things at once: system instructions, few-shot examples, retrieved documents, tool definitions and results, and the running conversation.
• Prompt engineering ≈ wording the instructions; context engineering ≈ managing everything in the window at inference time.
• The goal is not "more text" — it's the right text, in the right order, within a fixed budget.

02The window is a finite budget

Everything you put in the context window is measured in tokens — roughly, chunks of text. The window has a fixed size, so every component competes for the same limited space. A long system prompt leaves less room for retrieved documents; a giant pasted document leaves less room for the conversation. When the total exceeds the limit, something has to give: content gets truncated (cut off) before the model ever sees it. That is why relevance and ordering matter so much — you are spending a budget, and what you cut is just as important as what you keep. The interactive below makes the trade-off concrete.

• Push the total over 16,000 and the demo truncates the lowest-priority content first — it never reaches the model.
• Tighten retrieval and summarize history are real techniques: they shrink components so the important signal still fits.
• Bigger windows help, but they are still finite — the budgeting problem doesn't disappear, it just moves.

03"Lost in the middle": where you place it matters

Fitting information into the window is only half the battle — where it sits also changes how well the model uses it. A widely cited study, Lost in the Middle (Liu et al., 2023), found that language models tend to use information best when it appears at the beginning or end of a long-context input (shown in retrieval/QA studies), and show degraded recall for information buried in the middle. Performance can also fall as the context gets longer overall. Anthropic describes a related tendency it calls "context rot": as the number of tokens grows, the model's ability to accurately recall any given detail tends to decrease — sometimes before the hard limit is even reached. These are empirical tendencies, not iron laws — they differ by model and task, and some newer models mitigate them — but they explain a lot of practical advice.

• Put the most important material near the top or bottom of a long context, not buried in the middle.
• More tokens is not automatically better: a longer, noisier window can lower recall of the detail you actually care about.
• Vendor tip: place long documents near the top, wrap distinct inputs in tags/metadata, and ask the model to quote the relevant passages first before answering.

04The ingredients you're budgeting

Three of the most important things that compete for window space are worth understanding on their own, because each is a lever you can pull.

In-context learning & few-shot examples. Introduced with GPT-3 (Brown et al., 2020), in-context learning lets a model perform a task from a few examples placed in the prompt — without updating its weights. Add a few worked examples and the model imitates the pattern. Chain-of-thought prompting (Wei et al., 2022) is a related move: include exemplars that show intermediate reasoning steps, and complex reasoning improves. Both spend tokens to buy behaviour.

Retrieval-augmented generation (RAG). Rather than rely only on what's baked into the weights, RAG (Lewis et al., 2020) retrieves relevant passages at inference time and adds them to the context, combining parametric memory (weights) with non-parametric memory (retrieved text) to improve factuality. Adaptive variants like Self-RAG let the model decide when to retrieve and judge whether a passage is even relevant — controlling what actually enters the window.

Long context vs. retrieval. Some models now offer very large windows — Google documents 1M+ token windows for Gemini — which can sometimes replace a separate retrieval pipeline for "chat with your data." But long context and retrieval are complementary, not rivals: a big window still benefits from good selection and ordering, and retrieval keeps the window focused.

05Techniques that stretch the budget

When the useful content won't fit, you have options beyond "delete things." These are the workhorses of context engineering — and several appeared as toggles in the interactive above.

• Prompt compression. Methods like LLMLingua / LongLLMLingua reduce a prompt's token count while preserving its key signal, lowering cost and latency — and, in long-context settings, the LongLLMLingua work reports it can help mitigate position bias.
• History summarization. Instead of carrying every past turn verbatim, compress older conversation into a short summary, freeing budget for new, relevant content.
• Memory management. Agent memory systems (MemGPT-style tiered/paged memory) virtually extend the usable window by swapping information in and out as needed, like an operating system paging memory.
• Adaptive retrieval. Self-RAG-style approaches retrieve only when needed and filter for relevance, so retrieved text doesn't crowd out everything else.
• Bigger windows under the hood. Positional methods (RoPE interpolation, LongRoPE) extend the trained length, and systems techniques like Ring Attention serve very long sequences across devices — but figures from these papers are model- and dataset-specific, so treat reported numbers as illustrative, not universal.

06Check your understanding

07Take it with you & go deeper