A Bigger Context Window Won't Save You

Part 2 · Beyond the Prompt series

When an AI agent keeps getting your codebase wrong, the answer is to decide what it gets to see. A smarter model or a longer context window rarely helps.

When an agent starts messing up or guessing on real code, the instinct is to give it more so it can understand better. Paste in more files, upgrade to the model with the million-token window, dump the whole repo in and then it'll have everything possible to make better decisions. An agent handed everything ends up having the one detail that actually mattered get buried under thousands that didn't.

Context engineering helps to fix this by having you understand that what enters the model's context window is the most important part of the tooling process.

The window is a finite budget

A language model starts every request with zero context. Everything it knows about your task, your instructions, the relevant code, the conventions, the prior steps, has to fit inside one finite context window.

The counterintuitive part is that filling it doesn't help. Important pieces of context eventually get lost in long, drawn out sessions. Past a certain point, more tokens actually buy you worse performance, because whatever you cared about is now surrounded by things you didn't. The goal is to get the right context in and keep everything else out. Newer models can handle much larger context windows, but it's not hand-holding you on what you actually fill it with.

Persistent context: the things that are always true

Some context should never have to be reiterated over and over. Your architecture, stack, code conventions, the security non-negotiables: these are true on every task in your session. Making the agent re-infer them each time is just wasting tokens.

The move is to write them down once as durable, source of truth, often called a constitution, and anchor the agent to it on every request. An agent that carries your conventions and your hard rules into each feature without being reminded behaves consistently across a whole project instead of improvising a new style every session. A page of project rules outperforms paragraphs of per-task instructions, because it governs everything rather than just the task in front of you.

Retrieval: finding the handful of files that matter

Out of thousands of files in a repo, only a handful matter for a specific task, and the model has no idea which ones. It's a consistent problem, and the heart of working in a real codebase.

You ask for a feature, and the agent writes its own date helper, its own validation, its own API client, ignoring the three you already have. The existing code simply never entered the window, so as far as the agent could tell, it didn't exist. The fix is to point the agent to your existing implementations to follow and name the modules to reuse, explicitly, so the right context is present at the moment it's needed. Anchor the work to what already exists and the agent extends your system. Leave it guessing and you wind up with code duplication, or even worse, missing business logic.

Hygiene: keep the window clean as you go

Context engineering keeps going for as long as you run the session. During all this work, stale output, disagreements and other off-task items pile up in the window, and that eventually degrades everything that comes after, because the model is now sifting through all of that excess during each subsequent command.

This is where you get rid of what's no longer important, and when a thread is genuinely exhausted, you start fresh rather than dragging its full history forward. A clean window beats a full one almost every time. Treat the context as something you maintain as you go, and quality stops fading away slowly.