The Context Problem: Why Your AI Forgets Your Codebase

If you learn one technical thing about working with AI on code, make it this one. Nearly every strange behavior — the forgotten function, the duplicate component, the fix that breaks something else — traces back to the same mechanism.

The mechanism

A language model has no memory of your project. None. Each time it responds, it is handed a bundle of text — your message, some conversation history, and whatever parts of your codebase the tool decided to include — and it reasons over exactly that bundle. Nothing else exists to it.

That bundle is the context window, and it has a hard size limit.

This means the practical question in every AI coding session is not "how smart is the model" but "did the right code make it into the bundle?" A frontier model looking at the wrong files will perform worse than a mid-tier model looking at the right ones.

What this explains

"It wrote a function that already exists." The existing one wasn't in the bundle. From the model's view, it didn't exist, and writing it was correct.

"It used an old version of my API." Either your current version wasn't included, or it was and the model leaned on training data instead. This is why documentation-injection tools like Context7 exist.

"It fixed the bug but broke three other things." The three other things weren't visible. The fix was locally correct and globally wrong — the defining failure of large-codebase AI work.

"It was great last week and it's useless now." Your project grew past the point where it fits. Same model, same prompting, different fraction of the codebase visible.

"It keeps changing its mind about how to do things." Different requests pulled in different files, so it pattern-matched on different conventions each time.

Bigger windows don't solve it

Context windows have grown enormously, and some models now advertise capacities that could hold a whole repository. This helps, and it does not solve the problem, for two reasons.

First, attention degrades across long contexts. A model given 500,000 tokens does not attend equally to all of them. Details in the middle of a huge context get missed in ways they wouldn't in a focused one. Researchers call this "lost in the middle," and it means a precisely chosen 20,000 tokens routinely beats an undifferentiated 500,000.

Second, cost and latency scale with input. Stuffing everything into every request is slow and expensive, which is why no tool actually does it by default.

So every tool makes choices about what to include. Understanding that those choices exist — and that you can influence them — is most of the skill.

What you can actually do

Say which files matter. Nearly every tool lets you reference specific files explicitly. Doing this is the highest-leverage habit available. "Fix the checkout bug" is a search problem for the tool; "fix the bug in checkout.js where the total ignores the discount, see cart.js for how discounts are calculated" hands it the bundle directly.

Keep a project instructions file. Most tools read a designated file — CLAUDE.md, .cursorrules, and equivalents — into every request. Put the things that are always true there: the stack, the conventions, the data model, the directories to avoid. This is you manually fixing the context problem, permanently, in one place.

Start fresh sessions for new tasks. Long conversations fill the window with irrelevant history. When you change tasks, start over. Carrying an hour of unrelated debugging into a new feature request actively hurts.

Keep files small. A 2,000-line file forces an all-or-nothing choice. Ten 200-line files let the tool bring in only what's relevant. Modular code isn't just an aesthetic preference anymore — it's directly upstream of how well AI can work on your project.

Give it tools instead of text. This is the real unlock. Rather than pasting your schema, connect a database MCP server and let it query the schema. Rather than describing the bug, let it drive a browser and see the error. Retrieved-on-demand context beats stuffed-in-advance context, because the model pulls exactly what it needs when it needs it. See MCP explained for how this works.

Delete things. Dead code, abandoned features, unused files — all of it competes for space and confuses retrieval. Deleting a feature you don't need measurably improves AI performance on everything else.

The reframe

Stop thinking of yourself as someone giving instructions to an assistant. Think of yourself as someone assembling the briefing that a very capable contractor will work from, having never seen your project before, with no ability to ask a colleague.

That contractor is genuinely excellent. They will also do exactly the wrong thing, confidently, if the briefing omits something important.

Most of what people call "prompt engineering" in a coding context is really context engineering: deciding what goes in the bundle. The wording of your request matters far less than whether the relevant code was in front of the model when it read it.

---

Related: Why vibe-coded projects stall at 80% — the context ceiling is the first of five mechanisms behind the stall.