Why AI Struggles With Large Codebases

A few months ago, I found myself chasing a feature that I was convinced didn't exist.

The task seemed straightforward. I needed to understand how a particular workflow behaved, so I searched the repository for the implementation. Nothing. I searched again using the name from the design documents. Still nothing. After a few more searches I reached what felt like the obvious conclusion: the feature had never been shipped.

The problem was that I was wrong.

The feature existed exactly where it should have been. It was running in production. Other parts of the system depended on it. The only thing missing was the name I expected to find. Somewhere between the original design and the final implementation, the feature had been renamed.

At first this felt like a small mistake. A bad search query, a momentary misunderstanding.

What actually went wrong is that I treated "no results" as an answer. It isn't one. A search that finds something tells you the thing is there. A search that finds nothing tells you only that the string you typed didn't appear — and I had typed the name from the design document, not the name someone had chosen while implementing it.

Nothing in that empty result separated "this was never built" from "this is called something else." I supplied the distinction myself, wrongly, because I had to supply something to keep working.

An assistant hits the same wall, faster

Coding assistants search. They grep, follow imports, open files, and assemble a picture of the system — often faster than I did that afternoon. What they inherit along with that ability is the same empty result and the same inability to read it.

The difference is what happens next. I eventually got up and asked someone who had been on the rename. An assistant has to act on what it found, so a miss becomes "this doesn't exist," and the reasonable next step is to write the thing. That's how a repository ends up with two of something.

The size of the repository matters here only because it decides how likely you are to guess the right name. In a project you wrote last month, the name in your head is probably the name in the file. In one you inherited, or one that has been through a few renames, it frequently isn't.

What I wanted the tooling to say

Not a better ranking of results. An exact answer to a narrow question: is there a symbol with this name, and if not, what's close to it? Parsed symbols can answer that, including a real "no" — and when the index can't answer, it can say which commit it's answering from instead of guessing.

That's most of what cix turned into. The rename that started all this would still have happened, and the code would still have moved. But "does this exist under some other name" would have been a lookup rather than an inference, and the empty answer I got would have meant something.