A path for improving LLM coding tools

Introduction

One thing you might notice when working with LLM-based coding tools is how much they struggle to get the right information into their context windows. LLMs usually know almost everything there is to know about libraries, code patterns, algorithms, language syntax, etc. When you ask them to plan and build a greenfield application—provided you specify correctly what you want—they usually do a fairly good job. However, if you ask them to change a specific feature in a somewhat large codebase, they often struggle to find their way around, frequently introducing bugs and duplicating code.

When using Claude Code, I've noticed two patterns: it either searches for information by reading entire files, or it reads small chunks using tail/head/grep. Now imagine if any human had to work like this—constantly choosing between bringing entire documents into your mind (your "context window") or sipping bits of information through trial and error as you try to find your way around. In this mode, there's no opportunity to quickly scan documents with your eyes, spend more time on one section, quickly discard what you've just read, or compress previously read information so you know where to return if needed. And I'm not even trying to be exhaustive about this problem.

Not only an LLM struggle

Now, I don't know about other developers, but I don't think LLMs are the only ones to struggle with code organization and navigation. We've come a long way with search engines and all, but with code it still feels like we're using abstractions that were adequate back when the first higher-level programming languages were created, some 30 or 40 years ago. As a developer who has used many editors—from Notepad in the '90s to PyCharm nowadays—I should say I got an immense productivity boost when I adopted JetBrains tools. While a range of IDE and text editor options exist nowadays, one thing about JetBrains IDEs keeps me locked in: their amazing capability for indexing and tracking references in a codebase. It's obviously a good idea to stay organized, especially when collaborating with others—or with your future self—but once you use a tool like that and give the IDE the proper syntactic clues (type annotations and such), navigating a codebase and finding what calls what becomes mostly a matter of ⌘-clicking function calls to see where they go, often even through library code.

A different abstraction

But thinking about this, what I believe the IDE is doing is essentially emulating an abstraction where we're no longer using files. There are variables, functions, and classes, and they do need to be written somewhere in a specific order for the compiler or interpreter, but when navigating them, we're jumping between them as if they were entities not tied to any specific location—simply nodes in a graph. The code could just as well be written in a single file, tape, or context window—as long as you can avoid name collisions (through some sort of namespacing) and quickly find and retrieve the right 'item,' you're done.

But why should only humans get to have nice things? Sure, taking this reasoning to its limits, we might conclude that the problem lies with modern programming languages themselves. We have modules, packages, crates, and all sorts of file-based code organization, but to my knowledge, none require each node to have its own file while being easily referenceable through the filesystem. Following this logic, only by creating a new language free of such design flaws could we solve the problem. Or perhaps this problem isn't even worth solving, or solving it would create even less manageable problems than the one described above: imagine the entire Linux kernel source code after preprocessing, in a single file, with every contributor working on part of this monolithic system. I'm certainly not proposing that.

On the other hand, there does exist a system where a "single file" isn't messy at all. Consider an SQL database—I'll use Postgres since it's the one I'm most familiar with. You have a database that contains schemas (think namespaces), which contain a myriad of objects: tables, views, procedures, constraints, indexes, etc. You may have a single-digit number of any of those, or hundreds of them. Within each table you have rows that might reference other rows, and within procedures you can reference other procedures in the same or different schemas, tables, and so on. None of this is tied to a "file"—you never grep a row and end up getting nearby rows as well, or end up getting just part of the row. Everything is directly addressable.

Obviously, this abstraction isn't free—there are files underneath it, along with a planner, indices, and sophisticated saving, syncing, and all sorts of machinery to keep this abstraction running. But compared to the work an LLM has to do by searching code through grep/head/tail and sending the results over the web, where GPU clusters process this through millions or billions of weights just to figure out that the function we need to edit is on line 27 of something.py, extending down to line 42—a relational database seems like a pretty cheap abstraction to me.

And if we're going down the rabbit hole, we could eventually even have the entire AST mapped through relationships, so that renaming anything could be easily achieved with an UPDATE, and bad references could be prevented by foreign keys.

Back to reality

But this thought experiment is going too far, and going that deep will certainly reveal problems I'm not even aware of—problems that someone with compiler experience will probably spot instantly.

But maybe there's a version of the problem that solves a large chunk of the problem with only a small subset of the complexity: what if we just indexed namespaces?

Suppose an LLM could query:

SELECT method FROM classes WHERE class_name = 'MyClass' AND module = 'mypackage.mymodule';

Or even SELECT body for that class if that's what the model wanted to see? Or select line numbers?

Conclusion

I believe that while the file abstraction might be too ingrained in our work to move away from, and while pulling the entire AST into a relational database might be too much to start with, indexing namespaces in a relational database might be a good start. I also wonder why JetBrains, which has already done much of the hard work, doesn't provide such an interface to LLMs working within their IDEs.

In any case, making this kind of index available through an MCP server doesn't seem like a huge endeavor, and it might be something I try in the near future—which makes me think: why doesn't this exist yet? Maybe I'll find out when I try to implement it.