Posted by wilhelmklopp 5 days ago
https://github.com/agent-ix/engineering-assurance https://github.com/agent-ix/quoin
I don't get why we need global memory for code? Aren't code comments (even if invented for humans) the ideal place where to put "memories"?
In other words, there's a story-document between a SherlockHolmesBot and UserPlaceholder that's growing like a crystal formation. Some software sees "Sherlock Homes Says" and then "performs" the dialogue at us, and we then assume Sherlock Holmes exists with a mind and memory, rather than being a facet of a text-story.
1. Some context doesn't have an obvious place to write it in the code. If you're explaining why a tricky function is implemented a certain way, you can leave the explanation above the function or within the function. If you're explaining something more general, there might not be a natural place to put it.
2. Relatedly, some context doesn't have an obvious location to read in the code. E.g. you can comment on a schema that a table is intended to be append-only, but an agent could easily miss this comment if it doesn't gather context all the way down to the raw schema. Any research process has unknown unknowns. There may not be a canonical place to look.
3. Adding comments for every single human intent might be a bit noisy. E.g., if a code reviewer flags something that I know isn't a bug, I want the system to learn from this, and I want that knowledge to take effect outside of just code review, but I might not want every single code review thread to yield a codebase comment.
2 & 3: How to be sure that the agent will pick the "append only hint" from the memory? Especially when it starts growing and cannot be part of the context?
In addition to all the product details that aren't in the codebase or docs, like "keep this logical path because it is used by our one big client."
Mostly what I think is needed is a richer worldview available to the LLM so it understands not just the code but can understand the product and its real world usage and constraints.
Now, I agree it'd be fantastic to have that automated, but how do we then share that context with the humans that will drive the non-self-driving actions?
How can you self drive an app on windows? There's no clear UI framework, design pattern, nothing which can bullet proof your app.
I bet you'd have better luck on Plan 9.
Remove the choice. Make frameworks which have limited options which are usable by default.
I recently built a calculator to try to quantify the gains an org could expect as they become more "AI-native." I tried to account for a bunch of things including env, which I call "AI roadway," but I'm definitely missing codebase readiness (thank you, author), and also bottleneck analysis. Even without those considerations, though, gains are usually modest. It's hard to get to 2x.
Tool is here if useful: https://timvasil.com/ai-native
In general I've seen other issues like this where small errors and irrelevant comments in the codebase spin out into larger problems that consume annoying amounts of time/tokens. Maybe Anthropic and OpenAI don't notice this because they're in an "infinite monkeys with typewriters" scenario, but it's noticeable to me when the agent in my CLI has been spinning for 15 minutes contemplating irrelevant details
Sometimes I feel like it would be best to only give the agent access to clear API boundaries (say public interfaces to certain modules) and let it work out a new system from scratch given the expected inputs and outputs. Then plug this independent solution into the system. Of course it can still overfit these interfaces but it’s less than having access to the entire codebase.
Thinking about it, maybe it’s possible to let one agent extract those interfaces, then use them as grounding for a new session.
One less nice way of achieving the same is to tell the agent once in a while that it should think completely from scratch (from first principles). But this relies heavily on instruction following in the reasoning part, which sometimes works and sometimes fails.