Posted by alash3al 10 hours ago
Not casting aspersions on you personally, I’d really like this from every project, and would do the same myself.
I doubt many people will honestly admit they did no design, testing and that they believe the code is sub par.
It does give me an idea that maybe we need a third party system which can try and answer some of the questions you are asking… of course it too would be LLM driven and quite subjective.
I'd doubt any engineer that doesn't call most of their own code subpar after a week or two after looking back. "Hacking" also famously involves little design or (automated) testing too, so sharing something like that doesn't mean much, unless you're trying to launch a business, but I see no evidence of that for this project.
Well no, but if people want to see a statement like this, and given that most people will want to be at least halfway honest and not admit to slop, maybe it will help nudge things in the right direction.
If you care that much and don't have a foundation of trust, you need to either verify the construction is good, or build it yourself. Anything else is just wishful thinking.
We even ask when cakes are made in house or frozen even though they look and taste great (at first).
1) An up-to-date detailed functional specification.
2) A codebase structured and organized in multiple projects.
3) Well documented code including good naming conventions; each class, variable or function name should clearly state what its purpose is, no matter how long and silly the name is. These naming conventions are part of a coding guidelines section in Agent.md.
My functional specification acts as the Project.md for the agent.
Then before each agentic code review I create a tree of my project directory and I merged it with the codebase into one single file, and add the timestamp to the file name. This last bit seems to matter to avoid the LLM to refer to older versions and it’s also useful to do quick diffs without sending the agent to git.
So far this simple workflow has been working very well in a fairly large and complex codebase.
Not very efficient tokens wise, but it just works.
By the way I don’t need to merge the entire codebase every time, I may decide to leave projects out because I consider them done and tested or irrelevant to the area I want to be working on.
However I do include them in the printed directory tree so the agent at least knows about them and could request seeing a particular file if it needs to.
Digging deeper I can see it is effectively pg_vector plus mcp with two functions: "recall" and "remember".
It is effectively a RAG.
You can make the argument that perhaps the data structure matters but all of these "memory" systems effectively do the same and none of them have so far proven that retrieval is improved compared to baseline vector db search.
In a way, if it does accomplish that, it is a vectordb needing glorification.
The only approach I've found that works is no memory, and manually choosing the context that matters for a given agent session/prompt.
A friend told me he would like Claude to remember his personality, which is exactly what Gemini is trying to do.
A machine pretending to be human is disturbing enough. A machine pretending to understand you will spiral very far into spitting out exactly what we want to read.
In practice, as it grows it gets just as messy as not having it.
In the example you have on front page you say “continue working on my project”, but you’re rarely working on just one project, you might want to have 5 or 10 in memory, each one made sense to have at the time.
So now you still have to say, “continue working on the sass project”, sure there’s some context around details, but you pay for it by filling up your llm context , and doing extra mcp calls
If I am working on a real project with real people, it won’t have the complete memory of the project. I won’t have the complete memory. My memory will be outdated when other PRs are merged. I only care about my tickets.
I am starting to think this is not meant for that kind of work.
How does it fight context pollution?