I’m surprised Karpathy thinks this is a viable solution for a quasi-continual learning system. Yes, it’s cool to experiment with these sort of ‘intermediate knowledge systems’, but the real goal within this current LLM paradigm remains clear: new information within a knowledge system should be manifest through updating the weights! Many efforts are taking a crack at this, which this really helpful talk [0] by Jack Morris goes into. I’m in full agreement with the other comments here: this ‘LLM Wiki’ merely results in “2nd-order information” that will only muddy the picture.

[0] “Stuffing Context is not Memory, Updating Weights Is": https://www.youtube.com/watch?v=Jty4s9-Jb78

A list of systems that implement this or related ideas: https://zby.github.io/commonplace/notes/related-systems/rela...

This list is also part of my own contender in this race: https://zby.github.io/commonplace/ - my own LLM operated knowledge base (this is the html rendering of that KB - there is also the github repo linked there).

The main feature is that I use it to build a theory about such systems - and the neat trick is that llms can read this theory and implement it so the very theory works as an LLM runtime too.

It works for me - but it has some rough edges still - so I guess it is not for everyone.

I don't think this is taking it as far as it can go.

Everything should live in the repo. Code and docs yes. But also the planning files, epics, work items, architectural documentation and decisions. Here is a small example of my Linux system doc: https://github.com/gchamon/archie/tree/main/docs

And you don't need to reinvent the wheel. Code docs can like either right next to it in the readme or in docs/ if it's too big for a single file or the context spams multiple modules. ADRs live in docs/architecture/decisions. Epics and Workitems can also live in the docs.

Everything is for agents and everything is for humans, unless put in AGENTS.md and docs/agents or something similar, and even those are for human too.

In a nutshell, put everything in the repo, reuse standards as much as possible, the idea being it's likely the structure is already embedded in the model, and always review documentation changes.

I built AS Notes for VS Code (https://www.asnotes.io) with the option for this usage pattern in mind. By augmenting VS Code so it has the tooling we use in personal knowledge management systems, it makes it easy to write, link and update markdown / wikilinked notes manually (with mermaid / LaTeX rendering capability also) - but by using VS Code we have easy access to an Agent harness that we can direct to work on, or use our notes as context. Others have pointed out that context bloat is an issue, but no more so than when you use the copilot harness (or any other) inside a large codebase. I find I get more value from my AI conversations when I persist the outputs in markdown like this.

Built a versioned schema standard on top of this workflow: github.com/arturseo-geo/llm-knowledge-base

I built a tool[0] in a similar vein to this. It’s for the code base specifically, uses hashes to detect source file changes and distills everything down with an llm into a single asset that explains each file. All addressable via a cli.

I find it helps a LOT with discovery. Llm spends a lot less time figuring out where things are. It’s essentially “cached discovery”

[0] https://github.com/tristanmatthias/llmdoc

I built a tool to exactly help an LLM navigate and search a knowledgebase of md files. It helps a lot to build such a wiki by providing basic content search à la grep but also structured search for frontmatter properties. It also helps to move files around without breaking links and to fix links automatically. It is a CLI tool, mainly meant to be driven by AI tools.

Check it out: https://github.com/ractive/hyalo

An LLM that maintains a Confluence space. That looks like an interesting idea!

I built a very similar system into my own assistant type project. In all honesty though I've not used it enough to know how well it works out in practice.

Sounds like a solution in search for a problem.