Seems like it would be simpler to give the agent tools to issue ChromaDB (or SQL) queries directly, rather than giving the LLM unix-like tools that are converted into queries under the hood using a complicated proprietary setup.

This is one of the most confusing claims I've seen in a long time. Grep and others over files would be the equivalent of an old fashioned keyword search where most RAG uses vector search. But everything else they claim about a file system just suggests that they don't know anything about databases.

I'm not familiar with how most out of the box RAG systems categorize data, but with a database you can index content literally in any way you want. You could do it like a filesystem with hierarchy, you could do it tags, or any other design you can dream up.

The search can be keyword, like grep, or vector, like rag, or use the ranking algorithms that traditional text search uses (tf-idf, BM25), or a combination of them. You don't have to use just the top X ranked documents, you could, just like grep, evaluate all results past whatever matching threshold you have.

Search is an extremely rich field with a ton of very good established ways of doing things. Going back to grep and a file system is going back to ... I don't know, the 60s level of search tech?

The real thing I think people are rediscovering with file system based search is that there’s a type of semantic search that’s not embedding based retrieval. One that looks more like how a librarian organizes files into shelves based on the domain.

We’re rediscovering forms of in search we’ve known about for decades. And it turns out they’re more interpretable to agents.

https://softwaredoug.com/blog/2026/01/08/semantic-search-wit...

Relative to making docs accessible to AI via filesystem tools, I've been looking around to see what kinds of patterns SDK authors are using to get AI coding agents to use the freshest documentation, and Vercel is doing something interesting with their AI SDK that I haven't seen elsewhere (although maybe I just haven't looked hard enough).

The "ai" npm package includes a root-level docs folder containing .mdx versions of the docs from their site, specific to the version of the package. Their intended AI-assisted developer experience is that people discover and install their ai-sdk skill (via their npx skills tool, which supports discovery and install of skills from most any provider, not just Vercel). The SKILL.md instructs the agent to explicitly ignore all knowledge that may have been trained into its model, and to first use grep to look for docs in node_modules/ai/docs/ before searching the website.

https://github.com/vercel/ai/blob/main/skills/use-ai-sdk/SKI...

I am really enjoying this renaissance in CLI world applications. There's so much possible.

I'm working on a related challenge which is mounting a virtual filesystem with FUSE that mirrors my Mac's actual filesystem (over a subtree like ~/source), so I can constrain the agents within that filesystem, and block destructive changes outside their repo.

I have it so every repo has its own long-lived agent. They do get excited and start changing other repos, which messes up memory.

I didn't want to create a system user per repo because that's obnoxious, so I created a single claude system user, and I am using the virtual file system to manage permissions. My gmail repo's agent can for instance change the gmail repo and the google_auth repo, but it can't change the rag repo.

Edit: I'm publishing it here. It's still under development. https://github.com/sunir/bashguard

I don't know - we are discussing techniques - like having information in files, or in a semantic database, or in a relational database - as if there was one way that could dominate all information access. But finding the right information is not one task - if the needed information is a summary of expenses from a period of time then the best source of it will be a relational database, if it is who is the head of the HR department in a particular company - then it could probably be easy found on the company intranet pages (which are kind of graph database). It does not really matter much if the searcher is a human or LLM - there are some differences in the speed, the one time useful context length and the fact that LLMs are amnesiac - but these are just parameters, the task for humans is immensely complicated and there is no one architecture and there will not be one for LLMs.

I also vibed a brainstorming note with my knowledge base system. The initial prompt: """when I read "We replaced RAG with a virtual filesystem for our AI documentation assistant (mintlify.com)" title on HackerNews - the discussion is about RAG, filesystems, databases, graphs - but maybe there is something more fundamental in how we structure the systems so that the LLM can find the information needed to answer a question. Maybe there is nothing new - people had elaborate systems in libraries even before computers - but maybe there is something. Semantic search sounds useful - but knowing which page to return might be nearly as difficult as answering the question itself - and what about questions that require synthesis from many pages? Then we have distillation - an table of content is a kind of distillation targeting the task of search. """ Then I added a few more comments and the llm linked the note with the other pages in my kb. I am documenting that - because there were many voices against posting LLM generated content and that a prompt will be enough. IMHO the prompt is not enough - because the thought was also grounded in the whole theory I gathered in the KB. And that is also kind of on topic here. Anyway - here is the vibed note: https://zby.github.io/commonplace/notes/charting-the-knowled...

This mirrors something we ran into building an AI pipeline for audio content. The problem with traditional RAG is that chunking destroys the structure that actually matters — you end up retrieving fragments that are semantically similar but contextually useless.

The filesystem metaphor works because it preserves heirarchy. Documents have sections, sections have relationships, and those relationships carry meaning that gets lost when you flatten everything into embeddings.

Curious how this handles versioning though. Docs change constantly and stale context fed to an LLM is arguably worse than no context at all.

And you did not teach it to access chroma directly, because there is no adapter? Or because it is so much better at using FS tooling?

But in the end, I would expect, that you could add a skill / instructions on how to use chromadb directly

To be honest, I have no idea what chromadb is or how it works. But building an overlay FS seems like quite lot of work.

I am not familiar with the tech stack they use, but from an outsider point of view, I was sort of expecting some kind of fuse solution. Could someone explain why they went through a fake shell? There has to be a reason.

This is definitely the way. There are good use cases for real sandboxes (if your agent is executing arbitrary code, you better it do so in an air-gapped environment).

But the idea of spinning up a whole VM to use unix IO primitives is way overkill. Makes way more sense to let the agent spit our unix-like tool calls and then use whatever your prod stack uses to do IO.