Top
Best
New

Posted by denssumesh 4 days ago

We replaced RAG with a virtual filesystem for our AI documentation assistant(www.mintlify.com)
170 points | 85 commentspage 3
tschellenbach 3 days ago|
I think generally we are going from vector based search, to agentic tool use, and hierarchy based systems like skills.
ghywertelling 3 days ago||
Agents doing retrieval has been around for quite a while

https://huggingface.co/docs/smolagents/en/examples/rag

Agentic RAG: A More Powerful Approach We can overcome these limitations by implementing an Agentic RAG system - essentially an agent equipped with retrieval capabilities. This approach transforms RAG from a rigid pipeline into an interactive, reasoning-driven process.

The innovation of the blogpost is in the retrieval step.

skeptrune 3 days ago||
Vector search has moved from a "complete solution" to just one tool among many which you should likely provide to an agent.
dust42 3 days ago||
If grep and ls do the trick, then sure you don't need RAG/embeddings. But you also don't need an LLM: a full text search in a database will be a lot more performant, faster and use less resources.
devops000 3 days ago||
Why not a simple full text search in Postgres ?
r1290 3 days ago|
Right. Give the agent permissions to search across certain tables. Wonder why reinventing file based detect for the db side is just a current fad?
HanClinto 3 days ago||
> "The agent doesn't need a real filesystem; it just needs the illusion of one. Our documentation was already indexed, chunked, and stored in a Chroma database to power our search, so we built ChromaFs: a virtual filesystem that intercepts UNIX commands and translates them into queries against that same database. Session creation dropped from ~46 seconds to ~100 milliseconds, and since ChromaFs reuses infrastructure we already pay for, the marginal per-conversation compute cost is zero."

Not to be "that guy" [0], but (especially for users who aren't already in ChromaDB) -- how would this be different for us from using a RAM disk?

> "ChromaFs is built on just-bash ... a TypeScript reimplementation of bash that supports grep, cat, ls, find, and cd. just-bash exposes a pluggable IFileSystem interface, so it handles all the parsing, piping, and flag logic while ChromaFs translates every underlying filesystem call into a Chroma query."

It sounds like the expected use-case is that agents would interact with the data via standard CLI tools (grep, cat, ls, find, etc), and there is nothing Chroma-specific in the final implementation (? Do I have that right?).

The author compares the speeds against the Chroma implementation vs. a physical HDD, but I wonder how the benchmark would compare against a Ramdisk with the same information / queries?

I'm very willing to believe that Chroma would still be faster / better for X/Y/Z reason, but I would be interested in seeing it compared, since for many people who already have their data in a hierarchical tree view, I bet there could be some massive speedups by mounting the memory directories in RAM instead of HDD.

[0] - https://news.ycombinator.com/item?id=9224

skeptrune 3 days ago|
We would also be super interested to see that comparison. I agree that there isn't a specific reason why Chroma would be required to build something like this.
jrm4 3 days ago||
Is this related to that thing where somehow the entire damn world forgot about the power of boolean (and other precise) searching?
ctxc 3 days ago||
haha, sweet. One of the cooler things I've read lately
badgersnake 3 days ago||
So you did GraphRAG but your graph is a filesystem tree.
yieldcrv 3 days ago||
I love the multipronged attack on RAG

RIP RAG: lasted one year at a skillset that recruiters would list on job descriptions, collectively shut down by industry professionals

aplomb1026 3 days ago|
[dead]
More comments...