Ask HN: How are you doing RAG locally?

Posted by tmaly 1 day ago

I am curious how people are doing RAG locally with minimal dependencies for internal code or complex documents?

Are you using a vector database, some type of semantic search, a knowledge graph, a hypergraph?

321 points | 129 commentspage 5

andoando 4 hours ago|

Anyone have suggestions for doing semantic caching?

Bombthecat 13 hours ago||

AnythingLLM for documents, amazing tool!

SamLeBarbare 11 hours ago||

sqlite + FTS + sqlite-vec + local LLM for reranking results (reasoning model)

yandrypozo 5 hours ago|

this's pretty cool, which LLM are you using currently?

lsb 12 hours ago||

I'm using Sonnet with 1M Context Window at work, just stuffing everything in a window (it works fine for now), and I'm hoping to investigate Recursive Language Models with DSPy when I'm using local models with Ollama

softwaredoug 13 hours ago||

I built a Pandas extension SearchArray, I just use that (plus in memory embeddings) for any toy thing

https://github.com/softwaredoug/searcharray

dvorka 14 hours ago||

Any suggestion what to use as embeddings model runtime and semantic search in C++?

mooball 5 hours ago||

i thought rag/embeddings were dead with the large context windows. thats what i get for listening to chatgpt.

beret4breakfast 13 hours ago||

For the purposes of learning, I’ve built a chatbot using ollama, streamlit, chromadb and docling. Mostly playing around with embedding and chunking on a document library.

sidrag22 12 hours ago|

i took a similar path, i spun up a discord bot, used ollama, pgvector, docling for random documents, and made some specialized chunking strategies for some clunkier json data. its been a little while since i messed with it, but i really did enjoy it when i was.

it all moves so fast, i wouldnt be surprised if everything i made is now crazy outdated and it was probably like 2 months ago.

ehsanu1 14 hours ago||

Embedded usearch vector database. https://github.com/unum-cloud/USearch

juleshenry 7 hours ago|

SurrealDB coupled with local vectorization. Mac M1 16GB

More comments...