Posted by Luc 12 hours ago
It's perfectly safe to use LLMs to surface underlying data and knowledge without having to trust that it's giving you correct advice.
Use the LLM as search technology: string match -> semantic search -> search intelligence.
This is especially helpful when you don't know the right question to ask. Semantics only help you find text that is semantically near the input query. Semantics won't help you when what you are looking for is connected to what you have by something other than semantics, like causation, mechanism, or interpretation.
Trying to figure out why your crops aren't growing well will likely require you to read several articles across a range of sciences, probably scoped to the specific conditions you are dealing with. A semantic search is not going to surface those first.
Allowing an LLM to chain together several sources for you to read first does not require you to ask it for a conclusion.
They're amazingly useful tools, but they're nowhere the level believers want them to be, not yet.
I tried using dictation but I really disliked speaking to it.
You can write a really messed up sentence and it almost always worked out what I meant.
Quite amazing actually
its got a ton of useful options, preconfigured, integrates pretty easily, and can use any local LLM or data.
How can I build a bow drill so I can start a fire?
It got it partially right: spindle, bow with string tied the correct way, and fire board. But then it flubbed by saying you put the spindle between both hands and move your hands back and forth to rotate the spindle and create the necessary friction… this totally ignores using the bow to rotate the spindle! It also mentioned “fanning the sparks” but there is absolutely no way your bow drill is going to create sparks.
I asked a frontier model to check the survival LLM’s conversation trace and it pointed out the errors.
I didn’t see obvious errors in the other 3 prompts I used, but this one seemed pretty egregious for a survival LLM.
I thank the creator for making this, but take it with a grain of salt.
This seems redundant and requiring more overhead while providing less reliability than something like Kiwix to me.
That way the tiny model isn't acting as a potentially incorrect interpreter but you still get semantic search to help you find the right documents quickly.
I suppose there is something to be said for a good summarization, I just worry that these tiny models aren't really GOOD summarizers.
I'm all up for using the better/simpler task for the job. That's been a thing lately also with the Jev/decision models hype, most of those things can either already be done by a pretrained classifier[0], or you can start with a bigger model and then train your smaller/adhoc one when you have some data (it doesn't even take much for most applications, you can get about 95% accuracy on email classification with 50-100 examples)
0: http://jeffyclassify.com/blog/when-llms-fail-at-classificati...
The bigger issue in this context would be canning, such as incorrectly explaining the mechanisms to avoid botulism (acid and/or pressure canning), or perhaps incorrectly wiring in repairs. However, again, it’s best to practice with these prior to emergencies anyway.
It should be noted though, that the model does source its claims, so you can fact check it and should review this information regularly.
Consider they won't have access to the plentiful practice opportunities we have now, most resources would have to be spent with a practical goal.
A field guide to foraging for edible mushrooms will probably be more practical than naïve use of an LLM: “can I eat this mushroom?” is an exercise in trust that I will maybe never be comfortable with.
But “how do I effectively forage for edible mushrooms” is a great stepping stone.
One of these situations prohibits the other.
Kiwix is great, but I'd also like a language model to be able to deeply interrogate all of that information much faster than I can read.
Just arrange its harness so that its responses are grounded in the source material and require source citations for everything. Then read the citations to confirm it's not a hallucination.
Except the build process there requires a specific recent version of swift. Hardly a good definition of "anything"
You can run bonsai2 27b on basically any gaming GPU from the last 15 years including on game consoles.
Everyone has access to this stuff now if they bothered to look beyond the marketing from big labs trying to collect subscription revenue.
I work on compilers, kernels, and operating systems using exclusively consumer GPUs.
If things are so bad that you need to factor in the non-operability of any electronic devices, how many indestructible paper libraries are you planning to build and manage in advance, and at what budget?
If there's any place to get an answer it's here.
Bravo my friend. And I mean that in a good way. Love me some diversity.
That seems to have been mostly solved through scale. I was asking Opus 5.5 about some obscure stuff and it kept hedging "this is from memory since I don't have internet access" (it's sandboxed), "I'm reasonably sure about this but not about the numbers" etc.
Even though it was obscure enough that Google couldn't easily find the paper that Opus remembered (I couldn't find it at all with the traditional search results; the AI summary found it on my third try), it actually didn't hallucinate it.
In fact I can't remember Opus 5.5 ever hallucinating. They've clearly done some work on that.
Obviously this local LLM is going to hallucinate all over you though.
Undoubtedly useful in the right contexts, particularly for local repair or homesteading, but “survival” does imply that this would be more useful in an SHTF situation.
Also, I am surprised that the difference between 32GB and 128GB in right answers is so small:
* Mac with 128 GB or more memory NVIDIA Nemotron 3.5 Super 21 of 22 right 80 GB
* Windows PC with 32 GB or more NVIDIA Nemotron 3.5 Lightning 19 of 22 right 25 GB
So on a regular consumer machine this is going to be wrong nearly half the time. Great! Let’s hope it’s only on the simple stuff and not the “how do I perform an emergency appendectomy”.
Honestly, just buy some books.
I’m pretty sure it also told you how to butcher a sheep.
I used to be obsessed with it as a kid. The spectre of nuclear war and a three volume handbook on how to survive the impending apocalypse with nothing more than a plastic bucket, a packet of potassium permanganate, some charcoal and string. The late 80s and early 90s were a wild time to be a kid.
That said, I'm really looking forward to small and quick (latency wise) models are a reality. I can easily see a world where every keystroke, terminal command, etc. goes through a "autocorrect" type model. You could even blend natural language plus terminal program segments and the AI just infers what you intend. Either built-in or a second classifier model, determines if the autocorrect change needs human review first.
Not sure what they do on the windows side.
This kind of stuff really feels like knowledge that probably shouldn't be offload to AI but rather learned and practiced. Again, I don't have a bunker in my backyard but seems like a weird use case.
I have some property that doesn't really get cell reception in the summer, and even in the winter the reception is at-best spotty. I use granite and qwen together with a manually curated set of books, articles, etc. to get work done while disconnected. I tend to update the set of resources before leaving town because I'm generally intentionally disconnecting to focus on deep work. Building a workflow that makes that possible has helped me build out a setup that replaces a ton of my day-to-day anthropic/openai use even when I'm near fiber.
It is interesting how, even pre-llms, offline computers change the nature of a lot of activities. E.g., about once a year I'll walk do a random walk in the woods for a few days without paying much attention to maintaining a bearing. With maps and compasses it's a whole skill set to find your way back. With a phone and downloaded maps you just kinda press the power button and check your heading every hour or two.
The "survival" use-case is interesting. I fear a lot of the practical applications of that knowledge are either time-sensitive (e.g., RAG on a "Stop The Bleed" course won't help because by the time you learn the patient is dead) or require tools+skills that aren't easily improvised completely from scratch in high-stress situations (e.g., building shelter or even really simple stuff like building a fire when it's very cold out). And for other stuff common sense prevails (e.g., power out -> keep fridge closed).