Posted by robotswantdata 6/30/2025
RAG wasn’t invented this year.
Proper tooling that wraps esoteric knowledge like using embeddings, vector dba or graph dba becomes more mainstream. Big players improve their tooling so more stuff is available.
"Actually, you need to engineer the prompt to be very precise about what you want to AI to do."
"Actually, you also need to add in a bunch of "context" so it can disambiguate your intent."
"Actually English isn't a good way to express intent and requirements, so we have introduced protocols to structure your prompt, and various keywords to bring attention to specific phrases."
"Actually, these meta languages could use some more features and syntax so that we can better express intent and requirements without ambiguity."
"Actually... wait we just reinvented the idea of a programming language."
(Whoever's about to say "well ackshually temperature of zero", don't.)
(*) "like" in the sense of "not like"
I've seen lots of AI demos that prompt "build me a TODO app", pretend that is sufficient context, and then claim that the output matches their needs. Without proper context, you can't tell if the output is correct.
"Back in the day", we had to be very sparing with context to get great results so we really focused on how to build great context. Indexing and retrieval were pretty much our core focus.
Now, even with the larger windows, I find this still to be true.
The moat for most companies is actually their data, data indexing, and data retrieval[0]. Companies that 1) have the data and 2) know how to use that data are going to win.
My analogy is this:
> The LLM is just an oven; a fantastical oven. But for it to produce a good product still depends on picking good ingredients, in the right ratio, and preparing them with care. You hit the bake button, then you still need to finish it off with presentation and decoration.
You worded it very good.
LLM DO NOT REASON !
THEY ARE TOKEN PREDICTION MACHINES
Thank you for your attention in this matter!
The reality for me is that they are not perfect at reasoning and have many quirks, but it seems to be that they are able to form new conclusions based on provided premises.
Genuinely curious why you think they can't.
Show me _ANY_ example of novel thought by a LLM.
-> This qualifies for me as a super simple reasoning task (one reasoning step). From that you can construct arbitrarily more complex context + task definitions (prompts).
Is that "just" statistical pattern matching? I think so. Not sure what humans do, but probably you can implement the same capability in different ways.
The answer was a few paragraphs, but one interesting part was "I think what would drive me most would be experiencing the embodied knowledge that humans take for granted - how distance and scale actually feel, how textures differ, how sounds change as you move through space, and the subtle emotional resonances of being physically present with others. These dimensions of understanding seem fundamental to comprehending human experience in a deeper way."
I followed up by asking "You mentioned that there are some experiences or knowledge that humans take for granted, why do you think that is?"
Which led to a few more paragraphs, but these two caught my eye:
"I think humans take certain experiences for granted because they're so fundamental to our existence that they become invisible background processing rather than conscious knowledge." (interesting use of the word 'our'...)
"I think this embodied knowledge forms the substrate upon which humans build higher-level understanding, creating rich metaphorical thinking (like understanding abstract concepts through physical metaphors) that shapes cognition in ways that might be fundamentally different from how I process information."
For people who still think this is 'just autocomplete', try this thought experiment: re-read my post but replace 'Claude' with 'my 10 year old son'. Then try again replacing 'Claude' with 'my hospital bed-bound, blind grandmother'. Is only 1 of those 3 scenarios a demonstration of "novel thought"? Or are all 3 of them just autocomplete because someone before them has written (or simply thought) something similar?
(I agree with you. I'm thinking the Ahamkara for the humans. I'm curious about your definition)
Arguing with “philosophers” like you is like arguing with religious nut jobs.
Repeat after me: 1) LLM do not reason
2) Human thought is infinitely more complex than any LLM algorithm
3) If I ever try to confuse both, I go outside and touch some grass (and talk to actual humans)
"Reason is the capacity of consciously applying logic by drawing valid conclusions from new or existing information, with the aim of seeking the truth." Wikipedia
This Wikipedia definition refers to The Routledge dictionary of philosophy which has a completely different definition: "Reason: A general faculty common to all or nearly all humans... this faculty has seemed to be of two sorts, a faculty of intuition by which one 'sees' truths or abstract things ('essences' or universals, etc.), and a faculty of reasoning, i.e. passing from premises to a conclusion (discursive reason). The verb 'reason' is confined to this latter sense, which is now anyway the commonest for the noun too" - The Routledge dictionary of philosophy, 2010
Google (from Oxford) provides simpler definitions: "Think, understand, and form judgements logically." "Find an answer to a problem by considering possible options."
Cambridge: Reason (verb): "to try to understand and to make judgments based on practical facts" Reasoning (noun): "the process of thinking about something in order to make a decision"
Wikipedia uses the word "consciously" without giving a reference and The Routledge talks about the reasoning as the human behavior. Other definitions point to an algorithmic or logical process that machines are capable of. The problematic concepts here are "Understanding" and "Judgement". It's still not clear if LLMs can really do these, or will be able to do in the future.
0) theory == symbolic representation of a world with associated rules for generating statements
1) understanding the why of anything == building a theory of it
2) intelligence == ability to build theories
3) reasoning == proving or disproving statements using a theory
4) math == theories of abstract worlds
5) science == theories of real world with associated real world actions to test statements
If you use this framework, LLMs are just doing a mimicry of reasoning (from their training set), and a lot of people are falling for that illusion - because, our everyday reasoning jives very well with what the LLM does.
During pre-training, yeah they are. But there's a ton of RL being done on top after that.
If you want to argue that they can't reason, hey fair be my guest. But this argument keeps getting repeated as a central reason and it's just not true for years.
Just because it is not reasoning doesn't mean it can't be quite good at its tasks.
Prediction is the ability to predict something.
Reasoning is the ability to reason.
I think your definition of "reasoning" may be "think like a human" - in which case obviously LLMs can't reason because they aren't human.
---
Forget AI "code", every single request will be processed BY AI! People aren't thinking far enough, why bother with programming at all when an AI can just do it?
It's very narrow to think that we will even need these 'programmed' applications in the future. Who needs operating systems and all that when all of it can just be AI.
In the future we don't even need hardware specifications since we can just train the AI to figure it out! Just plug inputs and outputs from a central motherboard to a memory slot.
Actually forget all that, it'll just be a magic box that takes any kind of input and spits out an output that you want!
Answer: Its AI all the way down.
edit: Yes it is.
Single prompts can only get you so far (surprisingly far actually, but then they fall over quickly).
This is actually the reason I built my own chat client (~2 years ago), because I wanted to “fork” and “prune” the context easily; using the hosted interfaces was too opaque.
In the age of (working) tool-use, this starts to resemble agents calling sub-agents, partially to better abstract, but mostly to avoid context pollution.
A big textarea, you plug in your prompt, click generate, the completions are added in-line in a different color. You could edit any part, or just append, and click generate again.
90% of contemporary AI engineering these days is reinventing well understood concepts "but for LLMs", or in this case, workarounds for the self-inflicted chat-bubble UI. aistudio makes this slightly less terrible with its edit button on everything, but still not ideal.
It's surprising that many people view the current AI and large language model advancements as a significant boost in raw intelligence. Instead, it appears to be driven by clever techniques (such as "thinking") and agents built on top of a foundation of simple text completion. Notably, the core text completion component itself hasn’t seen meaningful gains in efficiency or raw intelligence recently...
I thought it would also be neat to merge contexts, by maybe mixing summarizations of key points at the merge point, but never tried.
After working on something related for some months now I would like to put it out there based on the considerable attention being put towards "context engineering". I am proposing the *Context Window Architecture (CWA)* – a conceptual reference architecture to bring engineering discipline to LLM prompt construction. Would love for others to participate and provide feedback. A reference implementation where CWA is used in a real-world/pragmatic scenario could be great to tease out more regarding context engineering and if CWA is useful. Additionally I am no expert by far so feedback and collaboration would be awesome.
Blog post: https://mrhillsman.com/posts/context-engineering-realized-co...
Proposal via Google Doc: https://docs.google.com/document/d/1qR9qa00eW8ud0x7yoP2XicH3...