Posted by robotswantdata 6/30/2025
The concept of prompting - asking an Oracle a question - was always a bit limited since it means you're really leaning on the LLM itself - the trained weights - to provide all the context you didn't explicitly mention in the prompt, and relying on the LLM to be able to generate coherently based on the sliced and blended mix of StackOverflow and Reddit/etc it was trained on. If you are using an LLM for code generation then obviously you can expect a better result if you feed it the API docs you want it to use, your code base, your project documents, etc, etc (i.e "context engineering").
Another term that has recently been added to the LLM lexicon is "context rot", which is quite a useful concept. When you use the LLM to generate, it's output is of course appended to the initial input, and over extended bouts of attempted reasoning, with backtracking etc, the clarity of the context is going to suffer ("rot") and eventually the LLM will start to fail in GIGO fashion (garbage-in => garbage-out). Your best recourse at this point is to clear the context and start over.
In OpenAI hype language, this is a problem for "Software 2.0", not "Software 3.0" in 99% of the cases.
The thing about matching an informal tone would be the hard part. I have to concede that LLMs are probably better at that. But I have the feeling that this is not exactly the feature most companies are looking for, and they would be willing to not have it for a cheaper alternative. Most of them just don't know that's possible.
In this arrangement, the LLM is a component. What I meant is that it seems to me that other non-LLM AI technologies would be a better fit for this kind of thing. Lighter, easier to change and adapt, potentially even cheaper. Not for all scenarios, but for a lot of them.