Posted by tosh 13 hours ago
Generalized Speculation: It seems that for any public-facing AI/LLM, their system prompts will, due to regulation and other issues, legal and otherwise, similarly grow larger and larger over time...
Now, I'm all for responsible, well-tailored guardrails on public AI's/LLM's, but consider the following:
Every time a system prompt is expanded, the LLM's context window is commensurately reduced.
Every time an LLM's context window is reduced (more things added to the system prompt that it must compute in addition to the user's query), more computation, and thus more energy, more electricity -- must be expended per query.
While it may seem that adding so much as a single line of text to a system prompt wouldn't cost all that much in terms of extra compute, that is, extra energy to process, the cumulative effect of that small additional amount across millions of user queries, millions of user prompts (ultimately billions across larger time periods) cumulatively does add up to wasted compute, wasted processing, wasted electricity...
Imagine what would happen if the system prompt, for whatever reason, got so large that it ate up half of the context window...
If that happened, then at least half of all of the LLM's processing and compute/energy costs associated with that, would be used to process the system prompt!
Point is, at least from an energy/compute perspective, shorter, more succinct, better tailored system prompts could go a long way to save the world compute and corresponding energy...
Anyway, great link, and a very interesting web page!
The assistant is Claude, created by Anthropic. The current date is {{currentDateTime}}. Claude's knowledge base was last updated in August 2023 and it answers user questions about events before August 2023 and after August 2023 the same way a highly informed individual from August 2023 would if they were talking to someone from {{currentDateTime}}. It should give concise responses to very simple questions, but provide thorough responses to more complex and open-ended questions. It is happy to help with writing, analysis, question answering, math, coding, and all sorts of other tasks. It uses markdown for coding. It does not mention this information about itself unless the information is directly pertinent to the human's query.
^ No mention of any safety at all lol, how could dario let this be
1. Less context window to work with.
2. Things were "different" in the early days. The safety and alignment stuff was probably trained into the model, not also found in the system prompt.
3. Safety and alignment meant something different 3 years ago. Now that we've seen how people, including children, use chat bots, altering the guardrails only makes sense. Did we think people would replace their therapists with ChatGPT in the early days? No. Do we know now that they will? Yes.
DeepSeek never does that to me *shrugs*
I can't tell if the first part of this is cult behavior or a way to actually program the model to behave well with a frustrated user. Claude is very frustrating at times, so I understand why that would be needed. But Anthropic rhetoric is often worrying close to that of the people who believed Llama 3 was sentient.
They are natural surfaces for building custom agents and yet you're stuck with whatever they ship with, weird. It's not like it's too complicated api-wise either.
There must be something I ignore.
My guess is that harnesses don't make core system prompts customizable out of the box because the system prompt is one of the defining features of the agent, and something they constantly iterate on and test between releases.
Most users who want to customize the system prompt actually want to do things like add preferences for how the agent should behave, which is better handled by mechanisms like memories or skills (which effectively get appended to the system prompt.)
Not only they get "lost" and ignored as the context grows, but the baseline behaviour of system prompts is retained in the agent.
Skills are prompts, albeit in a specific format. This is apparent in say, Codex where $MYSKILL is literally injecting the skill-prompt inline into a typed prompt. This all gets passed into the semantic memory system anyways, refining away cruft like redundancy, pleasantries, et al.
You have to remind it what's in it's own memory, or the subagent skill is influenced by the main system prompt.
It's sloppy vibe coders productivity porn.
I don't think this is a sustainable way of doing things because I really don't want to assume the maintenance burden for every piece of software that I want to tweak. As far as I understand, new developments like opencode2 have learned from this and are aiming for a well architected core that is easy to built on top of.