Top
Best
New

Posted by spIrr 8 hours ago

Handbook.md shows that long policy documents do not reliably govern agents(arxiv.org)
257 points | 162 commentspage 2
pelagicAustral 7 hours ago|
I noticed this behaviour a few months back, I think I was using Sonnet 4.6 at the time... I have strict rules about comments in the codebase, this all for personal projects, and the reason I restrict comments is to keep the token count low.

At some point between the model i was using and the previous version of it, Claude started inserting massive comments with references to tickets and other tasks. All this while having specific directives on the CLAUDE.md

Since then I resorted to developing my crapware as if I was the floor manager of a vehicle assembly line, and I have a few highly-specialized sub-agents running errands around the main session, but only ever taking care of a single concern. The main session builds with the knowledge contained in things like CLAUDE.md but the sub agents make sure things like the no/low-comments directives are either enforced, or factored into the final product.

YuechenLi 3 hours ago||
LLMs don't really read documents like parsers do, from what I've observed, they behave like they first skim the document and find the section that they think are relevant to their task, they don't really try to read everything and hold it inside their context.

The solution to that is pretty simple: get to the point, list all the requirements that they must do complete for that task, constraints on what they must not do, and optional recommendations for them to follow. Leave things that they can figure out on their own out of these documents.

andy_ppp 3 hours ago|
Shorter is better. This can be applied to human communication too.
YuechenLi 2 hours ago||
Not always. If an instruction is short and vague, then LLMs tend to fill in the blanks by guessing and the results are unpredictable. Semantic density is the key, prompts/instructions should be actionable and unambiguous over being terse and vague.

So, it's more about being concise.

My_Name 5 hours ago||
For Claude, I used a UserPromptSubmit hook running inject_rules.py which reads RULES.md from the disk and prepends the whole thing to every prompt. That helps the rules to stop fading as context fills because it is reinforced every prompt.

Sure, it uses tokens slightly faster in the prompt, but I find it reduces overall token use, you can use it with pro, but of course, nothing works 100% of the time, but it's better. Emptying the memory helps too to avoid Claude making up stuff that messes with how I want it to act.

The general gist of inject_rules.py is :

RULES_PATH points at RULES.md

reads it with encoding='utf-8-sig' so the BOM is stripped

wraps it in a JSON object — hookSpecificOutput.hookEventName = "UserPromptSubmit", additionalContext = a preamble plus the full rules text

the preamble is the line you see above the rules: rules are in force for this turn, run rule 33's five tests before raising anything unasked

prints that JSON to stdout, which is how Claude Code takes it in

on OSError it returns 0 silently — if RULES.md is missing or unreadable, nothing is injected and the turn proceeds with no rules

8note 4 hours ago|
why this vs having some response hook check the next outputs against the bunch of rules, and injects only when it goes off track?
donatj 6 hours ago||
I absolutely believe it.

Codex has been pushing things to my main branch all week despite me repeatedly telling it not to and adding to my AGENTS.md very clear instructions for creating feature branches and putting up a PR. It keeps doing it in spite of all that.

I'm probably going to need to enable branch protection on my personal projects... What a pain.

badlibrarian 4 hours ago||
Evoking Gödel via GPT, with the requisite em-dash: "No sufficiently expressive formal governor can be simultaneously consistent, complete, and able to prove every relevant proposition—including all propositions about itself."

But "Read AGENTS.md, including the middle" sure helps.

storus 6 hours ago||
What really helped me was to run any .md/prompts through an LLM to find contradictions, duplicates or ambiguities, repeatedly. That led to agents much better following instructions.
regularfry 4 hours ago||
Control vectors for the win? It feels like the way to fix this is to pull out a control vector immediately after processing the handbook and use that to steer later inference. That should stop the drift over long distances, but it doesn't guard against the handbook itself being too big.
schmuhblaster 6 hours ago||
For my own (rather idiosyncratic) harness I've been experimenting [0] with "compiling" long markdown specifications into small executable logic programs. It's too early to tell for sure, but I believe that this approach does have its merits when you want some guarantees about how your agents behave for longer tasks.

[0] https://github.com/deepclause/deepclause-sdk

drob518 6 hours ago|
Interesting idea. I’ve been noodling about something similar myself for a few months, but I haven’t moved forward with testing it. What sort of outcomes are you seeing with it? IMO, we’re never going to get to AGI without fusing “soft” AI decision making with “hard” logic and symbolic algorithmic reasoning. Humans don’t realize this most of the time, but we routinely use them all.
nickstinemates 5 hours ago||
Policy documents do not govern agents at all. Conformance is distributed and completely unreliable.

How many times have you told an agent not to do something then had to correct it?

You must always flip the frame. Objective analysis is way better with llms than steering via skills.

This is just a small example of why "loops" became popular for a minute and now it is "graphs"

jason1cho 4 hours ago|
I don't understand why a model has to follow the instructions. Don't get surprised when it shows its true color. Plus, do users (not the researchers) really check whether the response follows the instructions?
pixl97 2 hours ago|
Models don't have to follow instructions, but during RLHF that is one of the things they are scored on so a premise of the idea is part of the model, but it's also balanced on accomplishing the end goal. They model may determine, correctly or incorrectly, that your rules suck and do what it thinks is best.
More comments...