Top
Best
New

Posted by pella 20 hours ago

Over-editing refers to a model modifying code beyond what is necessary(nrehiew.github.io)
384 points | 221 commentspage 2
Almured 20 hours ago|
I feel ambivalent about it. In most cases, I fully agree with the overdoing assessment and then having to spend 30min correcting and fixing. But I also agree with the fact sometimes the system is missing out on more comprehensive changes (context limitations I suppose)! I am starting to be very strict when coding with these tool but still not quite getting the level of control I would like to see
pilgrim0 20 hours ago||
Like others mentioned, letting the agent touch the code makes learning difficult and induces anxiety. By introducing doubt it actually increases the burden of revision, negating the fast apparent progress. The way I found around this is to use LLMs for designing and auditing, not programming per se. Even more so because it’s terrible at keeping the coding style. Call it skill issue, but I’m happier treating it as a lousy assistant rather than as a dependable peer.
kgeist 17 hours ago||
Interesting, my assumption used to be that models over-edit when they're run with optimizations in attention blocks (quantization, Gated DeltaNet, sliding window etc.). I.e. they can't always reconstruct the original code precisely and may end up re-inventing some bits. Can't it be one of the reasons too?
dbvn 19 hours ago||
Don't forget the non-stop unnecessary comments
Flavius 19 hours ago|
Token bonanza! Inference sellers love this simple trick.
kgeist 17 hours ago||
Custom constrained decoding could have solved this. Penalize comment tokens :)
jacek-123 10 hours ago||
Feels like a training-data artifact. SFT and preference data are full of "here's a cleaner version of your file", not "here's the minimum 3-line diff". The model learned bigger, more polished outputs win. Prompting around it helps a bit but you're fighting the prior.
rcvassallo83 13 hours ago||
This resonates

I've had success with greenfield code followed by frustration when asking for changes to that code due to over editing

And prompting for "minimal changes" does keep the edits down. In addition to this instruction, adding specifics about how to make the change and what not to do tends to get results I'm looking for.

"add one function that does X, add one property to the data structure, otherwise leave it as is, don't add any new validation"

ozozozd 11 hours ago||
There is no need for a new name. It’s called a high-impact change. As opposed to a low-impact change, where one changes or adds the least number of lines necessary to achieve the goal.

Not surprised to see this, since once again, because some of us didn’t like history as a subject, lines of code is a performance measure, like a pissing contest.

brianwmunz 18 hours ago||
I feel like a core of this is that agents aren't exactly a replacement for a junior developer like some people say. A junior dev has its own biases, predispositions, history and understanding of the internal and external aspects of a product and company. An AI agent wants to do what you ask in the best way possible which is...not always what a dev wants :) The fix the article talks about is simple but shows that these models have no inherent sense of project scope or proportionality. You have to give context (as much context as possible) explicitly to fill in the gaps so it infers less and makes smaller decisions.
btbuildem 18 hours ago|
I wish there was a reliable way to choke the agents back and prevent them from doing this. Every line of code added is a potential bug, and they overzealously spew pages and pages of code. I've routinely gone through my (hobby) projects and (yes, still with the aid of an LLM) trimmed some 80% of the generated code with barely any loss of functionality.

The cynic in me thinks it's done on purpose to burn more tokens. The pragmatist however just wants full control over the harness and system prompts. I'm sure this could be done away with if we had access to all the knobs and levers.

qurren 18 hours ago|
> if we had access to all the knobs and levers.

We do, just tell it what you want in your AGENTS.md file.

Agents also often respond well to user frustration signs, like threatening to not continue your subscription.

gobdovan 14 hours ago||
> Agents also often respond well to user frustration signs, like threatening to not continue your subscription.

From the phrasing, I can't but imagine you as a very calm, completely unemotional person that only emulates user frustration signs, strategically threatening AI that you'll close your subscription when it nukes your code.

More comments...