Top
Best
New

Posted by salkahfi 4 days ago

How does misalignment scale with model intelligence and task complexity?(alignment.anthropic.com)
241 points | 79 commentspage 2
tbrownaw 4 days ago|
Longer thinking sections have more space for noise to accumulate?
bjt12345 4 days ago||
> This suggests that scaling alone won't eliminate incoherence. As more capable models tackle harder problems, variance-dominated failures persist or worsen.

This is a big deal, but are they only looking at auto-regressive models?

nayroclade 4 days ago||
The models they tested are already way behind the current state-of-the-art. Would be interesting to see if their results hold up when repeated with the latest frontier models.
StilesCrisis 3 days ago|
I think we have all seen the latest models turn into a hot mess.
louiereederson 3 days ago||
i interpret figure 2 as showing that incoherence increases with model gens, albeit on a small sample size
cadamsdotcom 4 days ago||
When humans dream, we are disconnected from the world around us. Without the grounding that comes from being connected to our bodies, anything can happen in a dream.

It is no surprise that models need grounding too, lest their outputs be no more useful than dreams.

It’s us engineers who give arms and legs to models, so they can navigate the world and succeed at their tasks.

sayamqazi 4 days ago|
Also since dreams are built from the combinations of experiences that brain already knows so we cannot die in a dream as our brain does not know how to replicate what it would feel like after being dead. Basically LLMs also cannot produce truly novel ideas.
danny_codes 2 days ago||
Did the definition of alignment change? Historically it meant “AI’s goals are good for humans”. This paper seems to be measuring.. how well AI follows directions?
bazzmt 4 days ago||
"model failures become increasingly dominated by incoherence rather than systematic misalignment."

This should not be surprising.

Systematic misalignment, i.e., bias, is still coherent and rational, if it is to be systematic. This would require that AI reason, but AI does not reason (let alone think), it does not do inference.

eande 4 days ago||
The findings are based on older models and assuming recent models behave similarly, what kind of prompt style one should use then to improve the outcome to avoid the increase in variance especially when you ask a model to solve really complex problems?
makerdiety 3 days ago|
Intelligence is inherently not aligned with humanity, you mean? Why am I not shocked?
More comments...