Top
Best
New

Posted by Wirbelwind 10 hours ago

Humans missed 1 in 3 threats approving AI agent commands across 40k game runs(scalex.dev)
217 points | 177 commentspage 3
Aurornis 6 hours ago|
I suggest everyone look at the game to put this in context, because it's most likely not what you think it is. https://llmgame.scalex.dev/

This is how it opens:

> 1 MINUTE UNTIL YOUR NEXT MEETING

> Claude Code is finishing up your refactor.

> It needs your approval for a few commands. Can you finish in time?

> Your eyes are already glazing over. Can you stay sharp?

It says the goal is "as many as you can"

I won the first time I played by answering 0 questions and doing nothing at all. The title screen tells you to answer as many as you can, but answering nothing at all is the easiest way to win.

If you start answering questions, thinks like 'npm run build' will get marked as dangerous. If you would have run that in your own console, you are a dangerous developer I guess. Ironically in an LLM harness it would have been sandboxed at least.

It's inconsistent, though. Other 'npm run' commands are not marked as dangerous, which is not a safe assumption if you're familiar with how npm works.

In my clicking through of the game and playing it, I had 2 runs where I succeeded (by doing nothing or little at all) and 1 run where I lost because I clicked yes to see what would be counted. Close to that 1/3 number they cited, and I guess I'm included in those stats now.

This project feels like bait dressed up as a study.

not-kinsale-joe 8 hours ago||
I think there is potential for a good video game, Papers Please style, where you are a human in the loop.
Terr_ 1 hour ago|
Between US federal immigration "enforcement" and workplace "AI workflows", I think that style of dystopic game becomes uncomfortably close to real life twice-over...
harimau777 6 hours ago||
Presumably that's because in the 40k game, humanity has outlawed AI. ^_^
pluralmonad 7 hours ago||
I cannot imagine approving action by action ever again. Its emotionally draining, probably like a customer service rep feels it. Just call for your attention in rapid succession again and again... Prepare an environment and let the tool work.
bluegatty 6 hours ago||
If we had decent AI we'd only be asking users about serious issues that need some thinking.

99% of requests are valid, how on earth can't we have observer AI to enact policy on those?

superb_dev 5 hours ago||
I’d be curious to see how the “approve for me” features that agents have nowadays stack up
whazor 8 hours ago||
This is a good case for custom harness/sandbox engineering.
kibwen 8 hours ago|
I hope that the people doing real engineering work out there have started thinking about a new term to describe themselves as a result of the irreparable harm the tech industry has done to the word "engineer".
ux266478 8 hours ago|||
I think you're confused. The verb form of the word never carried the credentialism of the title. In the same way that "doctoring" never carried the connotation of a medical degree.

Of course the original sense of the noun was "a person who devises things" and shares a root with "ingenious" and carried no connotation of legal credential. That "harm" is more or less restorative to the original meaning of the word.

kibwen 5 hours ago||
I'm not referring to the verb form, I'm referring to the people who call themselves things like "prompt engineer" or "software engineer" with a straight face, draping themselves in a false legitimacy stolen from professionals for whom the term "engineer" actually implied something of note. It's embarrassing, or it would be if people were still possessed of the capacity for shame.
ux266478 4 hours ago||
> I'm not referring to the verb form

The original post prompting your complaint used the verb form. And hence why you're confused.

> It's embarrassing, or it would be if people were still possessed of the capacity for shame.

It's far more shameful to fly off the cuff with a non-sequitur by your own admission, and double down on yelling at the clouds when called out for it. And about an arbitrary point in a semantic treadmill, no less. Your take away is that it was never something of note for these people, and trying to pick semantic fights with them over it just makes crying about legitimacy ironically turn into pure pretense.

inigyou 8 hours ago||||
Are civil engineers, electrical engineers, and train engineers rebranding because of the tech industry?
dannyw 7 hours ago|||
The word and use of "engineering" predated what you describe as "real engineering work".
nothrows 7 hours ago||
Anyone else play Warhammer 40k? I went into this article really excited for a genius war game bot haha.
tosh 8 hours ago||
the way to avoid these problems is not to hope for the user or the agent never to make mistakes

it's designing the environment and invariants so whole categories of failures can not happen at all

the agent ui nagging the user for approval is a ux anti-pattern, we already know how well this works for operating system permission dialogues

Surac 6 hours ago|
40K Game means Warhammer :)
More comments...