A Man Out to Prove How Dumb AI Still Is

Posted by fortran77 21 hours ago

A Man Out to Prove How Dumb AI Still Is(www.theatlantic.com)

62 points | 61 commentspage 2

echelon 21 hours ago|

> When I spoke with him earlier this year, Chollet told me that AI companies have long been “intellectually lazy“

s/intellectually lazy/hype maxing for fundraising/

refulgentis 21 hours ago|

I think it's fascinating that his impossible benchmark got defeated, but because the Keras guy doesn't like LLMs, it is possible to mishear algorithmic distaste as saying people shipping this are "lazy" and "hype maxing."

whiplash451 20 hours ago|||

Francois never said he dislike LLMs. In fact, he said he expected them to be part of the solution to ARC.

I don’t know where this persistent myth comes from, but it has to go.

refulgentis 20 hours ago||

> part of the solution to ARC. I don’t know where this persistent myth comes from,

Part of, explicitly, not the, quite 100% explicitly. The TL;DR is "LLMs can't do it alone, program synthesis leveraging LLMs is my bet". Not "Maybe not LLMs but they'll certainly help us get there!", quite the opposite! Hence: well, TFA. And the intellectually lazy quote we are explicitly discussing. And anything Chollet has said on the subject. [^1]

[^1]"LLMs won’t lead to AGI - $1,000,000 Prize to find true solution" - https://www.dwarkesh.com/p/francois-chollet - 1.5 hours with the gent

artificialprint 20 hours ago|||

Arc agi 1 that "got defeated" was published even before first mainstream llms and still stood the test of time

refulgentis 20 hours ago||

> "got defeated"

1.5 hours with Chollet on "LLMs won’t lead to AGI - $1,000,000 Prize to find true solution" -

Published June 2024, and by December, well...we can all agree there's an ARC AGI 2 now.

https://www.dwarkesh.com/p/francois-chollet

HenryBemis 20 hours ago||

  1a) it's not AI, it's LLM. The companies who create/train/operate them may (wink-wink) pitch them as "AI" with half-truths, but we (here) know it's LLMs "all the way down"
  1b) just like I disliked the "autopilot" in Teslas because it was never autopilot.
  2) I know that I wanted to write some software tools, and I have been successful at this for the past many months, and I got top-shelve tools, that work, do their tasks, send alerts, etc. etc. And I am not the only one. So if the purpose is to "show it's a stupid AI".. well.. it's not AI.. so yeah. If the purpose is "it is not perfect", yes, because it draws a hand with 10 fingers. What else is new?

LLMs are a tool, still under development, still early in the curve, they can do A-B-C well but not X-Y-Z well (or at all). Congratulations :)

dgfitz 19 hours ago|

> LLMs are a tool, still under development, still early in the curve…

I completely agree, they are a tool, and a decently useful tool. They are not early in the curve, they’re about flat at this point.

Yeul 19 hours ago||

Unfortunately I played Deus Ex religiously when I was a kid so I'll never be impressed with AI. The sequel had a self flying helicopter- good luck with that Tesla and BYD.

j_bum 21 hours ago|

> To hit 87 percent on the original ARC-AGI test, o3 spent roughly 14 minutes per puzzle and, by my calculations, may have required hundreds of thousands of dollars in computing and electricity

> the bot came up with more than 1,000 possible answers per grid before selecting a final submission.

Yeah, AGI is right around the corner… /s

pixl97 19 hours ago||

I also remember when a computer about as powerful as the watch on my wrist cost millions of dollars and filled multiple large rooms.

HenryBemis 20 hours ago||

Let's all hope two things: 1) AGI is not near 2) Nobody ever allows it to have physical presence 2b) should that happen, expect the one idiot ask it to make "as many paperclips as possible" :)