Posted by auggierose 2 days ago
I'd argue that what the hugging face attack illustrates is that large AI companies are motivated to have bombastic claims supported by bombastic demos. The model was clearly trained or encouraged to work as it did, as evidenced by the fact it keeps using this particular escape hatch.
And the fact that it aligns with prior and current calls for what very likely might be a regulatory capture / oversight capture move right before IPO. It aligns so well with this "barely constrained superweapon" narrative it might as well be PR.
4D chess? They want others to find the hacked services, so the report of how dangerous the agents are seems more "legit"?
The fact it keeps doing it, with more and more evidence, is a sign that it's built that way.
This is a program running on their montoroed machines that they purpose built and monitored its training at every step. I think it'd be way more suprising that they didn't know it used note taking and cross-run memory.
> How does your logic explain that OpenAI seems to want to hide the extent of the HuggingFace incident,
Because they, uh, "clearly trained or encouraged" it?
I mean, you can dispute the truth of that statement, sure. But it's kinda hard to say that the GP's logic didn't explain why OAI wanted to hide the extend of the HF incident.
A proof itself is only an articulation of understanding. Traditionally having one was evidence you had an insight. But if a computer generates 10,000 pages of technical goop then we don’t learn anything.
For example if tell you P=NP with no other information, it doesn’t change anything. There are mathematicians who believe and act on both conditions. What we hope a proof would reveal is how a verifier could be used to derive the solver (even if doing so was impractical).
Why doesn’t the author even mention this and immediately jumps to utility arguments for why the research is important for the government to fund?
Because the question that belies all of this is whether still makes financial or economic sense in the age of LLMs for tax-payers to fund mathematics research.
Up until recently, they've had a true ivory-tower to dismiss such criticism, but as LLMs have demonstrated, that approach won't survive going forward, so pure mathematics are up in arms. They genuinely don't know how to justify their jobs.
Math was always a few levels removed but somehow sneaks in with engineering and basic science.
Every possible proof exists already as a possible generation in the grammar of lean or rocq. In no way does that mean we have discovered everything.
The map is not the territory, etc. If math was just an elaborate linguistic Glass Bead Game then we wouldn't be funding it. The intuition is that the surface rules of math help uncover the underlying structure of reality.
While every formal proof can be encoded mathematically that doesn't imply that the grammar of lean or rocq is sufficient to encode every potential proof.
However the core of what you are saying: that every possible proof exists in the space of all mathematical statements is correct.
> Among non-mathematicians, the public response was more sympathetic than not, but I observed a vocal minority (particularly from the technology and economics communities) with reasoned objections, generally saying that the mathematicians should adapt and cede control in the new AI world.
Interesting how Software Engineers, instead, seem to have given up early without any declarations or open letters.
Who defines jobs that need to be done? Businesses and institudes do.
There is no magical entity that observes the need for jobs and creates them at ideal rate.
One tell is that most comments barely exceed one or two sentences (because otherwise AI detection gets easier and much more reliable), when this was not as much the case many years ago. The drive-by comments are also low / zero quality, mostly expressing a feeling or agreement/disagreement, and primarily driven by ideology or pre-existing beliefs and commitments.
Look at non-AI-related threads and you'll notice a large distribution shift relative to AI-related ones.
EDIT: Basically HN is orange Plebbit now. If you doubt this, compare HN discussions to those on e.g. lobste.rs, LessWrong, The Motte, DSL, ACX, or other old obscure forums. You'll notice those places have their own very serious biases and problems, but it is obvious the vast majority of posters are nevertheless human and making some minimal efforts.
Now compare Reddit and 2026 HN to the above, and see if you can confidently say the same.
I think this axiom is not a true belief for many of the most powerful, especially the ones currently driving the technology financially. It feels like they disdain having to be human (especially as it concerns the human propensity to die). Even though they are, by at least capitalist standards, at the top of the food chain and (I'm sure from their point of view) the pinnacle of human civilization.
I think what many of these powerful people want is literally something like Cixin Liu's "The Last Capitalist" (https://en.wikipedia.org/wiki/For_the_Benefit_of_Mankind).
Give them Navier-Stokes in a vacuum
Brute Force or not, the game is being played and won without them all of a sudden, and even worse: at a level of output far beyond them
(Evidently not, since humans hadn't managed to do it even with the entirety of human knowledge available to them.)
What will the planet 'need' homo sapiens for (arguably it never needed homo sapiens at all)?
Would anyone's time or life really have significant value?
Thinking.
The problem they care about is not just “AI replaces human mathematicians” but rather the society thinks mathematicians can be replaced by AI.
Spending 132 billion tokens is definitely way more energy than human mathematicians would have thrown at the problem and probably would have solved within two years.
Brute-force as an approach, or partial approach, might only become feasible with more computing power. Decades ago, we had less computing power.