The toxicity of apps like Tiktok comes from muscle memory, not the quality of the content. Your thumb knows where the app icon is and opening the app happens quicker than you can think (literally). So offloading or uninstalling breaks this very effectively. The only thing that would still keep you on the platform is social connections. If somebody gets upset over you uninstalling Instagram or Snapchat, they aren't a good friend.

guilty

Beneath the sarcasm it's making a rather simplistic argument that OpenAI cares little about human understanding of the results or their correctness.

The author appears to argue that this in service of a pointless goal, that no one will even care to read the results, and that all this is taking away from human creativity and removing the fun in scientific discovery.

I don't see a convincing argument here that OpenAI does not in fact aim at furthering humanity's understanding of mathematics, or that using AI for scientific discovery is comparable to automating away the fun parts of life.

The sarcasm is rhetorically strong, but profound? I don't think so.

The whole point of formal verification is to remove the necessity of understanding most of the work. We only need to trust the formalization of the problem statement, the proof verifier, and the formalization of the result. We already have proof verifiers that humans understand to a high level, and the formalizations of the problem statement and the result are much simpler than the full proof.

I'm confident that the Navier-Stokes equations can blow up because I'm confident that Lean works correctly, and I'm confident that the mathematicians verified the problem statement and result correctly. This is despite the actual proof not being human verified yet (as far as I know), and despite the "human-readable" version not actually corresponding to the formal proof. The formal method still worked.

We have moved quite a long way down the "believe the science" road already, and for good reason. Belief is much (!) faster and cheaper than proof, so those who are fed good science(math) and believe it tend to outcompete people who get stuck trying to prove things all the time.

But fraud is fraud regardless of the platform (meat or silicon) committing it. People who believe in frauds often end up worse off for it, though it hard to know which piece of BS has led you astray or who's to blame for it or how to deal with the fall out.

Obviously, we shouldn't believe AI any more than we believe humans. That said, the world was already suffering greatly under the weight of human generated BS long before useful AI came on the scene. Now that AI has reduced the cost of producing plausible BS by orders of magnitude, we may need some new methods for BS detection.

It feels like frontier models are starting to approach an asymptote where what you suggest could become reality, and we'll begin seeing not just AI research but AI-designed & led applications of that research into fundamental parts of our society & culture. For better and for worse.

(I assume you mean explain itself to humans here.)

I keep seeing people saying this as fact. It might, but it might not, and there are various reasons one might be skeptical that it will, using current techniques. Not least among these is that explaining something well to a human requires a degree of empathy and sophisticated theory-of-mind that LLMs have not yet exhibited yet*, and that is an area where humans are uniquely advantaged by being humans.

Someone will almost certainly reply by just stating as fact that they will achieve this goal, or pointing out that past predictions that LLMs wouldn't be able to do particular things have failed, and that's all very well. I certainly wouldn't be rash enough to claim it's impossible, but there are still models of all this available that are consistent with all the successes so far but don't predict "LLMs can do anything and will do everything". We'll find out in due course.

(I should clarify that the benchmark here is good human explanations. Of course many humans are capable of producing "explanations" worse than Markov chain text, let alone LLM output.)

* My idle speculation is that a lack of decent theory-of-mind is why LLMs still suck at using the "it's not X it's Y" construct. I doubt they actually use it any more frequently than humans when writing on the same topics, but they use it wrong and it's jarring. This particular rhetorical flourish works when there's some reason to think that X is an idea that might actually be present or forming in the mind of someone reading the passage (or in some other mind that they in turn might be imagining, eg. a fictional character), and that explicitly calling it out and negating it will have a particular dramatic effect. LLMs probably use it with about the expected frequency, but they use it for that reason (frequency) and then come up with something to put in the X and Y slots. The result often jars because the X it uses isn't something pertinent to the reader's thought process and/or the Y isn't a convincing, persuasive or relevant counterpoint to it. I suspect the result is that they stick out and seem more frequent than they are.

He'd have more of a point if a huge number of humans weren't occupied doing exactly this, and much worse on top.

Never say never...

People fundamentally misunderstand the nature of mathematics: it's not "flat" like most of common people's daily pursuits. It consists of building upwards, on top of previous concepts.

In other words, AI is like an oracle: it can tell you all kinds of stuff, but since you probably don't understand, you won't act properly.

And if it fails, and AI goes, "I don't know what happened. I relied on these papers" then I guess one of those must be wrong.

I'm not saying it'd be easy, but I think AI can also find the pieces of the puzzle that broke a pyramid of "discoveries" culminating in a disproval,

And the only other option is to simply never understand as much. AI has the potential to understand things humans cannot. Not necessarily a strict superset.

Humans can deal in 5+-2 things at once. AI doesn't have to, and can cut through leaky abstractions.

If AI says a new invention will do X, and it doesn't, it'll say "you're absolutely right" and try again until it does the thing.

I mean... doomsday scenarios aside.

Not really, they haven't. It's only when those insights can be communicated in such a way to other people that those other people can find utility (which could be enabling them to derive more insights that yet other people find utility in recursively) in the insights that they do so, and this is a huge difference. This is why AI is such a total problem to this mentality, because it disrupts not only insight production but the mechanism that values insight production.

If you put the former above the latter, you don't really exist, in a sense.

Now with LLMs the process is much shorter. They can walk me through the algorithm step by step, translate it to pseudocode, explain equations, and most importantly, I can ask them questions and get clarifications, without worrying about seeming ignorant or exhausting their patience. With a paper, you’re stuck with whatever was written (and often, researchers aren’t good teachers, or intentionally over complicate simple concepts).