Transformers can actually learn pretty difficult manipulations, even how to calculate difficult integrals, so I don't agree that they can only solve problems they've learned by heart.

The reason I believe it can be achieved in this time frame is that I believe that you can do much more with non-output tokens than is currently being done.

I suppose not.

Things I think will be hard for LLMs to do, which some humans can: you get handed 500 pages of Geheimschreiber encrypted telegraph traffic and infinite paper, and you have to figure out how the cryptosystem works and how to decrypt the traffic. I don't think that can happen. I think it requires a highly developed pattern recognition ability together with an ability to not get lost, which LLM-type things will probably continue to for a long time.

But if they could maths more fully, then pretty much all carefully defined tasks would be in reach if they weren't too long.

With regard to what Touche brings up in the other response to your comment, I think that it might be possible to get them to read up on things though-- go through something, invent problems, try to solve those. I think this is something that could be done today with today's models with no real special innovation, but which just hasn't been made into a service yet. But this of course doesn't address that criticism, since it assumes the availability of data.

Yes, general means you can present it a new problem that there is no data on, and it can become a expert o that problem.

synthesizing new ideas: in order to express the idea in our language it basically means you have some new combinations of existing building blocks, just sometimes the building blocks are low level enough and the combination is esoteric enough. It's a spectrum again. I think current models are in fact quite capable of combining existing ideas and building blocks in new ways (this is how human innovation also happens). Most of my evidence comes from asking newer models o3/gemini-2.5-pro for research-level mathematics questions which do not appear in existing literature but is of course connected with them.

so these arguments by fundamental distinctions I believe all cannot work--the question is how new are the AI contributions. Nowadays there's of course still no theoretical breakthroughs in mathematics from AI (though biology could be close!). Also I think the AIs have understanding--but tbf the only thing we can test is through testing on tricky questions which I think support my side. Though of course some of these questions have interpretations which are not testable--so I don't want to argue about those.

I bet you median humans make up a nontrivial amount of things. Humans misremember all the time. If you ask for only quotes, LLMs can also do this without problems (I use o3 for search over google)

maybe you havent been exposed to actual median humans much.

I agree with second part but not the first (far in capability not in timeline). I think you underestimate the distance of median wihout training and "hello world" in many economically meaningful jobs.

While we don’t know an enormous amount about the brain, we do know a pretty good bit about individual neurons, and I think it’s a good guess, given current science, to say that a solidly accurate simulation of a large number of neurons would lead to a kind of intelligence loosely analogous to that found in animals. I’d completely understand if you disagree, but I consider it a good guess.

If that’s the case, then the gulf between current techniques and what’s needed seems knowable. A means of approximating continuous time between neuron firing, time-series recognition in inputs, learning behavior on inputs prior to actual neuron firing (akin to behavior of dendrites), etc. are all missing functionalities in current techniques. Some or all of these missing parts of biological neuron behavior might be needed to approximate animal intelligence, but I think it’s a good guess that these are the parts that are missing.

AI currently has enormous amounts of money being dumped into it on techniques that are lacking for what we want to achieve with it. As they falter more and more, there will be an enormous financial interest in creating new, more effective techniques, and the most obvious place to look for inspiration will be biology. That’s why I think it’s likely to happen in the next few decades; the hardware should be there in terms of raw compute, there’s an obvious place to look for new ideas, and there’s a ton of financial interest in it.

A quick glance at your other comments shows that your account seems to be purpose-built to come up with the most inflammatory response every single time, you might very well just be a chatgpt prompt.

I never suggested they were unambiguously bad, I meant to propose that it is a valid concern to talk about.

In addition, with your argument, should you not legalize all drugs in the quest for maximising profits to a select few shareholders?

AFAIK, the workings of addiction is not fully known, I.e. it’s not only those with dopaminergetic dispositions that get ”caught”. Upbringing, socioeconomic factors and mental health are also variables. Reducing it down to genes I fear is reductionist.

Counterpoint: eugenics are bad.

You are saying suffering is allowable/good because eventually different people won't be able to suffer that way. That is an unethical position to hold.

This sounds very close to the “It’s ok to abuse and kill animals (for meat), they’re not sentient”

[flagged]