Posted by mooreds 7/6/2025
I think one essential missing ingredient is some degree of attentional sovereignty. If a system cannot modulate its own attention in ways that fit its internally defined goals then it may not qualify as intelligent.
Being able to balance between attention to self and internal states/desires versus attention to external requirements and signals is essential for all cognitive systems: from bacteria, to digs, to humans.
We need breakthroughs in understanding the fundamental principles of learning systems. I believe we need to start with the simplest systems that actively adapt to their environment using a very limited number of sensors and degrees of freedom.
Then scale up from there in sophistication, integration and hierarchy.
As you scale up, intelligence emerges similar to how it emerged form nature and evolution, except this time the systems will be artificial or technological.
Like not necessarily physical things, but mental or digital things.
Humans will create a better LLM, (say GPT-5) than all the other LLMs that currently exist.
If you tasked any current LLM with creating a GPT-5 LLM that is better than itself, can it do it? If not then it's probably not AGI and has some shortcomings making it not general or intelligent enough.
Even the most pessimistic timelines have to account for 20-30x more compute, models trained on 10-100x more coding data, and tools very significantly more optimized for the task within 3 years
The key is to learn how to use them for your use case and to figure out what specific things they are good for. Staying up to date as they improve is probably the most valuable skill for software engineers right now
Just look at the discussion here, you would think the other persons AGI is same as yours, but it most likely isn’t, and it’s comical when you look it from this birds eye view.
He doesn't care if he is right or wrong.
This does not make current AI harmless; it is already very dangerous.
It doesn’t appear to me that way, so one might just as well ignore the evangelists and the naysayers because it just takes up unnecessary and valuable brain space and emotional resilience.
Deal with it if and when it gets here.