If you come up with a genetic algorithm scaffolding to affect both the architecture and the training algorithm, and then you instantiate it in an artificial selection environment, and you also give it trillions generations to evolve evolvability just right (as life had for billions of years) then the answer is yes, I'm certain it will and probably much sooner than we did.

Also, I think there is a very high chance that given an existing LLM architecture there exists a set of weights that would manifest a true intelligence immediately upon instantiation (with anterograde amnesia). Finding this set of weights is the problem.

You know that your own consciousness exists, that's where certainty ends. The rest of us might just pretend. :)

So configure your LLM of choice to not.

As capable as they get, I still don't see a lot of uses for these things, myself, still. Sometimes if I'm fundamentally uninspired I'll have a model roll the dice, decide what I do or don't like about where it went to create a sense of momentum, but that's the limit. There's never any of its output in my output, even in spirit unless it managed to go somewhere inspiring, it's just a way to let me warm up my generation and discrimination muscles. "Someone is wrong on the internet"-as-a-service, basically.

Generally, if I come across an opportunity to produce ideas or output, I want to capitalize on it for growing my skills and produce an individual and authentic artistic expression where I want to have very fine control over the output in a way that prompt-tweak-verify simply cannot provide.

I don't value the parts it fills in which weren't intentional on the part of the prompter, just send me your prompt instead. I'd rather have a crude sketch and a description than a high fidelity image that obscures them.

But I'm also the kind of person that never enjoyed manufactured pop music or blockbusters unless there's a high concept or technical novelty in addition to the high budget, generally prefer experimental indie stuff, so maybe there's something I just can't see.

The same person could use a forklift at work, and lift weights manually at the gym.

Just pick the right tool for the job: don't take the forklift into the gym, and don't try to overhead press thousands of pounds that would fracture your spine.

I notice you make no concrete defense of the value of having an AI write an essay for you.

Russ Hanneman's thigh implants are a key example. Appearances are all to some people. Actual growth is meaningless to them.

The problem with AI, is that they waste the time of dedicated, thinking humans which care to improve themselves. If I write a three paragraph email on a technical topic, and some yahoo responds with AI, I'm now responding to gibberish.

The other side may not have read, may not understand, and is just interacting to save time. Now my generous nature, which is to help others and interact positively, is being wasted to reply to someone who seems to have put thought and care into a response, but instead was just copying and pasting what something else output.

We have issues with crackers on the net. We have social media. We have political interference. Now we have humans pretending to interact, rendering online interactions even more silly and harmful.

If this trend continues, we'll move back to live interaction just to reduce this time waste.

If the motivation structure is there I don’t see an inherent reason for people to refuse cultivating themselves. Going with the gym analogy lay people did not need gyms when physical work was the norm, cultivation was readily accomplished.

If anything there is a competing motivational structure in which people are incentivized not to think but to consume, react, emote etc. Information processing skills of the individual being deliberately eroded/hijacked/bypassed is not a AI thing. The most obvious example is ads. Thinkers are simply not good for business.

LLMs aren't world models, they are language models. It will be interesting to see which of the LLM implementation techniques will be useful in building world models, but that's not what we have now.

When you see a dog, or describe the entity, do you discuss the genetic makeup or the bone structure?

No, you describe the bark.

The end result is what counts. Training or not, it's just spewing predictive, relational text.

Sorry for the late response. Yes that is Hinton's argument, and the claim made by the believers. On the other hand, if the GAC explanation is correct, an explanation might be that what we humans write down (that is, the training corpus) is a model of the world, and LLMs reconstruct (descriptions of) human understanding.

They're a model of language, not of the world.