Knowing declarative you need to loop over elements and actually being able to write the for loop as procedural knowledge are two different shoes. I believe that this is the real danger.

Pilots have much automation in the cockpit but the pilot needs simulation hours actually flying not relying on the autopilot.

If you dont write code you will forget/loose much accuracy writing it, its just a matter of time.

I think the analogy to hyper-palatable, calorific food works well:

Humans adapted for a world with too little food. Then once there was too much, obesity and overeating became a problem for the first time. Self discipline is the cost we have to pay for this kind of abundance.

We now have a general-purpose way to offload mental effort, and are discovering in real-time the negative consequences of that.

I use AI for coding, but I feel I've moved past the honey-moon period and am now learning how to use it in a way that is not a detriment. If I care about the work I'm doing I don't want the AI to do it for me, even if it could. Deciding what work I want/should be doing myself vs what can be delegated is a new skill I, and I believe we all need to learn.

The reality is that most humans do very little actual thinking of their own anyway, and, if you believe that what LLMs produce constitutes a form of intelligence, it does seem "more intelligent" than most humans.

So: is outsourcing thinking a net improvement for a majority of users?

I use several models, daily, and they seem "reasonably conditioned" that they are only input to my thinking and not "my thinking". I correct them constantly; they are wrong (in reasoning/logic, in actual facts) frequently. They are demonstrably "not smarter" than I am. And yet I know many people who can "do more" with them as a "thinking" tool. I can say that "the problem" is they can't spot the errors, but they can't or won't do that in their ordinary lives, either, so, again, is it a net improvement for them?

Interesting times and all that.

It goes like: "Here is this thing I wonder about", and the LLM is like "Yeah sure, consider these things that are super related to what you are doing, that you probably know nothing about yet (but you know... if you are interested...)".

And that goes in any direction, for any depth. Anything that is made trivial now, is just replaced by something more consequential a level or two higher. You can just get much better at things that matter more.

I think partially it is because of the amount of data you can get out of an LLM and because it looks pretty good, a lot of people treat it as authoritative.

This means they send it around and then other people have to go through the work of actually validating it before being able to act on it.

So what this really means is that the person you go the information from offloaded their thinking to the AI, while cannibalizing on yours.

And just like people going to the gym to exercise their otherwise economically useless bodies, same thing will happen with the mind.

Folks, get over it.