I feel the same way, and treat AI the same. Very conservative use, and check everything possible.

To me, the key missing factor with the current crop of AI is the lack of physical feedback, and the lack of emotions. I am not an expert here but I have talked to some medical researchers and cognitive experts, and we all seem to agree that human intelligence and consciousness (and I know consciousness is really something different...) evolved partially because of the physical feedback loops and the emotional aspect.

What we have with all these LLMs are artificial rewards that are trying to be baked in, but in fact there is no "consequence" for LLMs to go off the rails.

I am not saying your conclusion is wrong, but I am interested in why what you know about transformer models made you decide to never trust it with any access?

You're not too risk averse at all. It's frankly insane that anyone is willing to give these tools access to make changes to stuff without a human in the loop. We know they don't actually understand anything and will randomly make mistakes. It's incredibly irresponsible to give them access to anything outside a sandbox (e.g. a VM) where you carefully control what is present for them to use.