> The most important skill in prompting is expertise in the domain you’re prompting for.

Amen. AI is a tool, a powerful tool indeed, but if you don't know how to apply it, the quality is seriously impacted.

Overall, I agree, when I ask things I'm an expert in and do professionally every day. I get very good useful answers. When, for example, our marketing people, ask about the science, they often get confusing and wrong answers.

I very much agree, but at the same time I feel like those type of shortcomings are fundamental and will be somehow fixed within the next year. The AI companies would just go bankrupt otherwise...

Sure. Not to speak about the ones who own the LLMs.

They just rewarded with a FREE SUBSCRIPTION one hundred thousands or so professional mathematicians! Undoubtedly, to advance mathematics! Hurrah!

I do agree. An LLM is like a motorboat that's tends to drift off course. If you know where you want to go, and can steer it to keep it on course, you will get there very fast.

Similarly there is research that shows the quality of LLM outputs strongly correlate with the education level (in the field) of the person promoting them.

If LLMs are good at busy work, and it is expertise that lets you distinguish between busy work and valuable one, then it makes sense that they reward expertise.

This is an interesting problem. A similar approach that worked for us was to only load the tools the agent actually needs for each task.

Is someone keeping a list of the excuses and varying instructions on how to hold it right? It would be fascinating historic documentation