I am considering an experiment where the task is to try to win by adversarially targeting Pluto (a high profile self play based RL model)

But right now the response time would be so slow that the bot would not be close to competitive except with the most low level players. Especially because its hard to click on things that are moving at the same time as well.

Sometime soon, I'd like to see an LLM be competitive using only computer vision, mouse movement, and locked into a human possible number of actions per minute.

TLDR they suck and many default to doing probe rushes.

You can train a specialized model for this, but a frontier model is ill-fitting.

Making actually good decisions is the bottleneck.

On a more serious note: all three races play quite differently so it makes sense to focus on just one. And protoss is mechanically the easiest and most straightforward race: fewer but stronger units, a single worker can be used to build multiple buildings at once, versatile basic static defence

Protoss vs Protoss only also helps downscope the problem given the time constraints