Top
Best
New

Posted by benswerd 6 hours ago

Brood War Bench(bw.swerdlow.dev)
67 points | 40 comments
pelagicAustral 1 hour ago|
Unrelated to the benchmark...

I love StarCraft. I started playing it right from the beginning, most of my friends right now are from that era. I literally met people that have spread to almost every continent when I was in my early teens. We played at internet cafes and did not have access to the internet, that was priced differently...

I miss those days so much.

Everybody was from a different background back then, and nobody was anything other than a guy that plays StaCraft at the cybercafe... And now, we are in our 40's and I know Math teachers, history teachers, oil rig operators, software programmers, professional gamers, lawyers and more... hahah So crazy to think about it... and I know them, we talk, what a world.

snicky 18 minutes ago||
Oh boy, I've spent more hours playing it than I dare to admit. I won over 10000 battle.net games ... on just one of my several accounts :) When the SC2 beta came out, I played about 50-100 games and never bought the full game, because I knew it would be like heroin to me, and I was already an adult that had to take care of himself.
nemo1618 44 minutes ago|||
Even at Burning Man, in the middle of the desert, there is a camp that hosts a StarCraft tournament every year (on the dustiest setups you've ever seen!) :)
PorciiVorbesc 27 minutes ago|||
So ... Zerging Man?
benswerd 44 minutes ago|||
So dope
benswerd 44 minutes ago|||
I agree.

My first time playing StarCraft was at summer camp around a decade after it came out.

All the smartest people played it so I wanted to too. Great decision, I have been continually impressed with the people who StarCraft introduced me to.

gary17the 32 minutes ago|||
My best greetings to all Starcraft Elders clanners of yesteryear :).
sidewndr46 55 minutes ago||
Later we even listened to Eminem and played violent video games. Most of have never even been charged with a crime, much less abused someone.
AntiRush 18 minutes ago||
Back in 2010, during the early days of bwapi, there was a Brood War AI tournament held by the Expressive Intelligence Studio at UC Santa Cruz. It's interesting to see how different the approaches were back then, vs this or Deepmind's SC2 work.

https://web.archive.org/web/20091124210529/http://eis.ucsc.e...

There's a great contemporary Ars Technica piece by a competitor:

https://arstechnica.com/gaming/2011/01/skynet-meets-the-swar...

As an undergrad I did a project using genetic programming. It was not very successful, but it was a lot of fun.

https://tomisin.space/archive/starcraft-genetic-programming/

GodelNumbering 54 minutes ago||
A friend of mine created GoBench[1][2] that evaluates LLMs on 9×9 Go using KataGo opponents as Elo anchors, you see real capability differences there, like Astra Max substantially leading all other models. I think strategy is a generally interesting area to evaluate LLMs on

[1] https://rolandgao.com/blog/gobench/

[2] https://rolandgao.com/gobench.pdf

chaostheory 40 seconds ago||
[delayed]
faeyanpiraat 49 minutes ago||
There is currently a bot beating everyone on the ladder. Just watched it today on Artosiscasts yt channel.
ericpruitt 3 minutes ago|
Link to the video for the curious: https://www.youtube.com/watch?v=1vsTqNwHquE . I would note the bot is not APM limited, and comments suggest (https://www.youtube.com/watch?v=1vsTqNwHquE&lc=UgxBrY4CkuZx7... , https://www.youtube.com/watch?v=1vsTqNwHquE&lc=UgyZ0XVFDqzJT...) it's likely using map hacks, both of which make the bot significantly less impressive.
minimal_action 20 minutes ago||
I think we're on the early days of games you connect with your agent to. Human + AI units one versus the other. Like knights with their horses. Not sure which is the horse..
WillMorr 6 minutes ago|
I've been running these with friends recently, it's very fun. I ran irl bot tournaments for board games a couple times but the agent era opens up a huge amount of possibilities.

Must recently I built out a MMORPG puzzle box thing, I wrote a general game architecture doc but left the specific puzzle design up to Fable. Nobody is actively playing rn but I left it up at bot.willmorrison.net.

c7b 6 minutes ago||
At last something that feels properly orthogonal to pelicans on bicycles.
tweakimp 28 minutes ago||
If you want to see human written bots in action or compete in the bot ladder yourself, try https://aiarena.net/
gadtfly 53 minutes ago||
Did it play by looking at screenshots and sending clicks, or was there other mediation/symbolization?

It sounds like it might have been actually played in real time, which would be very important to distinguish.

I have recently seen other harnesses letting agents play real-time games in what seems like discrete time slices, turning eg Portal into something turn-based https://www.youtube.com/watch?v=ruuGXFAmiOE

loeg 25 minutes ago|
There's a bot data stream already; it's probably hooked up to that rather than screencap. Yes, I believe these were playing in real time.
Game_Ender 1 hour ago|
Any details about the harness the agents were given? I am curious what representation of the screen and world state was provided to the agents and what tools they had available.
benswerd 56 minutes ago|
Oh sorry I should be more clear on that. Will add to report.

For agent harness I did Claude Code, Codex, Grok Build. This was primarily a cost driven decision — I have a lot of free tokens and I didn't want to pay API prices for this.

For game harness I used minimal BW-API issue command and get observation apis as tools. I felt this was the most fair way to do it on my small scale.

In the future I would like to integrate code mode and multiple games/I think if it was a best of 5 where each agent could learn from its past games and build its own automations over time that would be much more interesting.

More comments...