Posted by frays 8 hours ago
>and the old game is not coming back
For many people the CTF scene was already dead in 2021 because it had turned into something unrecognisable.
In reality it’s just different.
"That makes open CTFs pay-to-win. The more tokens you can throw at a competition, the faster you can burn down the board. Specialised cybersecurity models like alias1 by Alias Robotics are becoming less relevant compared to general frontier LLMs. The competition is turning into "who can afford to run enough agents, with enough context, for long enough.""
1) It’s OK to do just about anything to win a CTF, including installing malware on the organisers computers months before the actual event so you’ll have an easy time stealing the flags.
2) It’s not ok to try and win the CTF with a solution the authors did not intend.
Recently the #2 crowd has been winning because the hacking scene has turned corporate and boring. People started to partake in CTFs in the hopes of landing a job(!)
CTFs are indeed ruined for those people, I personally don’t mind.
For the people in group #1 LLMs change little. Attacking the challenges directly was always a last resort.
Hits different doesn't it
This year, multiple groups on the top of the leaderboard were clearly abusing LLMs. You can tell because they know nothing of what a CTF is nor the terminology, nor really the fields the challenges were about when they were talked to. They were obviously amateurs.
It was pretty depressing to hear how unaware they were of how obviously they did not fit in to the type that usually is on the top of the leaderboard. It seems they seriously think they were under the radar. If it was one group it could be a freak incident - some times someone just shows up and curbstomps competition. But there were many groups like this this year. They also had a certain smugness to it - one staff reported that a group was hinting to other teams about their "super weapon". Another group credited their "secret third team member they didn't want to talk about".
I use LLM frequently and experiment with it a lot, both at work and on my free time. Nowadays they are good enough to have value and I am interested in learning more about that. They let me spend more time on hard problems and avoid spending the day on simple CRUD. I say this to say that LLM doesnt have to equal bad, it is a tool, that's all. However, I generally avoid LLM communities because many LLM fans are lazy and unskilled people who are just happy they can feel they are worth something even if they have no skill. They don't really have much to provide of conversation. If anything, from reading the CTF crowd this year, the rise of LLMs has just meant more of these people can stomp on and harvest the CTF scene for self validation.
This is not me trying to gatekeep who can play CTF. Anyone is welcome, but there is one condition: You are here to learn and have fun.
The conclusion many I talk to has come to is that nowadays, it is harder to learn to put in hard work and become good at something because there are just too many ways to cheat and take shortcuts. I suspect in the future there will be a shortage of useful people - the kind that have critical thought and know the value of doing something properly. This doesn't mean "Not using LLM", but as said by many on HN before you need a certain seniority before LLMs are useful augmentations to your skills and not just stopping you from learning yourself.
I agree with the article. Anything but physical competitions with strong security - think professional e-sports with organizer-provided PCs, is over. But I think one of the most interesting things to take away from my CTF experience is that the bottom of the leaderboard was still full of amateurs slowly working their way up - it is a few rotten apples that ruin the fun for most, and there are still plenty of people who want to learn and deep-dive.
The text itself being exceedingly long for no obvious reason doesn’t help.
And if you think it was too long, what part would you have shortened? I never knew about the scene and found it interesting to read this personal take on it.
According to Pikka, the paragraph text is Taupe Grey (#92908a) on a Liquorice (#111110) background. That's... pretty far from black and white.