Posted by ejohnso49 6 hours ago
(links to more at https://news.ycombinator.com/item?id=50184935)
While I have been chasing around today, trying to resolve this on different servers, it turns out a three line nginx mitigation (blocking multipart uploads) I wrote in the wee small hours GMT has been quietly protecting a vulnerable machine I could not shut down (from attacks powered by every single mainstream agentic AI tool — Anthropic, OpenAI, Qwen cloud, Baidu, seemingly also AWS, Meta)
Including an hour before the PHP patch actually shipped.
So without the HN discussion I wouldn't have found enough people pointing at possible mitigations and I would have lost a server to this, and I thank everyone who pointed out that second mitigation in particular.
I absolutely loved being here today.
So many brilliant minds screaming and hating together, trying to survive the tide.
There aren’t many places like this left on the commercial internet, especially in this brave new AI-driven world.
I felt a great disturbance in the Force. But in a good way.
Together, we are strong. May the Force be with us all!
Is it so bad? Have you measured the number of attacks?
> So without the HN discussion I wouldn't have found enough people pointing at possible mitigations and I would have lost a server to this
I run a tiny server. I like this server a lot. I only patched an hour after the Debian package was ready and didn't think about it again. Do you mean to imply there's a real chance my little server has been compromised?
[Edit:] My server is quite public too. It hosts four SLDs and many subdomains and receives both human and bot traffic. Only one or two subdomains use PHP, but a reasonable attacker surely tries all of them.
It is not a torrent, no. Dozens up to hundreds, rather than hundreds to thousands.
But they would very likely have succeeded, right? It's a fairly turnkey vulnerability.
> I run a tiny server. I like this server a lot. I only patched an hour after the Debian package was ready and didn't think about it again. Do you mean to imply there's a real chance my little server has been compromised?
Can't say one way or another. Have a look at your log file for POST requests in quick succession for things like
/_next
/_next/server
/app
/actions
/api
/dashboard
All my "obviously PHP app" domains got visited. None of my bespoke app domains (which are not obviously PHP). Though over time, the bespoke domains do get a whole bunch of boring probes for WP vulnerabilities etc., like all websites get.
Surely it doesn't take a very sophisticated attacker to clean up after themselves at least a little bit?
If your attacker is someone known to you, some kind of persistent threat, then maybe they would? But most of this stuff is typically kind of worm-like; attack one box, use it to attack another, move on.
With this particular server I am in two minds about whether it is likely I have any persistent type hackers, but I must say what it looks like is automated attacks. AI script kiddies, almost. These same boxes attract all the typical PHP app probes (WP2shell etc.)
And FWIW what it actually mostly looked like was hits from some sort of cloud-based AI search API; they suddenly stopped at the same time as if someone else had pulled a plug. So maybe someone running an agentic search API decided not to be part of the probes.
It's a bit more a "certain C programming techniques are bad" problem.
This is more like log4j, heartbleed etc.
Won't be the last of these.
This is PHP world; it may be a share nothing architecture, but it's full of long-running VMs and very long-running apps.
You lost its privacy
Just saying "hey blocking multipart uploads can mitigate most of attacks early would save people so much trouble. Of course not every CVE will have workaround but many do and whole secrecy about releasing it is only to benefit of few big tech companies and nobody else
I finished moving the remainder of my stuff off of PHP and WordPress last year and haven't missed it at all. Backwards compatibility was typically pretty bad.
It seemed like maintaining existing working PHP code was always a cycle of: upgrade to newer PHP, that version has breaking changes, fix that code or update to a new WordPress (plus all of the plugins that would inevitably need updating too).
But this isn't about PHP code, really — this is about C code (with less-than-ideal exception handler patterns). This problem is more Heartbleed than WP vulnerability.
I quite like modern PHP, I don't really think it's necessarily any less secure (exception handler weirdness notwithstanding), and more to the point, I don't think not using it is any kind of automatic improvement in web app security. It's not like server-side JavaScript is intrinsically better.
I am certainly doing some thinking today; I switched off a couple of web servers so I didn't have to think about them while I was working on other stuff, and actually I think they might not get switched back on. But it's not so much that they are running PHP as that they were kind of neglected and it's always worth reconsidering what you need to keep around.
I think I have reached a point where low value stuff is not worth subsidising just for fun.
Unless your position that any statement whatsoever made in the second person that isn't a compliment is then an attack?
San Francisco is a magical place where "you are doing X wrong" without slathering on six paragraphs of fake kindness constitutes some kind of personal attack
You can make your substantive points without that, both in San Francisco and elsewhere.
He is, in fact, not mitigating the attack, and he is, in fact, by calling his non-workaround a mitigation, encouraging people to do ineffective things instead of patching. These facts remain true in the real world regardless of what you call a personal attack.
And yes, San Francisco has become top of the world for indirect fake kindness that transforms plain factual statements about the world into social offenses.
"The sun is bright. Don't look at it " -- RUDE!
"You might want to consider that the sun might be considered bright to some people and that you might be more comfortable looking at this other cool thing instead. But no pressure" -- gruff, barely tolerable
1) you're being rude, just stop it and get a grip
2) actually, this particular mitigation is very clearly stopping all the unusual patterns of POST request attempts I can see (it's not usually a busy server)
3) I know it is not the only mitigation that is necessary
4) … so it isn't the only mitigation I have applied
5) I am not working under a false sense of security, and I know it is temporary
6) I am not an idiot, and I am working on patching this machine, but this particular server has a unique and complicated configuration and needs a bespoke build
7) I have been prioritising much more serious risks for other people, all day, for no pay
8) again, to reiterate, you're rude.
Also I have survived on too much coffee and probably too many chocolate biscuits. So actually, sorry, on reflection :-) What a day.
I appreciate how your excessive-biscuit status has yet to be determined :)
For example, when I buy something from a webshop, why does that webshop need my personal information? Why can't I give it a token, which it can then use to send my package, because only the postal system knows the address coupled to the token, etc.
And we should also ban user tracking for advertising purposes, obviously.
We'll probably never convince end-users to "vote with their feet" and use Privacy-focused products. But we can absolutely, through legislation or just precedent, raise the Risk profile until it's better to leave the ad revenue on the table and run a private service.
It would completely destroy the data brokers and targeted ad industry but the world would be a better place as a result. Also not going to happen but I can dream.
This is the most surprising part to me. It sounds like Calif saw the leaks, interpreted that as embargo being broken, and then raced to publish their [anime dancing] marketing video? I'd have expected much better.
This reads like some guy tripped and fell on top of his mouse and in so doing, accidentally turned on screen recording, performed the exploit a few times, got the anime girl asset and soundtrack, dropped all of them into Premiere, added the weird effects, rendered the video, and published the video to Twitter. "Ow. What did I fall on? Oh noooo, I broke the embargo!"
Heck of a mistake.
some people are just lacking coordination
The video was posted 44 seconds before that time: https://x.com/lcfr_hax/status/2112906567662399830
This can't possibly be interpreted as some major violation of the embargo, so what are people talking about?
PHP 8.2.35 - https://news.ycombinator.com/item?id=50180873 - Oct 2026 (439 comments)
PHP2Shell: AI uncovers critical RCE in PHP - https://news.ycombinator.com/item?id=50180836 - Oct 2026 (146 comments)
PHP session.upload_progress heap UAF (CVE-2026-109618 / -109619) - https://news.ycombinator.com/item?id=50176077 - Oct 2026 (61 comments)
Security advisory: Critical remote code execution vulnerabilities, update now - https://news.ycombinator.com/item?id=50172041 - Oct 2026 (631 comments)
Tell HN: 2x PHP RCE CVEs with CVSS 10.0 Incoming - https://news.ycombinator.com/item?id=50167754 - Oct 2026 (60 comments)
> Chainguard unilaterally broke the embargo and published built images ...
Won't be the last time this happens. If there is no consequence of breaking the rules, people will keep doing that.
The point of the embargo was to prevent all rumors? But it did not hold.
Anyway, GPs point is that even if nobody leaked anything, and the only thing publicly known was the existence of the bugs, people would anyway have been able to find them independently. So what is the purpose of waiting 24h before releasing the patches?
Finding an exploit and publishing is is not against the law. In some cases, publishing immediately is the responsible thing to do. For example, when it's actively being exploited in the wild, and waiting just puts people at risk that don't have any recourse to protect themselves, since they don't know they're at risk. There can be nuance to that as well, but in general, as soon as it's being exploited, anything that makes it harder for end users to a) know there is an exploit, or b) tell if they're susceptible to it (e.g. through a POC) is just taking away people's ability to take the action they deem appropriate.
If a video is released showcasing the exploit and following mitigation recommendation, PHP should clarify what it is and if the mitigations are recommended or not. From the pre-announcement: > We reserve the right to release earlier should details of either vulnerability become public, or should evidence of active exploitation appear in the wild.
If no info is released by them, no other party should have released anything. If info is released or leaked from partners or involved parties, you should notify and explain what's going on.
But no, complete silence from PHP news or any PHP community member.
I get that it was a complicated situation, and I think PHP is the "less wrong" party involved, but I think this constructive and external criticism could help in the future. It has been an interesting day (GMT+2 here) and a rollercoaster of monitoring and mitigating without any reliable source.
Not entirely correct as they clearly announced that they were going to release a fix at a specific date and time; the mitigation doing the rounds was not adequate and if the PHP people republished it, it would lull people into a false sense of security. Overcommunication is also bad, especially if the official platform or the individual members are being watched by tens of thousands of people hungry for news or updates.
They knew (since august, arguably since 2024), they had a fix ready, everything was in place. Except for the untimely / accidental early leaks.
But yes, the preannounce was the right move and the leaks were unfortunate, complicating things. But if cloudflare, bunny and other actors start mitigating in their WAF, I think something has gone wrong and, either you act by releasing earlier or calm the waters saying "the exploits are wrong" as per stated in their post, they thought it was the case.
Everyone in such a ragtag team would have an overlapping day job where the same crisis management needs to happen.
I have a lot of sympathy for php team - it would be a lot of work to respond to every conspiracy theory on twitter, while trying to get everything ready, in the middle of the night no less.
A lot of speculated info was right: the mitigations were mostly right, the video showing exploitation was real and the release of a fixed image was real.
I think if you do a pre-announce, and you say that you could release it earlier or release new information and you don't, and there are reasons to inform your users, then even if it's the middle of the night for you, you should.
As they stated, they are a small team and they have changed their structure, and they are learning and improving. I think if something like this happens, info is vital for us, the administrators of PHP servers
> PHP security team, operating on consensus, struggled to choose
What an unfortunate waste of a security audit. (and consensus is an obvious flaw in decision making)
> PHP runs a large share of the web... Yet security work on PHP itself still depends on a small group ...
Another unfortunate big structural issue that's unlikely to change.
Are there structurally much safer ecosystems out there?
Several attributes make a safer ecosystem. An important one is diversity - technical diversity.
For an interpreted language, the runtime is important. Python has several runtimes, as does Ruby. Javascript also has several, from the few browsers, to node, deno and bun. PHP has for all practical purposes, only one. So if there's a security incident in PHP, it affects almost all users. But when e.g. JRuby has a vulnerability, it will affect only a fraction of the installed ruby applications.
For compiled languages, the tooling, libraries and runtimes is where diversity is important. With e.g. rust, the vast majority of CVEs come from libraries (crates). But since there often are several crates for a single concept, diversity helps mitigate here too.
I know you added a caveat and all, but no, PHP has a number of alternative runtimes just like the other languages you mentioned: https://github.com/gakowalski/alternative-interpreters#alter...
But again with all the ones you mentioned, they have a "core" runtime that the vast majority of production apps use, so if NodeJS has a vulnerability it would be just as big.
The interesting one will be whether "boring" ports - like recently Typescript's TSC from Typescript to Go and now to Rust - will be safer for being the same code and logic, converted to another "safer" language (by transpilers or LLMs).
When a corporation sponsors the language, such as Java, Go, .Net, there's an inherent economic component which acts as a forcing function.
PHP being a foundation sounds nice. But consensus wasn't strong enough to affect the key outcomes.
making php is the core of (checks notes) php foundation
or is the management full of completely not technical people who use it to boost their CVs?
(Of course there's Log4shell, but that could've happened to a Rust-based stack too, and might be more likely to, considering the proliferation of deeply-nested dependency trees full of tiny libraries in that ecosystem.)
Also, blameless post-mortems exist for a reason.
This is incredibly damming of Chainguard. I struggle to see justification for this, even with the chaos approaching the coordinated release.
In their place I would worry that this was the last time anyone shared an embargoed build with them.
Very irresponsible behaviour in my opinion; and if it's a decision made unilaterally by their CEO, probably a breach of their SOC 2 certification. What's next? "$X isn't patching this CVE fast enough, here's a binary blob of our fix, don't worry, we're making this free!"
What was Wiz/OpenAI supposed to do? PHP was already aware of the issue.
They released some binaries, built for their own docker image based on their own Linux distro. Other people couldn't use Chainguard's "fix" to patch their own systems. The only people who seem to have benefited from the stunt are the hackers who used AI to reverse-engineer the binaries and write exploits.
That was a giant fuck-you to the rest of the PHP ecosystem, nothing more.
[citation needed]
By which i mean, this seems untrue. If you mean the anime video, that is not a leak of a PoC.
That can be for marketing purposes or by mistake. Not sure how to deal with that.
Dont trust us with information lol.
https://www.chainguard.dev/athena
https://www.chainguard.dev/unchained/this-shit-is-hard-build...
Song is Petite Meller - High Forever (Marlon Hoffstadt Remix). Which is annoyingly catchy
thanks.
This industry is full of immature children who cannot keep their mouths shut for 24 hours. They have no clue what massive coordination this requires.
They're apparently not trustworthy, and behave as children.
>As a result of the leak and pre-announcement around 20:00 UTC, Calif, one of the security firms that had reported the issues mistakenly published a video. It showed their proof of concept working against a number of targets and revealed a few additional, though limited, details about the vulnerabilities.
You can see the same demo video on their site: https://calif.io/php2shell/#php-5-4-0-rce
Calif more than Chainguard, but still Chainguard too.
Calif was obviously twitching to claim credit for their findings, and what I find troubling is this eagerness was more important to them than giving users a chance to update their systems.
Even if they ended up waiting only a day post release I would still say it seems a bit too soon. Maybe a week after. Not everyone’s system have aggressive patching schedules and PHP is soo heavily used.
It's maybe not professional to release an omg panic video, but not even remotely the same thing as what chainguard did.
I dont know why the timing of the bug is surprising to you.
They took it down but someone (Kirill Firsov: https://x.com/k_firsov/status/2112641845926989944) added the anime stuff and reposted it. Totally irresponsible.
Edit: okay he didn't add the anime, but he reposted it.
I want to go back to talks by bored defcon guys or gals calmly stating how they obtained private keys of a globally trusted CA with just a bit of math. Which, mind you, ate the other part of the teams time.
Now I know how much I hustled today likely due to that atrocity of a video. I think I will take a vacation day and sit in a forest near a river on friday.
Embargos make sense if no one is exploiting it yet, but if its out in the wild, the best path forward is releaseing as soon as possible.
"Cloudflare successfully deployed multiple mitigations for all its customers while the vulnerability was leaked before the official release date and we observed attack traffic targeting our customers."
If that's not an in-the-wild exploit, it literally couldn't have been further away than one or two more iterations.
From Cloudflare’s perspective, there is no downside to saying what they observed, and deploying defenses.
We may never know if this was actually being successfully exploited prior to patch availability.
Was there even an incident co-ordinator here? Was any of the parties talking to each other?
But it's also possible, that some attackers got non-public inside information.
If the embargo is already broken in a meaningful way, it suddenly becomes much less clear if it is still in place after all. Emphasis on >less clear<.
so, if they saw evidence of exploits in the wild, it is not unusual at all to release the fix.
There is a MASSIVE difference between a small number of people exploiting in the wild, and a few hours after a release a bazillion script kiddies doing so. The move to release and break an embargo was childish, absurd, self-serving, and lacking in all common sense. Not to mention, breaking trust.
I agree with those in this thread who say, never going to do business with them. Ever.
I highly doubt their paying customers feel the same way
I would let Chainguard deal with the patches post official release like all other unreliable parties.
As a customer, I'd immediately assume no one in the security community would trust them again, and seek someone else to give my money to. Why pay a) an untrustworthy company who b) now is not trusted by anyone and c) won't ever be able to early release again anyhow? And if anything, are left out of important conversations.
Way to burn bridges. You only get to be an asshat once.
I have seen no evidence to back up this claim. Quite the contrary actually.
TIL observing an exploit in the wild, after the details leaked during the embargo and pushing a fix for your customer is "being an asshat".
Because if there are already publicly available exploits seen, it makes no more sense to keep holding patches back just so that OpenAI etc. can get a nice press release
1. There was a leak of the pre-announcement
2. Calif sees leak, goes and releases dancing anime girl video before pulling it, goes viral
3. Chainguard sees dancing anime video, doesn't recognize Calif, and thinks the exploit is in the wild
[Edit 3a]: Or alternatively, Calif's leaked video causes the PoC to be found by others by providing enough of a lead
4. Chainguard releases updated binaries thinking it doesn't matter, causing the real exploit to then be found early
5. Which then causes PHP Foundation to basically have a meltdown
Personally I blame Calif for the unprofessional video which absolutely completely looks like some script kiddie and for being too detailed [edit 2: assuming Calif produced it and the anime wasn't a prank edit]. In this case it wasn't just unprofessional fun but may have literally messed up the responsible disclosure.
I'm not sure about this part. I saw https://jackalsyn.com/posts/php-up-2026.html going around a bit over 12 hours ago, before the chainguard images went out, and plenty of other chatter of exploitation happening in the wild.
Still not great though.
For my rust applications, I use "cargo audit" which checks all dependencies against the CVEs. For my typescript and javascript applications, deno audit and/or yarn npm audit.
For the rest of the stack `debsecan` does exactly what you want on Ubuntu: report CVEs for installed software.
Maybe it would be worth it to have an agent that knows your stack, reads the top threads on hn all day long and messages you when it thinks there is something serious going on that could impact you.
PHP2SHELL, in particular, made me expand my weekly security review to include my external network resources.
The natural conclusion for this seems to be realtime monitoring CVE and pre-CVE vulnerability indications in exposed stacks.
Some have free licenses, many don't.
Sometimes build time is too slow (e.g., a repo that doesn't receive updates every day or so).
I hope that someone has some sort of a reasonable approach here / I would love to help work on something open source if not
But, certainly do your own threat assessment and develop and implement whatever plan works for you.