Top
Best
New

Posted by nodoodles 2 hours ago

NSA Internal Network Down(www.reuters.com)
220 points | 143 comments
lucascdotnet 2 hours ago|
I wonder (and have no idea) whether the increased reports of services being down is a case of the Baader–Meinhof phenomenon, or a actual increase in failure rate.
btown 1 hour ago||
If I was asked to armchair-pattern-match, I'd say that there's a combination of two things: (a) a wider fear-based environment where rapid modernization is being pushed heavily, and (b) an increase in tools that give false confidence on change management.

AI is tuned to speak confidently and ship independently; it is rare that it will suggest that one go back to a broad survey of stakeholders to ensure no Chesterton's Fence is being torn down (https://fs.blog/chestertons-fence/). "I've read the documentation and logs, and nothing suggests this system needs these legacy capabilities/components/capacity" is something we hear all day, and it is a siren's song if there ever was one.

Couple that with a general (and valid!) consensus that modernization of legacy systems must be accelerated for a variety of reasons - to mitigate this new and continuing "Black October" of hyper-agile agent-driven cyberattacks (whether the network is public or private); to have more flexible infrastructure for new features to stay ahead; to allow human teams to show their efficacy in a world where technology jobs are increasingly at risk - and you have a perfect storm where change management is no longer what it once was.

joe_the_user 32 minutes ago|||
I don't necessarily agree about the warnings. AI often gives me "now be careful" warnings about it's output along with the caveat at the bottom of each ChatGpt page.

I suspect the situation is that AI's caveats are just easier to ignore than a person's warnings. Plus all the other factors you list. Also, AI tends to give a person the kind of communication they ask for. I always ask for options but I suspect other people ask for just answer and so they get them.

NichoPaolucci 19 minutes ago||
[dead]
cyanydeez 38 minutes ago|||
I think he's just pointing out that AI has made news of IT failures "more interesting" and thus, more reports are getting out where in the past, they'd not be interesting.

I think you went off somewhere else.

joe_the_user 7 minutes ago||
The OP is giving an alternate explanation of the general phenomena that this headline is a part of. Nothing wrong with describing the framework that software "drift" is happening in.

Which is to say, a parent post here on HN doesn't narrow the topic to it's focus. That's what makes discussion here interesting.

Someone1234 1 hour ago|||
That's my impression; a total of zero of the popular recent ones have been related to "AI" or Cyber attacks, but every comment in every thread, talks about those two as if they were the underlying cause.

The FAA's recent outage for example, at its core, seems like people trying to rush needed upgrades that were slow for a specific reason (e.g. reducing the testing window for 90 to 30-days).

The Microsoft outage seems like three different things linked together (and poor "full failure" recovery prep, lengthening the outage).

The bank "outage" may not have even existed, 2/3 of the banks claimed they didn't have an outage at all, and the other said it was a limited technical issue. The articles were based on DownDetector spikes.

rkozik1989 1 hour ago|||
You need to be more skeptical of what organizations say when they have a lot to lose. Nobody takes responsibility for situations unless there's way to offload some or all of the blame. They have every incentive to lie about the situation to make it go away. Politics often plays the biggest role in these situations.
StilesCrisis 1 hour ago|||
I logged into my bank when the report came out and did not experience any hiccup. I think this was DownDetector-based reporting.
nosrepa 18 minutes ago|||
Anecdotally, my wife was unable to access her Wells Fargo account when the outage was reported on.
core_dumped 49 minutes ago|||
"works on my machine"
Someone1234 37 minutes ago||
Works according to 2/3 of the banks themselves. So everyone is lying or wrong, and DownDector-based media reports are beyond reproach?

This has reached a Witch-hunt levels of fever pitch.

CookieCrisp 4 minutes ago|||
It's absolutely a mass hysteria - I imagine in 40 years there will be documentaries on this like there are now about the Satanic Panic
monkpit 21 minutes ago|||
Just like AWS is always up because the status page is green, right?
Someone1234 40 minutes ago|||
> You need to be more skeptical of what organizations say when they have a lot to lose.

So let me see if I understand what I should be:

- Be more skeptical of official explanations, including where it includes a technical breakdown of how it occurred (e.g. Microsoft/Outlook outage).

- Be less skeptical of random commenters claiming, without proof, that it is AI or Cyber Attacks.

Because that's the implied point you're making. Be more skeptical of A and less skeptical of B, even though A has someone placing their reputation behind it, and B is anonymous commenters throwing made up theories out into the ether.

swasheck 50 minutes ago||||
> The FAA's recent outage for example, at its core, seems like people trying to rush needed upgrades that were slow for a specific reason (e.g. reducing the testing window for 90 to 30-days).

The explanation for this one still doesn't ring true for me and that's exacerbated by how many times the story has changed after telling the public that they didn't need to know the reasons.

Agree, though, that the rest do have reasonable explanations

egypturnash 32 minutes ago|||
Given how many companies seem to be working hard to have all programming work done by AI, and pushing out everyone who says this sounds like a bad idea, is "AI" really not going to be a significant part of "rushing out needed updates"?

Is "Claude says it looks good so let's skip the usual testing window and push to prod on a Friday" really not a sentence you can imagine being said in this current environment?

Someone1234 21 minutes ago||
I am talking about outages that have already recently occurred. You're making a distinct point about a hypothetical.

My point is that people are taking a potential future problem and applying to actual things that have recently happened, that didn't fit that mold. Which is incorrect.

When AI or Cyber Attacks cause an outage, people are absolutely right to say so. But claiming - without proof, that every outage is related to the two is noise.

kesslern 1 hour ago|||
This situtation reminds me of the East Palestine train derailment in Ohio. A bad train accident caused a big chemical spill, made national news. For months afterward, any issue with any train in the state was reported as news. People thought the trains were breaking down. In reality, train derailments happen pretty often but it's not usually a big deal so nobody hears about it.
orblivion 12 minutes ago|||
It must be more than just that because it's not just the experience of one person noticing things. It probably starts that way, but then the stories get shared and they bubble up more often. So it's another phenomenon on top of it.

(This of course is on top of whatever unusual things might actually be happening)

NitpickLawyer 1 hour ago|||
100%. Everything is rage-bait nowadays. And AI is the newest toy they can use. Every time we've had something go down in the past ~6months, "AI did it" was all over the comments.
rsoto2 1 hour ago|||
https://www.optimizely.com/field-notes/articles/how-we-use-a...

https://www.bleepingcomputer.com/news/security/optimizely-su...

Is it possible that the false of productivity is leading to worse products and massive loss in valuable headcount? Instead of AIs directly writing bad buggy code? Or is "everything rage-bait, and AI is all sunshine"

ragequittah 1 hour ago|||
I believe it's also possible that a large portion of the world wants this to be true more than it actually is so attribution to things that have nothing to do with AI go directly to AI as a knee-jerk reaction.
GolfPopper 1 hour ago|||
I'm not a fan of where I see LLMs taking us, but they're just the latest notch in decades-long institutional decay.
AeroNotix 58 minutes ago|||
If we're also just vibing it (like the majority of code these days) then I vibe that there are a lot of AI-apologists that desperately don't want their magic machine to be cast in a negative light, lest their easy i-win button gets taken away or devalued.
ragequittah 31 minutes ago||
I think both sides are exhausted by too many people on the other side have a non-nuanced argument. I fall in the middle. I love making stuff with AI, it's fun. It's become a better search engine for me and helped with many other things in my life. But it'd be really naive to not see the enormous downsides. They're everywhere and not pleasant.

There's a group of people who think AI is perfect and will solve all of humanity's problems. That's annoying. There's another group of people who think AI is the worst thing to ever happen to humanity and is the downfall of society. These people are more vocal in my experience and they're everywhere. If you ever use AI you're a bad person who they'll disown like you're racist. It's somehow become the most popular 'social justice' issue of our time even as ICE exists alongside it. These people see a problem and immediately try to figure out how AI is involved. That's far more annoying.

Best example I can give that came up in my circle recently is the video game "Big Walk". It was recently denied for a silly award nomination because the awards have a strict no-AI policy. Which to me is absolutely fine, go nuts. But the vitriol towards this game, that everybody absolutely ADORED before the developer disclosed that they fixed ~10 lines (!) of code with AI because they couldn't figure it out otherwise, is absolutely bonkers. People I respect treated it like they were mourning something that WAS great until this catastrophe happened.

cochne 1 hour ago||||
These kinds of things happened before AI. It's not clear whether the incidence has increased since AI, or maybe AI is leading to faster development and therefore higher rates of incidents may be due to shipping more code, faster, while perhaps maintaining a similar or better level of incident to code/progress ratio. Or, there could just be more attention as GP suggests. Pointing to one example of a company that's using AI and has has a major incident doesn't really tell us much, especially when most companies who haven't had major incidents are probably using AI in similar amounts.

It would be interesting if someone could do a proper analysis, though!

rsoto2 39 minutes ago||
It's not one example, it's the latest in a long string of examples over the last few years.
wolfinterval 57 minutes ago||||
I would assume if anything it is that the open weight models are lagging by 6-8 months and 6 months ago Mythos was announced.

It would hardly be shocking that hostile nations are able to cause some problems with Mythos level tools.

If anything, it is more interesting how we can invade another country and the idea that the country or a collection of hostile countries could hit us back in any way, shape or form is just an impossibility. An insane conspiracy theory.

Maybe it is not that but surely the probablity is not zero.

femboyvtuber 1 hour ago|||
[dead]
matt3210 1 hour ago|||
what was your system prompt again?
kilroy123 1 hour ago|||
Possible, but let me ask you and others. What does your gut say? Mine is screaming that something just isn't right right now.

I can't put my finger on it, but something sinister feels afoot.

eterm 59 minutes ago|||
UK government recently launched a "Be Ready" preparedness campaign.

https://beready.campaign.gov.uk/

It's nowhere near the kind of readiness and preparedness campaigns of the cold-war of course, but it's a step up in alertness that's higher than anything I can remember.

rwmj 53 minutes ago|||
It's a new campaign, but the messasge is not new at all. The previous government had a preparedness campaign in 2023 (https://www.gov.uk/government/publications/the-resilience-fr...). And you could go back to 2004 and the UK government's "Preparing for Emergencies" campaign where a leaflet was sent to every household (https://en.wikipedia.org/wiki/Preparing_for_Emergencies).
kilroy123 53 minutes ago|||
I know, I live in London right now, and people are worried.
trinsic2 49 minutes ago||||
Yeah, agreed. I look at patterns. When there are long term outages across multiple disciplines, something systemic is being changed for good or ill. In this case, it being right around the midterms, I'm thinking its ill. Organizations (not sure which, Peter Thiel?) are trying to push us into a total surveillance system, I'm thinking some of these things are related to that.
scottyah 31 minutes ago||||
My gut is screaming that everything is mostly normal, but the kids aren't alright.

Too many people are addicted to their smartphones and don't feel the same level of pride they used to in their work. All systems are shared, instead of one person being the ruler of a certain part of the stack, it's a revolving door and it's rare that anyone stays somewhere for even 5-10yrs. It leaves a wake of modifications that nobody else can keep up with. Also, the tech industry is moving fast just to move fast, a lot of old infra is still C++ and COBOL but good luck hiring anyone to use that, so they have to convert the systems to use the Latest Tech of the Day. The old, stable people don't have any interest in learning the new thing, and would rather retire with their knowledge. The new people want to rebuild from scratch in the New Thing and are convinced they need to job hop immediately after.

digitaltrees 39 minutes ago||||
Like maybe we are at war with a country that doesnt have the ability to fight with conventional methods and is engaged in cyber warfare?
0cf8612b2e1e 59 minutes ago||||
My personal barometer: when do government insiders sell all of their stock? They have already done all of the insider trading. Fleeing the sinking ship the second they hear of Bad News would be in line with previous behavior.
dualvariable 58 minutes ago|||
Most of the rest of them I can wave off as probably incompetence and maybe augmented by LLM-psychosis in decision-makers, and the general level of chaos in IT these days. The NSA internal network being down is pretty suspicious. Maybe the Trump administration rot has just spread that far, but that is making me think there could be actual cyberwarfare going on.

[To be clear: I mean exactly what I said, which is "there could be" which is not the same as "I think there is" -- so call it a 5%-10% chance of real cyberwarfare -- which means I'm not dismissing those claims out of hand, and find the claims at least plausible, even if I don't currently believe them]

le-mark 42 minutes ago||
Interesting thought but who are the combatants and why are they engaging now? Corporation? Nation states? Rogue ai? We have no news of significant consequences from the outages thus far other than inconvenienced travelers. Quite benign form of warfare thus far.
jazzyjackson 35 minutes ago||
Economic damage, psychological warfare, water and electricity being similarly vulnerable

It’s not nothing

But my gut says incompetence and rushing upgrades out of fear of cyberwarfare, not an actual attack.

burnte 28 minutes ago|||
In the case of government, we have an administration that works on whims and guesses, and moves too fast, breaking a lot of things. We have absolutely seen more human-caused-issues.
dmix 2 hours ago|||
The media will certainly try to give that impression either way. They serve the doomscrollers now.
causal 1 hour ago||
Yeah "cyber thing went down" has certainly become a reporting trend. I'm not sure we would have even heard about NSA internal network issues a year ago. May or may not be significant.
mrguyorama 10 minutes ago||
I 100% agree.

Though at the same time, the public isn't privy to even 10% of the incidents going on right now.

The information I have is consistent with general cybercrime and script kiddie patterns, just a much higher volume, and much larger diversity of targets.

220hertz 1 hour ago|||
I think it lies at the intersection of Occam's and Hanlon's razors. The simplest explanation is incompetence brought on by a lack of skill from cuts, and therefore stupidity not malice.
mrguyorama 8 minutes ago|||
At the level of national government, stupidity is malice.

You have aides and resources and money. Not having enough information or understanding is a choice.

It's criminal negligence.

warkdarrior 1 hour ago|||
It's probably both stupidity and malice, especially on the government side. Project 2025 makes sure that malice is there as a starting point and then the current loyalty-driven hiring practices means that stupidity is a key factor in the execution.
matthewbauer 1 hour ago|||
Must be a little bit of both right? Like FAA grounding + Microsoft outage are both notable on their own, but the subsequent things would usually be background noise IMO.

I also think generally that AI is probably both the cause and solution to these problems. We're certainly getting more code churn than otherwise, and in lots of cases that's really good! But you're going to have to deal with inevitable instability too.

EvanAnderson 1 hour ago|||
Aside: I forget the name of the Baader–Meinhof phenomenon when it hasn't been discussed in awhile. When I'm reminded of it I begin seeing it happening frequently for the next few days. I wish I was joking. I assume it's the Baader–Meinhof phenomenon phenomenon.
smolder 1 hour ago||
The term "frequency illusion" is a lot more memorable and intuitive. It's what I prefer.
ihumanable 1 hour ago||
I've heard the term "frecency bias" a portmanteau of frequency and recency to refer to people focusing intently on a set of outlier events that happen to occur with a heightened frequency very recently.

I'm sure most of us here have been on the receiving end of managers suddenly obsessed with a failure mode because it happened to occur twice in one month.

takeda 10 minutes ago|||
It's the same as when Boeing had failure then we heard only of Boeing and other failures for few weeks. It's not just the Baader–Meinhof phenomenon but because of the previous failures news are interested more to report on failures. Like when did we normally care that NSA had an outage?

I think the real ones were the Microsoft and FAA due to deploying the AI slop.

The other ones seem to be usual outages that normally no one reports on.

losvedir 58 minutes ago|||
aka The Summer of the Shark. https://en.wikipedia.org/wiki/Summer_of_the_Shark
orblivion 8 minutes ago||
This is so beautifully morbid.
mmooss 13 minutes ago|||
Someone this morning told me there might be a 3 month blackout in the US, somehow connected to other recent technological failures. I later learned the blackout theory is based on a The Simpsons episode, and even there it was a much shorter time period but has been exaggerated.

So certainly there is a big social-psychological phenomenon. Or it's gonna be a cold winter! :)

joe_the_user 46 minutes ago|||
Technically, the "Baader–Meinhof phenomenon" is people personally noticing X once it's been reported. What you are describing is technically a "news wave" or "Media Feeding Frenzy".

But I'd note, these "named after a thing" phenomena (eponnyms) are kind of terrible for exact description since their definition tends to be fuzzy, there's a temptation to fudge their borders and they create an insider jargon (if only I could a device that could reach of the computer and slap everyone who says "steelman" when they mean strengthen an argument, etc).

arisAlexis 52 minutes ago|||
It's an actual increase in offensive AI cyber capabilities
redwood 54 minutes ago|||
With the trilateral summit scheduled for Thursday it's possible some showing off is going on between the three letter agencies in the three countries... after all they can be first movers on vulns that are likely to be patched soon
lettergram 1 hour ago|||
Everyone is being pushed for rapid updates, the AI is doing the updating (error prone) and there's actually an increase in CVEs and potential risks.

The combination is a cycle

1. AI finds bugs

2. Engineers us AI to try and fix

3. AI testing approves, minimal engineering review

4. Engineers push release rapidly, it's critical after all

5. Bugs introduced

6. Cycle starts again

b112 23 minutes ago|||
My suspicion is that there are discovered zero days so severe, that the banks and NSA and so on, are being patched first.

With all the recent asshattery of leaking kernel exploits, I would presume trust is gone, and it's cover the big boys first.

(a linux kernel bug vulnerability could be patched without lkml involvement or distros even knowing)

stronglikedan 1 hour ago||
This is all just to distract us from something much, much worse! /s
dustfinger 36 minutes ago||
I may not understand expectations, or the law around this, but I find it a little surprising that the NSA would publicly announce that their internal and highly classified network is down. At least not before getting it back up, running and secure again.
netsharc 31 minutes ago|
In a spy movie world, someone deep in enemy territory would be relieved to see the Reuters headline, after failing to reach the NSA network and having no other channel to ask (picking up the phone to call the US embassy would get your door busted by the local men in black).

I imagine, for the paranoid spy, not being able to call home could also mean "the enemy has severed our comms, we've been compromised, burn everything and run!".

qphe95 13 minutes ago||
Thanks for the explanation! Good thing spy movies are just movies and not real!
tgsovlerkhgsel 2 hours ago||
That's a LOT of things going down, and I haven't heard of some major infrastructure outage likely to affect many sites in this manner.

Coincidence or is there some common explanation that I missed?

bhouston 2 hours ago||
There is a lot of things going down.

I think the opportunity right now where we have powerful AI hacking systems but not enough protections is to establish long-term presence in all networks of interest. Not bring them down.

colinrand 1 hour ago|||
Completely agree. Things are moving too fast and so are breaking.

For example, we updated a GCP managed base image of Ubuntu that was released with 350 CVEs patched, some were as high as 9.8. In doing so, they (GCP) changed the DEFAULT setting for hardware GRO from off by default to on. That's a massive networking change packed into a security update.

We only caught it AFTER it started causing issues in production. We didn't have the OS pinned because (decently valid reasons) or catch it in non-prod (decent reason + human mistake).

This is on a modern highly automated tested resilient well architected platform.

I can only imagine these other platforms are dramatically more exposed that we are, hence these common large scale outages.

pianopatrick 1 hour ago||||
That assumes that people did not already have long-term presence in networks of interest.

More plausible to me is that spy agencies had previously found and used some of these hundreds or thousands of vulnerabilities to hack into those networks just fine. But now other people like cyber criminals are using these vulnerabilities via AI. That is messing up both the attackers who had already gotten in quietly and do not want the attention and the defenders who now have to respond to a noisy intrusion they cannot ignore.

cj9 1 hour ago||||
Also, anyone anywhere that horded/designed exploits will be wanting to use their backlog of these "assets" before systems are patched. From criminal networks to nation states.

Outages could be grey-hat testers and emergency updates.

A temporary period like we are experiencing is to be expected when moving to a time where a single engineer can perform the equivalent work of a team of excellent engineers working for weeks. Security has always been a game of who has the capabilities, means, and requirement for any given attack surface.

daheza 2 hours ago||||
None of these have been due to hacking? It has all been just standard issues probably increased by cost cutting measures.
bhouston 1 hour ago|||
Or the increased use of AI agent coding.
johnofthesea 1 hour ago||
All of above and incompetent people in power. (Not implying in this specific case).
Florin_Andrei 49 minutes ago|||
We also have powerful incentives to deliver more "value to the shareholders", while using AI code generators we don't fully understand.
juliusdaniel7 48 minutes ago||
[dead]
wat10000 11 minutes ago|||
It reminds me of the aftermath of the DCA crash in 2025. For a few weeks it felt like planes were falling out of the sky as a bunch of (sadly somewhat commonplace) non-airliner crashes got national attention they otherwise wouldn't have.
postsantum 1 hour ago|||
Appstore connect is also down
chabes 1 hour ago|||
What do people expect when top government offices are occupied by former Fox News hosts?
winddude 19 minutes ago||
Elon Musk and Trump with DOGE. They cut ~300k federal workers, cut governments contracts across all departments, including the FAA, NSA, home land security... those organizations have been raising alarms since that the cuts to civilian contractors etc would lead to outages and issues since, including pentagon, DoD, etc. But you know Idiocracy.
gnabgib 2 hours ago||
CNN Source (27 points, 5 comments) https://news.ycombinator.com/item?id=50532849
lacoolj 29 minutes ago||
It would help if the moment the service came back online the title of the post could be changed to (FIXED) or something

Like the United Airlines thing yesterday, the moment I got to the article it said "was down for a few hours" but the post was still just something about United Airlines network being offline

qphe95 16 minutes ago||
So how are people getting their work done right now?
ericcholis 2 hours ago||
There's a part of me that doesn't want to know this....
Consermo_AI 50 minutes ago||
its getting spooky
wankerrific 32 minutes ago|
It’s absolutely not ai
More comments...