Top
Best
New

Posted by lpage 12/20/2025

NTP at NIST Boulder Has Lost Power(lists.nanog.org)
488 points | 210 commentspage 2
tguvot 12/20/2025|
We had some fun requesting key for accessing nist time servers. the process is (quoted from website)

NIST currently offers this service free of charge. We require written requests to arrive by U.S. mail or fax containing:

Your organization’s name, physical address, fax number (if desired as a reply method).

One or more point-of-contact personnel or system operators authorized to receive key data and other correspondence: names, phone numbers, email addresses. Up to four static IPv4 network addresses under the user’s control which will be allowed to use the unique key. By special arrangement, additional addresses or address ranges may be requested.

Desired hash function (“key type”). NIST currently supports MD5, SHA1, SHA256, and HMAC-SHA256. Please list any limitations your client software places on key values, if known: maximum length, characters used, or whether hexadecimal key representations are required. If you prefer, please share details about your client software or NTP appliance so we can anticipate key format issues. Desired method for NIST’s reply: U.S. mail, fax, or a secure download service operated by Department of Commerce.

NIST will not use email for sending key data.

ps. there actually seems to be improvement over what they had year ago. they added "secure download service". and previously they had message that nobody assigned to actively monitor mailbox so if you didn't get key, please email us so we will check it

samch 12/20/2025||
Can anybody speak to the current best practices around running underground power lines? I see these types of articles about above-ground distribution systems from time-to-time, particularly in California. I feel lucky that my area has underground power, but that was installed back in the 1980s. Would it be prohibitively expensive for Boulder’s utility provider to move to underground distribution? I can’t help but think it could be worth the cost to reduce wildfire risk and offer more reliable service.
pseudonymidy 12/20/2025||
Think of it like this: overhead power lines require you to dig a 5-7’ deep hole that’s 2’ in diameter every 90’. Underground power supplied through cable requires you to bury the cable minimum 3’ in the ground in rigid ductwork the entire 90’. Any time that cable runs under a roadway that ductwork needs to be encased in concrete. In urban and semi urban areas you also compete with other buried infrastructure for space - sewer, city/municipal infrastructure, gas, electrical transmission, etc.

While underground distribution systems are less prone to interruption from bad weather it depends on the circuit design. If the underground portion of the circuit is fed from overhead power lines coming from the distribution substation you will still experience interruptions from faults on the overhead. These faults can also occur on overhead transmission circuits (the lines feeding the distribution substations and/or very large industrial customers).

Underground distribution comes at a cost premium compared to overhead distribution. It’s akin to the cost of building a picket fence vs installing a geothermal heating system for your home. This is why new sub divisions will commonly have underground cable installed as the entire neighborhood is being constructed - there’s no need to retrofit underground cable into an existing area and so the costs are lower and borne upfront.

It’s more cost effective for them to turn the power off as a storm rolls through, patrol, make repairs and reenergize then to move everything underground. Lost revenue during that period is a small fraction of the cost of taking an existing grid and rebuilding it underground. This is especially true for transmission circuits that are strung between steel towers over enormous distances.

lazyc0der 12/20/2025|||
Boulder County is working on utility undergrounding, https://bouldercolorado.gov/services/utility-undergrounding, and in particular they are working on undergrounding at Chautauqua, https://bouldercolorado.gov/news/undergrounding-chautauqua, which is next door to the NIST facility.
shadowpho 12/20/2025|||
1/3-1/2 of the cost of the electricity we pay is distribution.

Some of it is physical infrastructure (transformers, wire, poles), but a lot of it is labor.

Labor is expensive in US. It’s a lot of labor to do, plus they’ll likely need regulatory approval, buying out land, working through easements.

At the same time you have people screaming about how expensive energy is.

Furthermore they have higher priorities, replacing ancient aging infrastructure that’s crumbling and being put on higher load every day.

tekno45 12/20/2025||
Its very hard to repair and keep track of underground.

https://practical.engineering/blog/2021/9/16/repairing-under...

manosyja 12/20/2025||
Germany here, never heard of any issues regarding underground power (or phone) lines. Ultra High voltage (distribution network) is above ground here, but no issued with that either.
DamonHD 12/20/2025||
So far I think I'm still seeing one of them in my peers list for my public-ish NTP server:

         remote           refid      st t when poll reach   delay   offset  jitter
    ==============================================================================
    +time-e-b.nist.g .NIST.           1 u  372 1024  377  125.260    1.314   0.280
DamonHD 12/20/2025|
...and maybe it's gone:

    #time-e-b.nist.g .NIST.           1 u 1071 1024  377  125.260    1.314   0.280
8organicbits 12/20/2025||
The referenced mailing list is this Google Group (https://groups.google.com/a/list.nist.gov/g/internet-time-se...) which has some other posts about this incident.
creatonez 12/21/2025|
Where are you seeing the other posts?
sgnelson 12/20/2025||
One question I have is did DOGE decisions have anything to do with this? Because I know they took knives to NIST.
grepfru_it 12/20/2025||
Residents and some businesses of Boulder have been without power since Tuesday. There was an issue about 10 years ago which caused 1000 homes to burn down and the power company was found liable. They change their actions. Then during the next high wind event, the power company preemptively cut power and businesses sued them for loss of revenue. Now the power company is playing it safe and turning off power to residents and keeping downtown businesses powered.

Maybe their generator failing was DOGE related, but wouldn’t have happened if state level shenanigans were better handled

_se 12/20/2025||
The Marshall fire was 4 years ago. Almost to the day.
cramcgrab 12/20/2025||
Actually DOGE involvement at the highest level would have resulted in Tesla solar and Tesla powerwall battery backups.
throw0101c 12/20/2025|||
> […] Tesla solar and Tesla powerwall battery backups.

Don't forget Solar Roof.

cramcgrab 12/20/2025||||
Apparently no engineers here anymore, all armchair politicians.
hiddencost 12/20/2025||
When the politicians take a hatchet to the underpinnings of society that we built, we have to get political.
cramcgrab 12/21/2025||
So sad, we used to innovate and stay one step ahead of them. That’s how wealth is created.
redbush237 12/20/2025||||
Lol, lmao.

Some relevant DOGE’s effects:

-time and frequency division director quit

-NIST emergency management staff at least 50% vacant

-NIST director of safety retired, and NIST safety was already understaffed when compared to DOE labs

-NOAA emergency manager on the same Boulder campus laid off

etc

the_gipsy 12/20/2025|||
Any day now
stackghost 12/20/2025||
Direct link, rather than a repost: https://groups.google.com/a/list.nist.gov/g/internet-time-se...
digdigdag 12/20/2025||
My local GPS + PPS based NTP server finally pays off.
gilrain 12/20/2025||
It’d be a good idea to protect our infrastructure from the climate we created.

It’s just a good idea, though, not a greedy one… so it won’t happen.

qmarchi 12/20/2025|
Man, they're having a hell of a time up in Boulder.
More comments...