Top
Best
New

Posted by bluehatbrit 1 hour ago

Tailscale didn't stop the Hugging Face intrusion(tailscale.com)
140 points | 48 comments
john_strinlai 1 hour ago|
>No “vulnerabilities” in Tailscale were found or exploited, and that might make it even more uncomfortable for us. [...] But, we're a security tool. Their intrusion is our intrusion, and it's our job to take it seriously.

im a happy customer of tailscale, so i am obviously biased, but i have a lot of respect for this. they could have just stayed quiet and i dont think anyone would have bat an eye.

guessmyname 9 minutes ago||
Expect similar articles (cough, ads, cough) in the next couple of days from every single company whose software was involved in the incident.
petesergeant 12 minutes ago|||
This article is just an ad / public-service-announcement for various paid Tailscale features, though?
behnamoh 49 minutes ago|||
Tailscale as a company reminds me of Valve and other good old tech-oriented people that I can "trust" that they know what they're doing. I'm a happy customer too and I hope they retain the essence of what distinguishes Tailscale.
traceroute66 27 minutes ago||
> I can "trust" that they know what they're doing

I wish I could say the same.

At $work we use Tailscale but only bare minimum and at arms length and only because we have to (people were having NAT issues with standard Wireguard).

None of their code has had a security audit, let alone regular ones. Yes they make a big song and dance about SOC2/ISO27001 but that is NOT the same thing, that's just shiny tick-boxes for compliance departments.

They seem to rely entirely on the random goodwill of others to do random audits of unknown coverage at random intervals, not exactly reassuring.

"Tailsale Lock" is, being polite, a hot mess. So many sharp edges and footguns.

Their introduction of TPM-by-default and then removing it a few weeks later because of a seemingly small number of edge-cases and very odd reasoning was just weird.

Yes they are nice guys to chat to and all that. But for a security tool they need to up their game seriously.

aborsy 20 minutes ago||
It does too many things, and the product has got too complex. I saw a year ago they were looking for someone just to help with complexity.

I use it but feel uncomfortable, that it has large attack surface and LLMs will find exploits in it.

Without taillock it makes no sense. Anyone on their coordination servers will be able to connect to your network.

smb06 1 hour ago||
Glad to see companies owning responsibility and putting out a message without corporate PR spin
DC-3 28 minutes ago||
If you can't see the spin on corporate messaging it means it's working (and consequently, to stretch the metaphor, your wicket is in danger).
ahofmann 21 minutes ago||
Wow, this article is super smart marketing by tailscale. Not only do they list all the nice and expensive features, that can help in such a situation but they also show that someone at huggingface made a very stupid thing by writing a reusable auth key in an env file. Everyone using mesh VPNs like tailscale, netbird etc. knows that this is like leaving the keys right at the door.
simonw 13 minutes ago||
> One of those 136 credentials was a reusable Tailscale auth key, used to create new Tailscale CI (continuous integration, used for automated testing) nodes in their tailnet. The agent copied that key into a series of external sandboxes and used it, over several days, to enroll a total of 181 nodes into Hugging Face’s tailnet. Those nodes each received a Tailscale identity tag granting all the access a CI node would get.

This feels like an alerting opportunity. I wonder what the lowest friction way would be for Hugging Face to have alerts if 181 unexpected nodes were added to a tailnet.

iamspoilt 39 minutes ago||
Quoting Tailscale: This is our very Canadian apology: sorry you stepped on our toes. The attack didn’t exploit Tailscale, and Tailscale didn’t cause the compromise. But, we didn't stop it. Next time, we will.
kennywinker 23 minutes ago|
Next time we will - provided you further lock yourself into our ecosystem.
brendoelfrendo 6 minutes ago||
I mean, they do happily explain the TailScale features that you can use to avoid this kind of issue, but the general advice of "don't leave long-lived keys lying around where they're accessible, do anything except for that" is pretty generic, good advice.
paxys 43 minutes ago||
I don’t think it was the VPN’s job in any case. Once the attacker has found a backdoor into the private network and obtained root access to a VPN’d machine, its game over no matter what your Tailscale config says.
bumbledraven 1 hour ago||
Does Tailscale offer a "security checkup" function? Best practices evolve over time, and it would be nice to know if I'm using the recommended configuration.
jaxxstorm 53 minutes ago|
I lead the customer engineering org at Tailscale.

We think this is a great idea and we're discussing internally potentially adding that to the console.

In the meantime, if you'd like to get an assessment, please feel free to open a support ticket (https://tailscale.com/contact/support?type=other&subject=sec...) and we'll happily take a look

tomrod 48 minutes ago|||
Just so you know, Tailscale tech has always impressed me, but your responsiveness here has cemented my appreciation of your org and I look forward to bringing yall in as our company hits the growth phase.

May y'all continue to be customer focused.

6thbit 18 minutes ago|||
I've always found tailscale's json config a bit intimidating. I greatly appreciate the new UI that makes it easier to define rules, alas, both going to relevant docs straight from it and determining 'is this rule just lazy/bad/unsafe' is hard and frustrating most of the time.

That's where I'd like to see this sort of checkup. Yell at me please if i just said anyone can ssh as root from any node!

jmartrican 59 minutes ago||
Prediction of a future transcript at a press conference after some major AI caused disaster: "We had no idea that the model would be capable of..... ".
luciana1u 57 minutes ago||
a security company writing a blog post titled 'we didn't stop the intrusion' is the most honest thing in the entire security industry this year
colek42 36 minutes ago||
I really wish they would support SPIFFE/SPIRE
monster_truck 42 minutes ago|
You know what would have gone a long way to stopping this? Not leaving credentials as env variables in containers. Vault is not that hard to stand up and utilize.

We need to bring shame back, the humans responsible are supposed to be professionals.

kenniskrag 34 minutes ago|
Why should that help to have no env variables?
More comments...