Posted by armcat 5 hours ago
But unfortunately, it’s not always this transparent. In the US, doctors (particularly PE owned ones) often use AI in their backend in one form or the other. In one case, I observed their transcription service was using some LLM to help process data into medical record.
In other cases, your data may even be processed/leaked/trained without your explicit consent through a third party vendor that doesn’t interact directly with doctors office (physician office -> order sent to pharmacy -> pharmacy uses LLM to process data).
This is at least the case for the insidious CVS/Caremark conglomerate.
The people sharing their PII usually don't know that it's Lovable.
although I don't use it enough to know the specific details of how it is infrastructured. But since the promise is that it writes any code that you want, it's logical that the arbitrary code is not a property of the code generating tool but of the user that generated the code!
Like mentioned elsewhere in this thread, there are doctors vibe coding shite and sending private medical data to Anthropic's servers, probably without even checking the "don't use for training lol" checkbox. Thousands of passports, invoices, whatever, are rn available on the open internet. People posting private company data on PUBLIC github repos (dealt with 2 such cases lol).
These individuals aren't held to the same standards, and right now nobody is enforcing the rules on them.
And maybe that's fine. Maybe we don't want to stifle innovation. But if my passport ends up on the dark web and I get identity theft'd, then I'd be pissed. There are REAL consequences.
Oh boy, open season on those entry points
But they have tons of VC money so they'll get aquired by someone in the incestuous circle of SV investors.
The defects listed in this report didn't require some complicated 0-day exploit chain - they were just sensitive data stored without any authentication at all, in a database that anyone with Internet access can view. In other words, it would be the equivalent of a company like Public Storage leaving all of its storage units with no locks on them at all.
Affected users should be able to sue for a lot in these basic security failures. And obviously for many of these small websites, the potential damages would quickly bankrupt the site owners, but that's the point. People would think a lot more carefully about how they protect sensitive data if there were real negative consequences for their failures.
To me, the issue is Lovable’s borderline reckless marketing: repeatedly claiming that “everyone” can build a website “without any technical knowledge” (verbatim from their website), while disclaiming all liability for bugs in their terms and placing responsibility for security on their customers.
Customers who hear day in, day out that “software is solved” and buy into Lovable’s narrative may simply have no idea that they could face substantial fines for privacy violations.
But in all cases, under GDPR, the one having the data breach is required to act. If they fail, there's steep fines, often proportional to your revenue. It will hurt the business.
E.g. here's what a ditch citizen can do and expect https://www.autoriteitpersoonsgegevens.nl/en/themes/security...
IANAL, but I've had several hosting companies. And while most situations where a customers' site is hacked and/or their data leaked, is clear and not the hosters' responsibility. That hoster does have an obligation to provide good tooling (i.e. ssh, not telnet, sftp, not ftp) and to promote good behaviour (i.e. not allowing the backup system to write the backups to public hosted directories). Or else can be held (partially) responsible for breaches.
Lovable has a responsibility, and, as you pointed out in your article, their proposition of "building apps without technical knowledge" (my words), might make theirs to be more than just putting lines in a EULA to waver that responsibility.
Imagine frontier abliterated models running at 2,400 TPS finding all these issues rapidly on web apps like this.
You are not ready security-wise when these models get faster. ("Formal verification" won't save you either)
Formal verification doesn't solve all the problems, but alongside concerns like correctness, restrained surface area, supply chain awareness, treating PII as toxic waste to avoid rathee than treasure to collect can leave projects/orgs in a much better place than they let themselves get too.
Long way to go from here though.
On the one hand, that’s kind of encouraging. On the other, it feels like the unskippable “don’t copy this movie” warning on a legally purchased DVD -- the one that gets removed from pirated versions.
It seems to punish the wrong people.