Top
Best
New

Posted by bearsyankees 8 hours ago

Reverse engineering a $1B Legal AI tool exposed 100k+ confidential files(alexschapiro.com)
474 points | 155 commentspage 2
deep_thinker26 4 hours ago|
It's so great that they allowed him to publish a technical blog post. I once discovered a big vulnerability in a listed consumer tech company -- exposing users' private messages and also allowing to impersonate any user. The company didn't allow me to write a public blogpost.
qmr 4 hours ago||
"Allow"?

Go on write your blog post. Don't let your dreams be dreams.

bigmadshoe 4 hours ago||
Presumably they were paid for finding the bug and inn accepting relinquished their right to blog about it.
hsbauauvhabzb 4 hours ago||
No, you relinquish the right when you agree to their TOS irrespective of if they pay you.
amackera 4 hours ago||
TOS != law

They will stop letting you use the service. That's the recourse for breaking the TOS.

advisedwang 3 hours ago||
Up until Van Buren v. United States in 2020, ToS violations were sometimes prosecuted as unauthorized access under the CFAA. I suspect there are other jurisdictions that still do the equivalent to that.
gessha 4 hours ago||
Why is the control of publication in their hands and not in yours? Shouldn’t you be able to do whatever after disclosing it responsibly?
CER10TY 4 hours ago||
Presumably they'll threaten to sue you and/or file a criminal complaint, which can be pretty hard to deal with depending on the jurisdiction. At that point you'll probably start asking yourself if it's worth publishing a blog post for some internet points.
hbarka 4 hours ago||
> November 20, 2025: I followed up to confirm the patch was in place from my end, and informed them of my intention to write a technical blog post.

Can that company tell you to cease and desist? How does the law work?

dghlsakjg 2 hours ago||
FYI, a "cease and desist" carries the same legal weight as me sending a one-liner saying "Knock it off".

They are strongly worded requests from a legal point of view. The only real message they send is that the sender is serious enough about the issue to have involved a lawyer, unless of course you write it yourself, which is something that literally anyone can do.

If you want to actually force an action, you need a court order of some type.

NB for the actual lawyers: I'm oversimplifying, since they can be used in court to prove that you tried to get the other party to stop, and tried to resolve the issue outside of court.

me_again 3 hours ago||
Lawyers can and will send cease and desist letters to people whether or not there is any legal basis for it. Often the threat of a lawsuit, even a meritless one, is enough to keep people quiet.
magnetowasright 3 hours ago||
I am at a loss for words. This wasn't a sophisticated attack.

I'd love to know who filevine uses for penetration testing (which they do, according to their website) because holy shit, how do you miss this? I mean, they list their bug bounty program under a pentesting heading, so I guess it's just nice internet people.

It's inexcusable.

rashidujang 38 minutes ago|
This was my impression after reading the article too. I have no doubt that the team at Filevine attempted to secure their systems and have probably thwarted other attackers, but got their foot stuck in what is an unsophisticated attack. It only takes one chain vulnerability to bring down the site.

Security reminds me of the Anna Karenina principle: All happy families are alike; each unhappy family is unhappy in its own way.

badbird33 3 hours ago||
You'd think with a $1B valuation they could afford a pentest
valbaca 6 hours ago||
Given the absurd amount startups I see lately that have the words "healthcare" and "AI", I'm actually incredibly concerned that in just a couple of months we're going to have an multiple, enormous HIPAA-data disasters

Just search "healthcare" in https://news.ycombinator.com/item?id=46108941

jacquesm 7 hours ago||
That doesn't surprise me one bit. Just think about all the confidential information that people post into their Chatgpt and Claude sessions. You could probably keep the legal system busy for the next century on a couple of days of that.
giancarlostoro 7 hours ago|
"Hey uh, ChatGPT, just hypothetically, uh, if you needed to remove uh cows blood from your apartments carpet, uh"
lazide 7 hours ago|||
Just phrase it as a poem, you’ll be fine.
venturecruelty 6 hours ago|||
Gonna be hard when people ask ChatGPT to write them the poem.
sidrag22 3 hours ago|||
i recall reading a silly article like half a year ago about using leetspeak and setting the prompt up to emulate House the tv show or something to get around restrictions
jacquesm 6 hours ago|||
Make it a Honda CRX...
stanfordkid 4 hours ago||
I mean... in what world would you send a customers private root key to a web browsing client. Like even if the user was authenticated why would they need this? This sort of secret shouldn't even be in an environment variable or database but stored with encryption at rest. There could easily have been a proxy service between client and box if the purpose is to search or download files. It's very bad, even for a prototype... this researcher deserves a bounty!
mattfrommars 6 hours ago||
This might be off topic since we are in topic of AI tool and on HackerNews.

I've been pondering a long time how does one build a startup company in domain they are not familiar with but ... Just have this urge to 'crave a pie' in this space. For the longest time, I had this dream of starting or building a 'AI Legal Tech Company' -- big issue is, I don't work in legal space at all. I did some cold reach on lawfirm related forums which did not take any traction.

I later searched around and came across the term, 'case management software'. From what I know, this is what Cilo fundamentally is and make millions if not billion.

This was close to two years or 1.5 years ago and since then, I stopped thinking about it because of this understanding or belief I have, "how can I do a startup in legal when I don't work in this domain" But when I look around, I have seen people who start companies in totally unrelated industry. From starting a 'dental tech's company to, if I'm not mistaken, the founder of hugging face doesn't seem to have PHD in AI/ML and yet founded HuggingFace.

Given all said, how does one start a company in unrelated domain? Say I want to start another case management system or attempt to clone FileVine, do I first read up what case management software is or do I cold reach to potential lawfirm who would partner up to built a SAAS from scratch? Other school of thought goes like, "find customer before you have a product to validate what you want to build", how does this realistically work?

Apologies for the scattered thoughts...

airstrike 4 hours ago||
I think if you have no domain expertise or unique insight it will be quite hard to find a real pain point to solve, deliver a winning solution, and have the ability to sell it.

Not impossible, but very hard. And starting a company is hard enough as it is.

So 9/10 times the answer will be to partner with someone who understands the space and pain point, preferably one who has lived it, or find an easier problem to solve.

joshvm 2 hours ago||
I would also split the concerns:

1. Compliancy with relevant standards. HIPAA, GDPR, ISO, military, legal, etc. Realistically you're going to outsource this or hire someone who knows how to build it, and then you're going to pay an agency to confirm that you're compliant. You also need to consider whether the incumbent solution is a trust-based solution, like the old "nobody gets fired for buying Intel".

2. Domain expertise is always easier if you have a domain expert. Big companies also outsource market research. They'll go to a firm like GLG, pay for some expert's time or commission a survey.

It seems like table stakes to do some basic research on your own to see what software (or solutions) exist and why everyone uses them, and why competitors failed. That should cost you nothing but time, and maybe expense if you buy some software. In a lot of fields even browsing some forums or Reddit is enough. The difference is if you have a working product that's generic enough to be useful to other domains, but you're not sure. Then you might be able to arrange some sort of quid pro quo like a trial where the partner gets to keep some output/analysis, and you get some real-world testing and feedback.

strgcmc 6 hours ago|||
I think it comes down to, having some insight about the customer need and how you would solve it. Having prior experience in the same domain is helpful but is neither a guarantee nor a blocker, towards having a customer insight (lots of people might work in a domain but have no idea how to improve it; alternatively an outsider might see something that the "domain experts" have been overlooking).

I just randomly happened to read about the story of, some surgeons asking a Formula 1 team to help improve its surgical processes, with spectacular results in the long term... The F1 team had zero medical background, but they assessed the surgical processes and found huge issues with communication and lack of clarity, people reaching over each other to get to tools, or too many people jumping to fix something like a hose coming loose (when you just need 1 person to do that 1 thing). F1 teams were very good at designing hyper efficient and reliable processes to get complex pit stops done extremely quickly, and the surgeons benefitted a lot from those process engineering insights, even though it had nothing specifically to do with medical/surgical domain knowledge.

Reference: https://www.thetimes.com/sport/formula-one/article/professor...

Anyways, back to your main question -- I find that it helps to start small... Are you someone who is good at using analogies to explain concepts in one domain, to a layperson outside that domain? Or even better, to use analogies that would help a domain expert from domain A, to instantly recognize an analogous situation or opportunity in domain B (of which they are not an expert)? I personally have found a lot of benefit, from both being naturally curious about learning/teaching through analogies, finding the act of making analogies to be a fun hobby just because, and also honing it professionally to help me be useful in cross-domain contexts. I think you don't need to blow this up in your head as some big grand mystery with some big secret cheat code to unlock how to be a founder in a domain you're not familiar with -- I think you can start very small, and just practice making analogies with your friends or peers, see if you can find fun ways of explaining things across domains with them (either you explain to them with an analogy, or they explain something to you and you try to analogize it from your POV).

jimbokun 6 hours ago||
One approach is to partner with someone who is an expert in that space.
bzmrgonz 4 hours ago||
My thing is, even ingesting the BOK should have been done in phases, to avoid having all your virtual eggs in one basket or nest at any ONE time. Staggering tokens to these compartments would not have cost them anything at all . I always say, whatever convenience you enjoy yourself, will be highly appreciated by bad actors... WHEN, not if.. they get thru.
More comments...