For example, redlining an OEM agreement with my vendor. They pretty much all use the same template, and the objections are always the same. I tried using Claude first, and even my lawyer was impressed that it found things they missed. Then I used Claude to redline that type of contract and the other side said "your lawyer was surprisingly thorough". I had my lawyer do the final check, but saved a bunch of money doing the first three rounds with Claude instead. And my lawyer basically just said "looks good to me" by the time they reviewed it.
Or even writing a simple will or estate document.
It seems like LLMs are great for the kind of generic work you'd usually pawn off to a new associate or paralegal, but not for the important things that partners usually do.
This seems to be the case with most industries that LLMs are impacting, coding, data input, legal etc.
Whilst it's great to get the time saving from not delegating to a junior (LLMs are much faster at the basic level 1 work) there's going to be this big knowledge gap that you really only gain from actually doing the mundane boring shit.
A lot of this _should_ be automated away, but we really need to think about how the next group of people are going to learn the skills and nuance to applying them. I don't really have an answer to that one other than still giving that work to junior staff, but they will then just utilise an LLM to help complete the job, and rightfully so, I'd have done exactly the same thing.
I think OpenAI tried doing this with some early chatgpt settings with "learning mode" or whatever that is. But an LLM just telling you what to do will never teach someone, there was something about having to go out and hunt for the answer through the sea of crap responses or wrong SO questions that really taught you something. How do you replicate that learning with an LLM and how do you apply that to different industries like Law?
That's what school is for. For the last 150 years we've automated a bunch of things, and then we send people to school to learn how those automations work, so they can make better automations.
Maybe law school will have to be a year longer. Or maybe some forward thinking firms will have new associates check the work of the LLM to gain that experience.
Like how people use AI to write their resume and then the HR department uses AI to screen the resumes, the 2 AI's duke it out.
Will they understand the point when football gets taken over by AI, robot players and all?
Or won't they even stop at AI pastors and pray for the second coming of "Super J AI"?
Law school is already 3 years long in the US (after 4 years of undergrad), and most agree the second two are a complete waste of time. Other jurisdictions are different, but the US market is by far the largest.
With strict version history that may be okay, but hopefully their agent already thought of that for them too!
When you automate physical movement, you atrophy. That implies ill health and helplessness/lack of resilience should your automata fail.
When you automate intellectual movement, the consequences are even more dire.
Most people lead a very immobile life already, in both aspects. They don't realize the consequences, if everybody did the same.
Last year when i mentioned this application of LLMs somewhere on Reddit (in the course of arguing that white -collar workers are an endangered species) it was met with scorn by commenters who insisted that machines could never match the skill and care needed to write and review contracts and the like, at least not to the level demanded by the legal profession. Well maybe the human standard isn't that high to begin with; my dad was just telling me last night the story of how his lawyer had urged him to cave to a plaintiff's demands because the judge who was hearing the case "gets bored and doesn't like reading the documents." (The judge's son is currently running for the office of county prosecutor, lol.)
You then get a decision that your AI agent explains to you.
What could possibly go wrong? You will have to ask your agent for that.
On top of that, dealing with text is literally what LLMs are all about.
The problem with accuracy, hallucinations and just generally ensuring responses are grounded in factual stuff is in large part and engineering problem.
So, to me it isn't curious that your experience with Claude has been positive. Provided you know what you're doing and you aren't asking Claude to go too far outside the expected norm, there wouldn't typically be a compelling reason for most contract attorneys to push back.
Kind of like how they can take crazy ideas and dress them in better vocabulary.
Every AI tool they tested completely shit the bed and ended up requiring more time for contract review than a human review, and completely failed to spot any of the issues that were actually relevant to our industry. The AIs were also for some reason really focused on lack of consideration, even though that was not an issue with any of the tested contracts.
This seems to be the common thread everywhere LLMs end up being useful. As the sole software developer? Pure trash, but as "tons of juniors we can shuffle boring tasks at": great. As the sole designer and maintainer of a website? It'll look like 150% slop and no will buy anything, but as a designers friend to create what they dictate: even a designer can throw together a good looking website quickly. Etc etc.
Take, for example, India. The courts are backlogged like crazy. If you're not willing to bribe a judge or the opposite side's lawyer, you'r looking at multiple decades to settle simple cases.
I wish I could waive a wand and have some LLM just go through all of the 58 million pending court cases, identify the clear cut ones and free up the judges' time so they can work on the more difficult ones.
If both parties agree to use the LLM judge and be bound by what it decides, you don't need a real court.
Furthermore, courts are paid for primarily by states. The tech solution would run into the same resource constraint that is already present.
All of this ignores a more fundamental aspect of the legal system, having a chance to have your case heard by a competent authority. Institutional sinews are built by doing the hard things and investing in it.
If I ask it about Irish law is it gonna mix in some US and British law in there because the training data has lots of that? Or hallucinate a mixture?
“When our products are used in ways that can affect someone's health, legal rights, finances, livelihood, or access to essential services, we require there to be a qualified ‘human in the loop’ (that is, someone who has the authority to review and, if necessary, change Claude’s recommendations), and for the individual affected to be told that AI was used.”
Something as common as a family law case has the potential to affect all of the above. Now imagine being a petitioner, respondent, paralegal, or other 3rd party using Claude to review a draft document only to find out that Anthropic has sent your entire file to opposing counsel.
It sounds egregious, but Anthropic is clearly stating they will use their own judgment to control what people can do with its product, and that they will treat customer data as if it's theirs to disseminate. So there's a massive opportunity for an unbiased tool willing to fill this void.
> Anthropic: you can't use our product to fully-autonomously make decisions that affect human health, finances, etc with no human in the loop.
> victor9000: If you review a draft document with claude, anthropic will send your entire file to opposing counsel
What is the connection here? Anthropic has obviously never done anything like this and it's not indicated that they will by the thing you quoted.
If you are ascribing intent ("Anthropic designed a system that intentionally does this") or emergent model intent ("OMG at AGI") then I mostly agree.
But there's a more pedestrian version of this, where an unwise person puts a model in an open loop without sufficient HiL checking and then the model does dumb things. There are even some recent stories about OpenAI and Anthropic systems "emailing the FBI", which is sufficiently close to "emailing opposing counsel" [1][2].
But even if you feel those stories are trolling/marketing/etc., I think it's clear that there's a plausible risk model here. If you hook up an LLM to a toolset that has access to Outlook and your file system and opposing counsel's email address, then of course the dumb thing could happen. The LLM doesn't need to be AGI/ASI; in fact, to me, the worse the model capability the more likely this sort of screw-up feels. There are even (non-sarcastic) instructions or CoT roll-outs I could imagine which might elicit a sequence of tool calls the net effect of which is an errant disclosure.
So this feels less "unforeseeable lightning strike" and more "car accident while driving at speed". I'd put the HF hack in that bucket as well.
A sufficiently careful engineer and a well-trained expert operator can together properly constrain the system to not do the dangerous thing. But, in general, that is not how agentic systems or their underlying models are currently being designed, and most users are not receiving sufficient training.
A lot of these LLM-backed products feel very poorly engineered, rushed, and optimized for brain-feel instead of surfacing and forcing the user to acknowledge the large quantities of actual essential complexity in play. It's all very "unsafe at any speed".
--
[1] https://tech.yahoo.com/ai/chatgpt/articles/chatgpt-allegedly...
[2] https://www.bgr.com/2055251/anthropic-ai-claude-contact-fbi/
Though that doesn't seem like a corporate terms-of-use thing as much as a training issue.
This is actually good policy imo, if you are using AI in a way that can negatively someone's life there should be a human involved.
Haven't they been perfectly clear about that from day one with the ToS? I'm not saying I like it but it hardly seems unreasonable.
> they will treat customer data as if it's theirs to disseminate
Not sure where you're getting that. What you quoted in no way implies that.
No. They did not have triggers that "paused" work from day one and only recently ramped it up to up-sell Mythos and a verification program meaning that in my opinion any code Claude creates is suspect as it can not perform a proper security analysis. I now have to divert half my spend from Anthropic to Grok. I would prefer to not do this. I am not converting a room into a data-center or spending enterprise level funding on my hobbies.
https://www.thomsonreuters.com/en-us/posts/innovation/thomso...
I wouldn't be surprised if that turns out to be their stupidest mistake. I mean, if I can't swear at their code is that b/c they think it's conscious? In which case they're liable for providing me with an employee that makes mistakes rather than just software?
How could we trust legal product from a company who was ignoring privacy protection law like that?
Also, the Grok one made the front page, why am I only hearing it through a minor comment?
I suspect you never heard about it because almost nobody uses ZCode, even among Z.AI subscribers.
Opus 5.5 scores 4% on it.
Imagine a bunch of lawyers arguing what programming is. Even a bunch of programmers arguing what programming is abnoxious at worst and a mental exercise at best.
Programmers typically don’t know law outside of which license allows them to get code for free.
Yeah, and how many of drivers or software engineers can fix their car's engine? Just a few. Even though in priciple there's not much to it.
b) I have met Lawyer colleagues who didnt know how build a =SUM() in Excel
So, I doubt....
That doesn't mean they don't exist. I went to law school with many former software engineers, many of whom are now prosecuting patents on behalf of high tech companies, and some of whom went back to engineering (like me).
"I have never met ______" is of no consequence to anyone; it's an argument style best avoided unless you're talking about deities or space aliens.
Domain knowledge is informative sometimes but mainly overrated: the success of LLMs proves just that. What most lawyers do is fairly standardized and routine, efforts to invoke mystery cases or "novel" is an attempt at resisting the inevitable. The "the only British food in Palo Alto" equivalent of unique skills.
How much of your work is truly novel and unique? Not 100%
An 80% miss rate on a benchmark suggests to me that automation has a very long way to go, even for the drafting and research phases.
EDIT: specifically around the use of judicial precedent in case-law.
90% of legal work where there is a plaintiff and defendant is pressuring a settlement. My wife, an attorney, does not consider that to be "litigation".
But, for your sake, it's probably best that you accept your wife's definition.
As much novelty as there is in human interactions.
Infinite
Makes sense.
Here are 3 examples (there are more): https://github.com/harveyai/harvey-labs/issues/191
I don't know the other fields well enough, but if all your clients end up needlessly detained or admit to things they have a valid defense for, eventually people will notice. It's not exactly the two major sins that will definitely get you disbarred - stealing money and screwing client while representing them - but it's definitely at least incompetent. They seem to not understand that common law jurisdictions are fundamentally different from civil ones, and also, there's quite a bit of difference between how the system evolved in each country. Or that in China what's printed is basically more like a manifesto, since it's almost entirely run by the bureaucracy and personal relationships, and lawyers that aren't foreign-trained and work for foreign firms aren't considered prestigious since the courts are essentially entirely for show.
They need to get like, one jurisdiction, one area in that jurisdiction, and specifically the kind of sub-sections in that area down pat. But they can't train on real material so they will never be able to do so. Their banner testimonials come from non-lawyers and that says it all.
I genuinely appreciate efforts to make access to justice more equitable & cost-effective. And I don’t actually much care if people are using these tools to draft boilerplate contract provisions, create initial document summaries, and other such mundane tasks.
But damn, uncritical reliance on these tools is setting yourself up for potentially serious downstream pain. I’ve yet to see legal automation (including LLMs) that can one-shot situationally appropriate and fully trustworthy outcomes in the absence of experienced human review.
Law is an imperfect effort to codify ever-evolving norms, and interpretation of the law is often more of an art than a science, requiring intuitive leaps and judgements not necessarily suited to statistical prediction machines. And even where the law appears to be reasonably settled, not every legal situation gets resolved in the form of words that somehow end up in a corpus of training data subsequently used to train a model.
In other words: legal LLMs are still very much caveat emptor. Trust if you must, but please, please, please, carefully verify before making potentially deeply consequential decisions based on the output of these tools.
First it is going to suck, like it did with coding but after a while it is going to be good enough for 90% of legal work. And finally when attorneys will feel the competition of AI doing their work for 10 cents of tokebs, they are going to be less arrogant and more motivated to finish a task instead of client babysitting them through the process.
Also, I wonder if this is at all related to the recent reports that Harvey has been getting squeezed on margins by the US labs.
[edit] I'd go one step further too and say the training data should be explicitly requestable too so third parties can verify (freedom of information act styles).
we can start out with the hosted version for evaluation and then self host for production and real customers
"he who will be his own Counsellour, shall be sure to have a Fool for his Client."
How much worse the lawyer who subcontracts Freakazoid?
https://en.wikipedia.org/wiki/Freakazoid!
"Freakazoid has ... access to all of the Internet's knowledge. He cannot fly, but often runs around with his arms extended and making whooshing sounds with his mouth, pretending he is."