Eventually, I hope we'll see many more specialised small models that run nicely on consumer and prosumer hardware. Like LLMs specific to income tax law for UK residents, or LLMs specific to building power delivery circuits for PD USB C devices. There probably is a point where the niche-ness does become to niche to justify the post training, but again, small models means much less training required.

Kind of begs the question if this even needs new/better data? Could you take the foundation model that's trained on everything, and then run a final fine tuning training run on just the set of foundation training data that you want the model to specialise in? Eg, extract just the training data that is about UK income tax law and then run a few epochos on just that? Any ML people know if this is a thing or if not, why it doesnt work?

I think part of the reason there's so much propaganda around chinese tech is to scare you away from possibly using them, not for your own safety, but so Meta/Google/MS ect dont lose access to the ability to spy on every aspect of your life.

I love how we have to pretend like China is some scary boogeyman because people on screens say they are.

I don’t think hallucinations are a major issue with agentic coding a year later? At least, not for me.

Legal writing does have the problem that there is no such thing as a correct argument that a goal-driven loop can target. But basic things like “does this case even exist” seem solvable with research agents, and even harder problems like “does this case support the argument being made” seem conceptually very similar to questions I now ask Claude daily.

If anyone here is looking for engineers to work on legal AI products, both of the students who worked on this project come highly recommended.

But also, interesting how the rate of cases peaked and seems to be going down now.