Posted by aaronbrethorst 1 day ago
If you ask me and a logistics dispatcher the task of building logistics dispatching software (whatever that is), I will get there first.
Knowing the caveats and pitfalls of this through years of (often-painful) experience is what, at least for me, allows me to preempt a lot of the sloppy assumptions or omissions that even the frontier models make when working on systems at scale. This means I can leverage my domain expertise on these high-level areas while delegating the grunt work that is harder to screw up to the agents. I find this enables me to work faster while avoiding the slop making its way into critical engineering decisions.
so it takes a domain expert to remove unnecessary things, similar to how stone carvers create by removing material, not adding
Knowing "too much" and not knowing what belongs to the core and what is a secondary detail is exactly a lack of domain expertise.
Bad luck for the remaining ones.
I'm not sure that even that will remain as valuable or work as a viable moat.
We live in an era of corporate consolidation and absolutely, positively having to meet the revenue target. We also have invested literal trillions of dollars into the AI technologies that made the first skill the author mentions less valuable. However, the result just isn't there. Like the author says, there's a need for domain expertise.
However, you had a bunch of investors plow that trillions of dollars into the current AI boom with the understanding that they could, at the very least, take anyone and have them create what used to take an experienced software engineer, and in far less time and cost, and they invested thinking that the corporate oligopoly would deliver this. They'll now do anything to get that money back. Anything.
If that means telling the corporate oligopoly to tell customers that they need to expect less in the way of domain expertise from the models, well, they'll do that. And since there are relatively few players (the literal meaning of oligopoly) and they all have incestuous financial relationships with each other, they have incentive to hold that line as an entire industry. Development of better tools to create better domain expertise models would take even more money, which the investors don't want to provide, and, short of soaking the public investment markets, can't even find the cash for. Thus, the customer has to lower their expectations if the investors are to not lose their asses on the AI bet.
Something has to break.
But that was never the hard part!
Come now.
After twenty plus years as a professional software developer I can name two hard problems, not more. One is related to the article, the other is not:
1. Getting that clear idea out of a stakeholder's brain. Traditionally this would be a specification but doesn't need to be that formal. Remember, remember the first panel of https://i.redd.it/i2aeyrivmjoz.jpg An LLM doesn't help here because it doesn't push back. It'll do whatever you tell it to do even if it's not what you really wanted. The software developer here operates very similarly as a translator and it always has been true a translator who speaks both sides well will be able to do the highest quality work. This is not at all new. It always has been the advice that if you know things like, say, logistics and software or any such pair then you'll be well off financially either because you can do this translation well or because you realize what's missing and can do a product for it.
2. The other problem, of course, is debugging. Since LLMs fundamentally work from a training set any debugging problem not blatantly obvious to a sr developer is hopeless for them.
They really do want to know the ins-and-outs of the HVAC service business, for example, because they hope their agents will be handling it in a few years.
The problem is that more and more people are getting convinced by the AI's that they're domain experts when they're really not.
I mean, they could ask an LLM "what does this code do, and will it always X when Y", but that's just nesting the verification problem inside another verification problem.