Top
Best
New

Posted by aaronbrethorst 21 hours ago

Domain expertise has always been the real moat(www.brethorsting.com)
781 points | 489 comments
hn_throwaway_99 11 hours ago|
While I agree that domain expertise has always been a moat, I believe the author is missing something critical: there is a big difference between being able to verify the output of a system is correct, and being able to tell a system how to generate the correct output to begin with.

Personal example: I had a software engineering colleague who was the best coder of financial management systems I've ever encountered. He gained these skills through years of in-the-trenches development. One of the things he told me, and that I also observed, was that the vast majority of financial experts (basically, the people in the accounting department of companies) had an extremely difficult time just telling him what the rules of any particular transaction should be. But what they could do was tell him whether the handling of any particular transaction was right or wrong. So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec.

In my experience, that is the primary difference between people I've known who are good software engineers and those who aren't: people who can specify the detailed rules of any system, vs. folks who take a "well, I know it when I see it" approach.

I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years. As an analogy, it's kind of like asking a native speaker for the grammar rules of their language. Often times they can't, but they'll just say "well, that sounds wrong." They may be "domain experts" in their language, but they'd have a hell of a time prompting an AI system on how to grade a test for grammar correctness.

xorcist 7 hours ago||
Anybody who has ever done programming professionally in the small scale knows this. Refining the requirements is the job.

In fact, I've never known an industry so keen on levelling its own moats as the software industry. We regularly invent things like 4GL, graphical programming and frameworks and engines such as Unity just to enable more people to do programming. People will happily teach programming for free in outreach programs (I am just one example). No other profession does this. Perhaps with the exception of mathematicians and other fields that are very close to programming.

I could teach an economist enough programming in a weeks that they could write an ERP module, but I could not learn enough economics in a week to write one. If the language was the barrier, we could invent a more effective one. We invent new languages weekly anyway. Having seen how quickly beginners can make things with Unity or Godot, I seriously doubt an LLM agent could improve much on this. Of course, if the job is writing yet another CRUD React app with a Java or Python backend, then sure, the LLM will be very effective. But compared to doing same app in something like Excel or MS Access? Not that much.

4er_transform 4 hours ago|||
Everything everywhere does this. All human progress has been making common and accessible the rare and expensive. Universities, online courses, textbooks, etc all exist to make economics as accessible as possible, there’s just a really tight reward loop with programming so your rate of learning it is more tangible
DanielHB 4 hours ago||||
I think you are underestimating how hard it is for average joe to learn programming basics. I remember a fellow in high school that just could not accept that = in programming is assignment not an equation (like in high school math)
xorcist 3 hours ago|||
I'd like to believe I have an inkling, having done a fair bit of teaching.

Still, imagine how hard other skills are to acquire. How much civil engineering can you learn in two weeks? How much violin playing? But you could absolutely get basic grasps on a general purpose programming language. With something specialized like Unity or Excel you would get tons of useful output.

The hurdle around assignment operator is as old as symbolic languages. That's why legends such as Niklaus Wirth wanted to use another operator, ":=", a notation that is still being used.

Anyway, it can be a hurdle, but one I find that most people get over pretty quickly.

zahlman 2 hours ago||
> The hurdle around assignment operator is as old as symbolic languages. That's why legends such as Niklaus Wirth wanted to use another operator, ":=", a notation that is still being used.

Well, sure; but more generally, the ability to accept other meanings for symbols (and keep subtly different symbols straight in one's head) is a mental skill, and individuals vary in their aptitude for it. (Presumably, this one is also relevant to natural language learning, since one must reckon e.g. with false cognates.)

rightbyte 51 minutes ago||
Nah I just think you are so overwhelmed that "=" having a different meaning lh and rh is just too much. It is hard to remember how hard it was.
repstosb 2 hours ago||||
You went to high school with Niklaus Wirth?
dapperdrake 1 hour ago|||
This is one of the few mistakes K&R made going from BCPL to C.
teleforce 7 hours ago||||
>We regularly invent things like 4GL, graphical programming and frameworks and engines such as Unity just to enable more people to do programming.

You're right on the money on this.

Earlier this month I went to visit a company for a complete demo prototype of a full one-to-one train simulator trainer mostly designed and programmed by a former civil engineer using Unity engine. According to the company, they could not do it if Unity engine (or similar) is not around because it will be prohibitively expensive to develop.

In a related news, Unity recently released AI eco-system namely Unity AI Suite [1].

[1] Unity AI Suite:

https://unity.com/features/ai

dzonga 5 hours ago|||
the 1 dimensional thinking that's perverse in the industry will be catastrophic.

we don't like doing the hard things e.g training juniors so they can be skilled seniors via good apprenticeship programs i.e on the job. now we r delegating to stochastic parrots.

in terms of systems thinking which is one skill you need to be a domain expertise - very few people are ever curious & are not willing or able to ask critical questions. hence the groupthink that's prevalent in the industry.

no wonder the quality of software never goes up - while the building blocks have gone up in quality. an analogy is like having super strong bricks but making brittle structures

abustamam 4 hours ago||
I use LLMs daily for coding. That said, I agree with your sentiment. Super strong bricks, poorly put together, makes a bad building.

Another analogy I like is a beginner playing a $100k Stratavarius probably can't produce anything near a professional violinist playing a $50 violin.

Personally I use LLMs to level up my systems thinking. I describe the domain, I have it brainstorm some scalable solutions, I look them up, I bring them up to the team, and we discuss, and I implement. It's a great workflow imo.

vmurthy 11 hours ago|||
>One of the things he told me, and that I also observed, was that the vast majority of financial experts (basically, the people in the accounting department of companies) had an extremely difficult time just telling him what the rules of any particular transaction should be

We have internalized more knowledge than we can explain sounds like the textbook definition of Polanyi's paradox:"Polanyi's paradox, named in honour of the British-Hungarian philosopher Michael Polanyi, is the theory that human knowledge of how the world functions and of our own capability are, to a large extent, beyond our explicit understanding" [0]

[0] https://en.wikipedia.org/wiki/Polanyi%27s_paradox

hn_throwaway_99 10 hours ago|||
Thanks so much! I hadn't heard of Polanyi's paradox before but it is exactly what I was talking about. The Wikipedia entry even highlights the problem for AI-driven development:

> Polanyi's paradox has been widely considered to identify a major obstacle in the fields of AI and automation, since programming an automated task or system is difficult unless a complete and fully specific description of the procedure is available.

reemyoass 2 hours ago||||
having worked on automation of accounting process, it has been a combination of ever evolving ruleset/edge scenarios for 10% cases plus standardised rules for 90% of the cases. i think the problem is that it is an evolving field. alot of times it borders law, which makes it murky and vague too.
cindyllm 1 hour ago||
[dead]
formerly_proven 8 hours ago|||
cf https://en.wikipedia.org/wiki/Dual_process_theory as a causative agent behind the paradox.
theptip 3 hours ago|||
I think this neatly captures the gap in the article; it’s thinking of domain expertise as static, rather than something that needs to be built/extracted.

It also goes the other way; A a good engineer can build experiments and discover the domain even without an oracle on the team; reading protocols or manuals or specs, for example. In other words, learn to be a domain expert themselves.

Personally I find the most efficient way to learn a concept to be building it; you are immediately faced with your blind spots. AI for sure helps improve cycle time on these discoveries if you use it that way; time writing boilerplate goes to ~zero and you can spend your time figuring out how to answer the real questions.

I do think that we as a profession need to learn new architectural and craftsmanship patterns to make it easier to learn from our code; concepts like Literate Programming which were too fussy for fast-moving teams may end up accelerating in the agentic world; I need to read a lot more code than I write now. But also things like Acceptance Test Driven Design; not new concepts, just newly increased in value.

michaelbuckbee 6 hours ago|||
WRT the native grammar, consider adjective order. Few native english speakers (me included) can off the cuff name the proper order, but everyone knows the "right" order.

https://dictionary.cambridge.org/us/grammar/british-grammar/...

mejutoco 4 hours ago|||
Another one: tekamolo, for German and adverbs this time.

Temporal, kausal, Modal, Lokal

https://www.olesentuition.co.uk/single-post/tekamolo-how-to-...

dapperdrake 1 hour ago||||
Excellent example.
chrisweekly 5 hours ago|||
Great example
derfurth 9 hours ago|||
Having worked with domain experts I concur on the difficult time they have expressing the rules of their domains.

Once I built a little domain-specific language for them, that was tested against old jobs to see if they contradicted the past; it was a nifty project and since then I am convinced that DSLs are underrated as a way to encode expertise.

Dansvidania 9 hours ago|||
Can you give an example (even if contrived) of how that would have looked ? I’m very curious !
derfurth 8 hours ago|||
I had the experts write markdown files that contained the rules looked somewhat like:

## 1A Rule name

Some prose explaining the rules liking to official documentation.

``` if municipality and inhabitants > 10000 then functionA else functionB ```

Then a trivial parser would extract the rules, the DSL was then handled by Lark[1]. So pretty simple, but it made collaborating with experts easier as simulated results would also output some markdown they could read.

1. https://lark-parser.readthedocs.io/en/stable/

tardedmeme 7 hours ago||
Sounds like "Literate Programming", where the code and comments are reversed: instead of everything being code except what's marked as a comment, everything is a comment except what's marked as code.
BarkenBark 8 hours ago|||
I am also very interested!
renegade-otter 8 hours ago|||
Scala to the rescue, then! :)
hliyan 5 hours ago|||
I think the answer is in the second sentence of the article. The most valuable person in this new world (using the author's own phrasing from the penultimate paragraph) is the one who is good at "building a working model of the domain in [their] head".

Depending on the domain, this may either be the domain expert themselves, or someone else trained in formal logic, data structures and organizing information into coherent hierarchies. If this is neither the domain expert nor the programmer, then it must be someone in between. In the old world, this role was called "business analyst".

dapperdrake 1 hour ago|||
Working model suitable for an electronic computer.
Mezzie 3 hours ago|||
I'm this person, and I do find AI to be quite helpful, though I'm mostly just playing around at this point.

I'm the daughter and granddaughter of programmers, and I learned the basics of how to code as a kid. I'm good at it and have a knack for it, but I didn't want to do it for 8+ hours a day and then spend my nights on it as well, so I didn't pursue it as a career. I did an undergraduate degree in Linguistics, which has been really helpful for having an intuitive sense for what 'language as data' can accomplish and for a strong understanding of the difference between language as data and language as meaning. I studied formal logic systems. Then I did a graduate degree in Library Science and worked in libraries for a decade and a half.

I can organize and define systems very well, and I'm trained in how to wheedle information people don't consciously know out of them without them knowing I'm doing it. I've spent enough time around actual devs to understand where my limitations are and when to loop in someone who knows more to check my work, and when it's important for the work to be super accurate versus when I can learn by fucking around. (Front end and design? Fuck around! Database structure? Fuck around but with an exceptionally robust backup system kept outside of the AI tools' purview + don't fuck around in prod. Storing credentials and people's information? Ask someone.)

The problem companies are going to have is I'm very disinclined to work for them doing this, particularly if they want us because they think we're going to be cheaper. Most people who are in this category a.) could be devs and opted not to, and there's a reason for that and/or b.) are the children, cousins, etc. of programmers. We're not stupid: we know we're just as disposable as they're trying to make devs.

the-clonenator 51 minutes ago|||
> which has been really helpful for having an intuitive sense for what 'language as data' can accomplish and for a strong understanding of the difference between language as data and language as meaning.

100%. I think, intuitively, software developers understand that there's a strong connection here, but most fail to translate that into practice. It always amuses me when someone comments on the triviality of creating CRUD apps. Setting aside the fact that people usually get the mechanics of it wrong (despite it being a solved problem), they overlook the difficulty of producing a good information design.I've developed software in a variety of industries, and I would say maybe 5% of the designs I inherit are well-done and represent the concepts they're trying to model in an elegant, parsimonious way. Rather, most examples are replete with ambiguity, orthogonal concepts smashed together into single elements, and misleading naming and relationships.

Mezzie 41 minutes ago||
This is so true.

And even when there is a well laid out information design, it often is laid out from the wrong point of view. Elegant information design should create a pattern subconsciously enabling people to build a mental model of the tool they're using without realizing they're doing it. But software whose information design works wonderfully from the point of view of a SDE is not going to be approachable to the average user. There are so many tools I've used where I can very clearly see the reasoning behind the decisions and the design and use it well and also be aware I could never explain how they work to the average person.

abalashov 2 hours ago|||
Having descended from a humanities social background and blundered into professional programming rather incidentally, a lot of this resonates with me.

I've frequently been credited as a person who can really string all the disparate elements of tacit knowledge together into a unified fabric in our particular subdomain, and helped a lot of people plug Swiss cheese gaps in their knowledge that way and come away with the feeling that it's all been tied together theoretically.

However, it's not immediately obvious to me how, in our LLM psychosis cultural moment, this facility shoots to the top of the value chain.

Mezzie 1 hour ago||
In theory, it's because we're going to be better at steering an LLM in some ways. A lot of the friction and hold up in building comes in the communication between people/departments/organization. If you eliminate that by holding the knowledge in one person/department, you see an efficiency gain.

What they're not saying is that they think we're more valuable because they think we'll be cheaper. They think they can have us do 2 jobs and pay us for 1: probably less than a decent SDE made in 2019.

Personally, I think it's likely to be a shitshow and backfire if that's how companies decide to try to go that route (especially the largest ones). First, if we wanted to be devs, we would be (like you). Most people with the knack for programming and thinking in systems know they have that knack, and if they haven't jumped ship to SDE before now, there's a reason. I could definitely hack a junior SDE role, skill wise, but I don't want to. Second, finding people like us is difficult. The hiring process (which is becoming more and more Gilliamesque by the day) is really bad at identifying us. There aren't credentials, and this sort of work tends to reside in the gaps, as you identified. It's harder for it to show up on a resume. Hiring is optimizing for exact matches and experience, and that's the opposite of how this skill set actually functions. I've found these skills are best developed by being placed in a room where you know very little about what's going on and forcing you to develop heuristics and approaches over time for getting that context. Thirdly, I can't speak for you, but I've developed this perspective over decades and if people want it, they're going to pay appropriately. If they think I'm going to do any of this at my current salary level, they're deranged. And lastly, while most people in our position might roll our eyes at some techie discussions and culture, we do fundamentally like techies/devs and we tend towards placing a greater value on things like relationships than a pure SDE does. (Just speaking in generalities). So 'is willing to replace and/or toss out a category of people I like and respect' is a hint to us to start out assuming this is a hostile negotiation. (Whereas SDEs as a cohort over the last 20 years extended a lot of goodwill at first). We're far more likely to work somewhere, get enough domain knowledge, and then bounce to start our own thing, especially since as a population we're more likely to have devs who will work with us as non-technical founders. Someone who's decent at marketing/sales/the stupid 'people stuff', understands a domain, and understands when a proper dev tells them what is and isn't possible and can even help with some of the most boring, rote parts of the technical side if needed/in crunch is an excellent non-technical founder, and as a group we're also more likely to have access to the technical connections that we'd need if we wanted to build something beyond our ability.

abalashov 39 minutes ago||
This is an underappreciated bit of insight and perspective, and I thank you for sharing it.

Thoughts that really stood out for congruence with my own experience:

> Most people with the knack for programming and thinking in systems know they have that knack

> Hiring is optimizing for exact matches and experience, and that's the opposite of how this skill set actually functions.

> I've found these skills are best developed by being placed in a room where you know very little about what's going on and forcing you to develop heuristics and approaches over time for getting that context.

> We're far more likely to work somewhere, get enough domain knowledge, and then bounce to start our own thing

Toutouxc 11 hours ago|||
I agree with your point that people “from” the domain aren’t automatically equipped to start AI-building software in the domain, exactly because they often lack one of the most crucial software development skills — being able to (and having the desire to) describe a complex system with a finite set of deterministic rules.

But I don’t think we should be calling these people “domain experts”. I think we should reserve that name for the other group, for the people who truly and deeply understand the domain, the whys and whats and why nots.

RaftPeople 1 hour ago|||
> But I don’t think we should be calling these people “domain experts”. I think we should reserve that name for the other group, for the people who truly and deeply understand the domain, the whys and whats and why nots.

I've spent a lot of time in my career extracting info from the business and I think most do understand the whys/why nots but aren't practiced at organizing all of those decades of experience into a higher level abstract model that can easily be communicated.

It's typically layers and layers of information with dependencies in many directions littered with exceptions. Just like our software design+dev experience, it takes a lot of practice to try to organize all of that info into a coherent presentable model.

RussianCow 11 hours ago||||
That ship has sailed. Anyone who works in a field is now considered a domain expert (or "SME") even if they're nothing of the sort. There should ideally be another term that's a superset, but I doubt it would ever catch on.
eloisant 11 hours ago||||
You're doing a bit of a "true Scotsman fallacy" here.
Toutouxc 9 hours ago||
It feels like without a bit of true Scotsmanship, the term “domain expert” doesn’t really mean anything.
spacebacon 7 hours ago||
Sure it does, it’s just a highly contested meaning that depends on a proper contemporary interpretation if to be received as intended in this context. Context that varies domain to domain. For the domain name buyer it’s quite different than the metacognitive scholar.
dapperdrake 1 hour ago||||
Then there are almost no domain experts.
xnx 7 hours ago||||
Agree. If you can't explain it to someone else, you don't really understand it yourself. These people are practitioners not experts.
watwut 7 hours ago|||
People you say are not experts are fully capable to explain whys and whats and why nots. In more details that you can absorb.

They will do that on examples tho. They will recognize and explain every single exception they see - but they are not able to list all exceptions up front. Because, that is highly unusual task

throwaway2037 11 hours ago|||
I think that you raise some excellent points here. I want to dive deeper into what is meant by domain knowledge -- as a I see it. This phrase comes to mind: "Those who know, do. Those that understand, teach." (Google search tells me that it is frequently misattributed to Aristotle.) People who cannot clearly articulate rules for their domain only "do". However, those that can clearly articulate will "teach". Thus, I would say true domain knowledge is the ability to teach that domain to others.
layer8 3 hours ago||
I largely agree, but want to note that there’s the counterpoint “Those who can, do; those who can’t, teach” (from George Bernard Shaw’s 1903 play Man and Superman), which also has some nugget of truth in it.
rightbyte 3 minutes ago||
Case on point is Youtube where a lot of almost good amateurs give a lot of advice on niche channels on things they hardly master.
andsoitis 6 hours ago|||
> So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec.

Right, so the spec is derived from examples, an interactive process that doesn’t require a programmer.

layer8 3 hours ago|||
Interpolating between the examples still requires understanding of the domain for the interpolation to be a sensible one. And it’s an iterative feedback process: thinking about possible interpolations leads you to cases that need clarifications from the domain experts. The domain experts won’t come up with all relevant examples by themselves, and they typically aren’t as good at thinking about interpolations like a developer who is trained to always consider all possible cases; they don’t think in terms of models the way a developer does.
andsoitis 2 hours ago||
LLMs already have interactive discussions with me on the topics I engage with, including asking expansive questions, so I do not think this is beyond the realm of the technology.
dapperdrake 1 hour ago||||
Talking to CPAs never gets you to Pacioli groups.

Talking to engineers and mathematicians gets you there.

skydhash 4 hours ago|||
The programmer skill is how to abstract the specs from all the examples. And then to formalize it. Actual coding is merely translation. And only beginners tend to focus on that.
petra 4 hours ago||
Sounds very similar to what AI is good at.
wtetzner 1 hour ago|||
I've yet to see AI be good at extracting good abstractions. More often than not it jumps to creating a battery of special case checks.
dapperdrake 1 hour ago||||
No. AI sucks at formal languages and implications.

Lossy decompression probably won’t ever be really good at this. If it’s missing from the data, then it won’t be there.

The AI math proofs were probably already out there in tiny pieces and nobody got all of the pieces together. That is valuable. It’s different from making a piece that is missing. And the AI will just hallucinate.

skydhash 3 hours ago|||
May work for simple cases, but how would you come up with something like typst, or thunderbird?
nonethewiser 3 hours ago|||
> But what they could do was tell him whether the handling of any particular transaction was right or wrong. So often times he would sit down with these accounting folks and go through lots of example transactions he came up with, and from there he essentially built up the requirements spec.

This combination of requirements is the business in most domains. Software or otherwise. Codefying different rules and executing them.

physicsguy 3 hours ago|||
I have found this with (mechanical) engineers. They know what they want to see but don’t understand the underlying details which are more mathematical than the average engineer is able to work with. So the people working on engineering software are often physicists or applied mathematicians.
dapperdrake 1 hour ago||
Ran into this, as well.

And they were really annoyed at being asked math questions.

elendilm 11 hours ago|||
Yes. The author stumbled on the p vs np problem but with humans. Verification is easy. Solving is the millennium prize question.
__MatrixMan__ 3 hours ago|||
I agree that it has some resemblance, but the striking thing about NP-complete problems is that an efficient solution to any of them is an efficient solution to all of them, which makes it worth trying exceptionally hard to find one.

It could be that whatever lackluster expertise you can squeeze out of an LLM is good enough to discourage investment in the real thing since unlike NP-complete problems, expertise isn't generalizable.

Npovview 5 hours ago|||
Agreed. But there is a new dynamic coming to place. Who can execute faster now that we have AIs? I think the combination of (Good SWE using AIs) collaborating with (Domain Expert using AIs) is the winning team. Things are accelerating on all fronts, the frontier may be jagged but AIs is making progress on all capabilites.
dapperdrake 1 hour ago||
Additional iterations are valuable, if and only if we learn from additional iterations. Otherwise we are just spinning our wheels.
marbro 10 hours ago|||
This is the exasperating part about learning to speak Spanish using a textbook; you must guess the grammar rules because the textbook won't tell you. So, you use the English rule and hope and pray that it is the same in Spanish, and you'll be right the majority of the time but often wrong. Spanish textbooks written 100 years ago tell you the grammar rules and are more useful than recent textbooks.
LEDThereBeLight 1 hour ago|||
I’ve noticed this too. Any idea why modern language books get this so wrong?
layer8 3 hours ago|||
While interesting, I rarely found knowing grammar rules beyond some very basic ones to help all that much for learning to speak a language, compared to language exposure and speaking practice.
hbwang2076 3 hours ago|||
Domain expertise matters more than most bootcamp grads want to hear. Knowing how to phrase a query for a specific domain is half the battle.
dapperdrake 1 hour ago|||
That is why coders who learn problem domains are where the money is at.

Works for me.

Already learned two or three problem domains well enough and now it is starting to compound.

miki123211 7 hours ago|||
> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years.

This is the part I disagree with.

In a non-agentic world, the expert and the programmer are two different people. If the expert finds a bug in the software, they have to describe that bug, send it over to a programmer to fix, wait for a new release and until that scenario occurs again, realize that their description was actually wrong, send a corrected description, rinse and repeat for a few iterations until the bug is finally fixed for good.

In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly, and that's it. Bug fixed; with 10 minutes of work instead of a few weeks.

layer8 3 hours ago|||
> realizes that the fix is incorrect because of sloppy thinking

If the domain expert doesn’t understand the generated code, they can only discover incorrect logic by specific examples (specific inputs), which is usually impossible to do exhaustively. The programmer, on the other hand, can see incorrect logic directly, generalized over all possible inputs and states. It’s the difference between testing and proving correctness.

Hendrikto 7 hours ago|||
> In a world of agents, the expert finds the bug, asks the agent to fix it, realizes that the fix is incorrect because of sloppy thinking, does a few iterations until the feature works correctly

That would require clearly (a) knowing what you want, and (b) expressing it unambiguously and in detail, including all edge cases. Essentially producing a spec.

Most people are not able to do either. Talking to an LLM does not change that.

j16sdiz 6 hours ago||
but.. they can iterate real quick!

As long as the expert don't run of patience, he may be able to do that.

Consider finance sector -- the industry is powered by excel spreadsheets made by non-programmer having no clue how programming works.

skydhash 4 hours ago||
Spreadsheet is a DSL for finance and a lot of other sectors. But the lack of formal logic capabilities shows when you seen the spreadsheet organization. Most only stand with ducktape all over the place.
codesnik 8 hours ago|||
I had the same problem in air travel industry. Our ticketing officers were never able to communicate how the ticket commission rules from our partners should be interpreted. I ended up adding a test framework, that allowed to add examples and counterexamples of existing flights to each rule, to verify that they are applied the way they'd assign commission by hand.
zipy124 8 hours ago|||
I fully agree here. It's also not uncommon to have the reply for specific items be "Ahh, well that is a bit of an edge case see because of X and Y" and then you learn there are many many "special cases" to the point that they aren't that special anymore.
RaftPeople 1 hour ago||
This comment brings back memories of a project.

We were working through many complex business flows and running into exactly what you describe in many areas. Some of the project people with less experience complained that we were spending too much time on exceptions and slowing down the project. We had to explain that when each exception happens 100 times a day with significant impact on business productivity, then it doesn't matter whether you call it an exception or not, it's important to solve for.

thibran 10 hours ago|||
You described reasoning by analogy, which two thirds of the population does.
gota 3 hours ago||
Please explain explicitly what you are implying, because I don't get it

Who are the 1/3 of the population that does not reason by analogy?

thibran 1 hour ago||
Instead of needing a familiar metaphor to understand a new system (reasoning by analogy), they grasp the core conceptual mechanics directly.

It’s not that the 2/3 can't think abstractly, it’s just that they use analogies as a temporary processing step to get there, while the 1/3 starts there.

yoda7marinated 7 hours ago|||
This is basically the gap BDD tries to close I guess. It assumes that domain experts can't fully articulate the rules upfront.
baxtr 11 hours ago|||
Some companies use product managers for this purpose. They translate the implicit domain knowledge of those experts into explicit requirements, mostly through interviewing.
boringperson 11 hours ago|||
> I have a strong suspicion that folks who have a high degree of domain expertise in a particular area will fail as software builders even in an agentic world because they will struggle to elucidate clearly the rules in their head that they've learned over years.

Maybe they need a build a hybrid expertise of "domain" and "software engineering". For example, robotic surgery requires expert surgeons to build sufficient expertise in robotics

Also, noticed a pretty high karma for a throwaway account.

LorenPechtel 1 hour ago|||
I very much agree here--the true job of us programmers is determining exactly what the problem is. The domain expert will fail because they haven't attempted to refine the details enough, they probably don't have the skill to anticipate the edge cases.

I've spent most of my career working with the same people--even people who have some coding experience but they never catch the low probability stuff. Or even not so low probability: business procedures were that the customer was quoted an all-inclusive price. Try to work backwards, what is the pre-tax amount? Um, that's not guaranteed to have a solution. My concerns get ignored, I go ahead and implement with code that will catch the offending penny and label it roundoff correction. While I'm not sure what happened I think some auditor hit that. I ended up having to walk them through the calculation, okay, what's the answer here? Only when they couldn't solve the problem could they accept that what I had done was the only answer. (The actual odds of such a failure are equal to the tax rate. I had not originally worked that out, just saw success was not guaranteed.)

The domain experts make this sort of mistake routinely in dealing with code. AI won't fix that.

xnx 7 hours ago|||
Are we thinking about the wrong side here? Wouldn't the LLM be a super-intelligent financial expert as well as a super-intelligent coder?
Hendrikto 7 hours ago||
It is neither.
starfallg 11 hours ago|||
I guess the flip side to that is the iterative conversation to develop these specifications is exactly what AI chatbots are good at.

It's also the main reason why very structured AI agent orchestration for software engineering modelled on rigid processes fails to really provide much value.

layer8 3 hours ago||
I suspect that the reason AI chatbots are not very good at developing such specifications is that they don’t build a domain model in their “head” — the only models they have are already hardcoded in their weights. A human learning a domain, on the other hand, forms new neural connections representing the model of the domain that they build in their head.
vale95ntino 1 hour ago||
I find the article interesting and (largely) correct in its analysis of how an individual can use AI. What I think is missing is that organizations (like enterprises, research labs, etc), have multiple people and multiple experts in different fields. So you can have BOTH the AI engineer and the domain expert in the field. Not only can you have this but this is actually common in most large companies. The difficulty lies in making the collaboration work and the information flow correctly so that the domain expertise of the expert can flow into the software product built by the engineer.

A mental model I use for this is that we have users, builders and experts when creating AI products. For most individual use cases a person is the user, builder and expert: I make a prompt about how to write code in a way that I like that I will later use. Coding agents moved into the direction of builders that were also experts in the field iterating on a product for third-party users. The next frontier will be finding the right patters for teams to capture expert knowledge handle the collaboration between engineer/builders and experts. Just having PMs handle that interaction will be a super bottleneck.

My cofounder and I are actually working on a project in that direction (https://www.getvalmar.com) --> we'd love any input on how engineers prefer performing feedback loops and getting input from subject matter experts :)

nickjj 1 hour ago||
It's funny how out of touch AI is with seeing the big picture of problems.

Here's an example I encountered last week:

Someone in my neighborhood is a 75 year old chemical engineer who likes computers and got into Linux a few months ago. I see him from time to time when walking around, he's a nice guy and overall has a scientist's mindset. He doesn't make a lot of assumptions and tries to think things through but sometimes he has big blind spots in unfamiliar fields.

I helped him install Linux and also hook up an SSD to one of his older machines and now it flies.

On his own he had an old sound card that he wanted to use on that machine. He asked me if the card is compatible. I told him it almost certainly is because the Linux kernel has drivers for a ton of devices. His motherboard's built-in sound card was fine but he likes tinkering with audio in general.

He managed to physically install the card correctly but called me and said there's no sound playing. Then he says he spent 12 hours troubleshooting the issue, using ChatGPT and Googling for assistance.

Over the phone he told me he tried many different things. Installing, tweaking and configuration ALSA, PulseAudio and PipeWire related tools and tweaking everything you can imagine. Nothing worked, no matter what happened, it never played sound through his speakers.

He asked me if I could come over to help so I did.

In 30 seconds I solved the problem.

I went to his sound settings in his desktop environment and saw that his sound card was being picked up. I looked at the back of his machine and noticed this card had 2 black ports with the bigger style jack for headphones. There was no usual green port which is usually used for output. I shined a flashlight to look closer. One of the black ports was labeled headphones and the other was unlabeled. His was connected to the unlabeled one. I swapped it to the other port and everything worked right away.

All of that to say, as a software engineer I have second hand embarrassment that a trillion dollars invested into AI didn't think to respond with "did you double check to see which port you connected the speakers to?". I asked him if AI ever suggested that and he said no, it immediately went into polluting his system with a bunch of unnecessary tools and chasing incorrect rabbit hole after rabbit hole. AI understands nothing.

praash 1 hour ago|
It's funny how well this reflects the contrast in internet advice between Windows and Linux issues. All users deserve advice beginning with thorough sanity-checks and potential quick-fixes before having to dig deeper.

Searching about common Windows issues results in misleading blogspam. Suggested "solutions" resemble blindly applied folk remedies.

I'm no stranger to breaking my desktop Linux after an hour of misdirected troubleshooting and desperately messing with core libraries. I'm still glad I can quickly find my way to ArchWiki.

steve_adams_86 20 hours ago||
Such a good example of this I encountered recently:

I was on a fishing trip. I asked the charter if he’d want to check out a free app I work on (https://oceanconnect.ca) in case it might be useful for his work.

I don’t know how people on the ocean use ocean data. I don’t really know what they want to know, or why. I wasn’t totally prepared for the incredible onslaught of questions and information pertaining to how people use the data or what we can do with the data, and it was so cool and exciting to get that perspective.

It was a good reminder that models are not the same as the systems they abstract, and knowledge to develop them has almost nothing to do with using them. This guy was a wealth of knowledge about how they use weather data on the water. In a sense, he knows more about the data than I do (even if he doesn’t realize it, or doesn’t understand it in its digital representation), and would be far better equipped to make a useful application for people like him if he could program.

I found myself thinking people like him could actually do amazing stuff with LLMs if they sat down and got their ideas out on a screen. I’d really like to interview people on the water daily some day to refine the product if we ever have the funding. That domain knowledge is highly, highly specialized and people know things you’d never guess after living in a complex domain for decades.

RussianCow 10 hours ago||
Product management is its own skill, and few true domain expects have it. Without some form of PM, the resulting software will end up a mess due to poor UX, too much bloat, etc.

I think AI is going to force most software engineers to pick up this skill in some form. Building is easy; knowing what to build is the hard part.

enraged_camel 6 hours ago||
>> I found myself thinking people like him could actually do amazing stuff with LLMs if they sat down and got their ideas out on a screen.

They can, and they are. You just don’t hear about it on places like HN because those people are not on this website. Which is why some people here make smug statements like “if LLMs are so good at programming how come we haven’t seen any useful apps made with LLMs???”

elendilm 12 hours ago||
"So the most valuable person in this new world is the one who has both skills because they can verify at both layers. They know the generated code is sound and they know the answers it produces are true."

This has always been true since the dawn of programming.

ozim 9 hours ago||
Cannot agree.

Whole TFA doesn’t take into account reason why software development was actually so valuable.

Single specialist in any domain is not that valuable. You may charge $200 for hour of his time sure. But to grow company you now need N specialists.

What software was doing was not making specialist obsolete replacing a specialist by encoding his knowledge - well many tried to do so but failed even in 80’s “expert systems”.

What software was doing was making it possible to structure specialist work, make it possible for a single specialist to serve more customers at the same time, make it possible to hand over work in a structured way to junior specialists, making it easier for senior to take over edge cases and spot check work of those junior specialists.

This setup allows company to not being tied to number of specialists to grow, this setup allows company to charge less per customer but take over more of the market share.

Whole premise that now each specialist will waste time dabbling in AI software development is ludicrous, especially if each specialist would be building his own tooling somehow.

whatisthiseven 3 hours ago||
That sounds more like the extractive setup of corporations like IBM, where return/specialist must be maximized against the number of clients that can be juggled simultaneously.

Whereas in larger technology firms, yes that happens to some degree, but only with the highest level specialists such as PEs, fellows, etc.

They are already so wildly profitable and valuable by the simple nature of computing itself: it scales second only to money with regards to compounding effects. Once you have software someone can use, it scales near infinitely to more users, thus value is extracted from the work of a single specialist for all time. No need to complicate it with trying to abstract work juniors can do (and fail) and then have seniors correct. What would they be doing in non-extractive firms?

alephnerd 11 hours ago||
Yes, but most people (especially a large portion on HN and Reddit) do not internalize it.

A SWE who has always worked in DevTooling companies will always be preferred by DevTooling companies over a generalist. A SWE who has always worked in AdTech will always be preferred by AdTech companies over a generalist. etc etc.

Software fundamentals - though useful - are table stakes skills at this point. No business wants to deal with the headache of on-ramping employees who have never worked in a specific domain or industry because it takes too long for a generalist employee to build the intuition needed to understand that segment of the industry.

59nadir 9 hours ago|||
> Software fundamentals - though useful - are table stakes skills at this point.

I'm having a difficult time even seeing what we're talking about here. I see "seniors" in our industry that don't know what I would call the fundamentals of programming or software development; apparently not all fundamentals are created equal.

chipsrafferty 11 hours ago||||
And therefore there will be a huge knowledge gap as companies refuse to hire anyone who hasn't worked in the field for 5+ years and people who want to work in that field but haven't don't get hired.
alephnerd 11 hours ago||
Not really. Most people continue to remain in a specific domain from their internship days, and professional networks develop.

Historically, startups were the traditional path for a generalist to build domain expertise because most startups couldn't be picky with talent, but the market has changed.

In all honesty, too much fat did develop in the tech industry over the last 6 years. Traditional hiring pipelines (eg. Limiting early career recruiting to grads from top 10-20 CS/ECE/EECS programs nationally along with Vets and some grads from decent regional programs) still net good calibre talent worth their weight in gold, but others just aren't working out.

adaml_623 11 hours ago||
I'm afraid you appear to be contradicting yourself by saying internship in one comment and stating that companies don't bother with onboarding employees with no knowledge in a previous comment.
alephnerd 11 hours ago||
An internship is fine for onboarding becuase you aren't paying a FT employee level salary or benefits, and expectations are your hire is still learning but has some aptitude or interest in becoming a domain expert.

On the other hand, hiring a mid-career SWE who spent much of their career in one domain who is transitioning to another is a significant risk without additional social proof such as referrals where someone actually vouches for their skills.

classified 7 hours ago|||
> No business wants to deal with the headache of on-ramping employees who have never worked in a specific domain or industry…

The usual sickness. If you don’t train people to become specialists and just expect them to fall from the sky, it’s only a question of time until you run out of specialists.

Frost1x 7 hours ago||
I find this take off. In general, LLMs collapse a bit of all knowledge work, including many domain experts. I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions I need to design the system and have an agentic system help me build parts. And it works, quite well.

The places it falls flat is domain expertise that isn’t well documented, like specific business processes. Knowing something will fail because X in dept A won’t like it and will undermine it (politics) or some silly process I didn’t know about forbids it.

So it’s not domain expertise, it’s idiosyncratic expertise that shines these days. Knowing where things deviate from a domain or the standard and being able to adapt around that. Years ago there’s a group I worked with and I was at the mercy of about 3 people I could ask questions to in order to make sure I was doing things correctly because digging into that domain was time prohibitive. Now, I can sit down one evening and dig fairly deep into a domain. I can understand common practice, approach, acronyms, nomenclature, so on. I can find other popular competing systems. I can rapidly figure out what I need.

The same is true for software, agents don’t collapse software they help people build fairly usable systems but they don’t find the expertise a senior level engineer has where designs or implementations may fail in practice. The idiosyncrasies of software and software systems, especially in your very specific set of constraints you need to operate in that may deviate from the rest of the world.

Hendrikto 6 hours ago||
> I work with domain experts often and it’s usually the most difficult part of the process. Now, I can ask an LLM the questions

And you will not notice the obvious errors in its answers to even basic questions, because you are no expert and should have really consulted with one.

When talking about their own area of expertise, almost everybody immediately notices the glaring faults. But then they are very impressed and take LLMs as gospel when talking about anything else. I do not get it.

Dumblydorr 7 hours ago|||
How would you know the agent is correct in domain expertise if you’re not possessing domain expertise yourself?
kakacik 7 hours ago||
Shallow generic domain knowledge is not really a helpful domain expertise, is it.

Lets take banks - knowing how banks work in general means little for that one specific bank who is paying you, which has different portfolio than most, different processes because everything is homebrew over decades of doing business and they differentiate from rest of the market in many ways and thats their key proposition, different data infra, set of internal apps, protocols and so on. There is no general llm knowledge you can distill in few minutes that would help you much.

You need to know your specific company, no way around it. In fact, to be proficient in those deliveries, you need to know those internal politics, processes, their quirks and so on, 100x more than some code fast delivery since this is majority of any bigger project. You need some reputation to get things done effectively.

This, for some time, won't be something llm can massively improve, unless those companies change themselves.

LorenPechtel 1 hour ago|||
I'm in the position of being shallow in the domain I code for--and I find it a very big help. Knowing what the real world implications of what the numbers become is a great help in understanding what truly is being requested. Just ran into that not too long ago--my code suddenly started slamming a side-cutting bit into the wood. Look into it....years ago there was a bug upstream of me that slammed the bit into the wood. Simple fix: spot the offending pattern, replace it with something sane. The upstream bug was "fixed" (not fully), my simple (lack of true knowledge of what was happening) fix proceeded to undo their fix. Then it got rewritten the way it should have been, a more general recognition of unacceptable movements and how to fix them.
enraged_camel 6 hours ago|||
>> You need to know your specific company, no way around it

Of course there are ways around it. For example you can build the foundation yourself (which will cover the 80% that is shared across every company in that industry) and give users the ability to build the rest themselves inside the software. This type of abstraction-based architecture is how platforms like Salesforce took off and became ubiquitous.

burnto 20 hours ago||
The software generalist described in this post has domain expertise as well. In software.

If you’re a great generalist software engineer today, you aren’t jumping to some random domain to escape AI. Software is your domain. You’re sticking with it as it expands and transforms.

dfunckt 12 hours ago||
Exactly, plus you now have a new superpower with AI — the ability to dig into and ramp up expertise in mostly any domain. I’d say the article gets it backwards.
thenoblesunfish 11 hours ago|||
True that AI lets you get information fast an efficiently. But it might just make you feel like you understand, even if you don't have the kind of hard won domain intuition that this article is talking about. I certainly often get delusions of grandeur when I learn a lot of something and haven't yet suffered with it in practice.
bawis 10 hours ago|||
Expertise is largely tacit knowledge, reading AI slop isn't expertise.
dfunckt 7 hours ago||
You can quickly get the "lay of the land" and discover primary sources which you can study. Learning from the AI, I agree, is rife with landmines.
jongjong 11 hours ago||
Exactly. The software engineering domain is huge. I could code just about anything when I was 16 years old after 2 years of intensive learning... It took me an additional 10 years to learn to design software that is secure, maintainable, efficient and scalable.
bob1029 9 hours ago||
> So the most valuable person in this new world is the one who has both skills

This has always been the most valuable person. A skilled software developer who is also deeply experienced in their target domain is untouchable. They can always move the ball forward at some rate, assuming they're making any attempt at all.

It takes a really long time to absorb some domains. The most painful one I've experienced is the banking industry. It took me a solid decade in the trenches before I felt comfortable running a status call with a bank's operations staff without a babysitter.

I like to think of this as a sort of cube of capability. The X axis is technical expertise, the Y axis is domain expertise and the Z is physics (ai/tools/computers). The volume of this space is what we are interested in. Scaling up to a wild amount of AI capability with the best developers on earth (infinite x-z plane) is still going to perform like shit (zero volume) if you have no practical domain expertise available.

Specificity, availability and recency are perhaps the most important parts of domain expertise. None of these tend to be in ample supply with frontier language models. You can get a general sense of how a bank operates from chatgpt, but the FDIC would take over your bank in the middle of the first night of operations because you didn't learn the secret handshake (how to interact with your customers, competitors, regulators, etc).

realist_not 11 hours ago||
As a doctor who learned how to program, I started by writing useless Chrome extensions. One thing I learned early was that programmers generally do not like learning the nuances of medicine, and many are quite open about how little they want to learn it. I have tried convincing my fellow residents to learn a simple language like Lua or even just Python, but the resistance is even greater, despite the fact that they constantly express how they have always wanted to learn programming like I did.

I even went as far as setting up their IDEs, configuring their environments, and encouraging them to just vibe-code. It seems that the mental friction involved in switching domains is too high for most people to justify the reward. Perhaps the reward itself is not compelling enough, or perhaps this is simply the limit of adult motivation.

I started programming with Python around the time GPT-3 arrived, when Cursor had generous free tiers and excellent starter plans. A few Raspberry Pis, laptops, desktops, and countless hours of tinkering later, I discovered how much I enjoyed solving problems with software. There is so much to learn from the programming world: the concept of open-source software, the idea that people from anywhere in the world can collaborate on the same codebase, and the fact that many do so with little or no expectation of reward.

As this post points out, in the project I am currently working on—a comprehensive Clinical Decision Support System—it feels almost second nature to translate the rules, hidden rules, social dynamics of hospitals, and the common mistakes that we and our juniors make every day into software. Taking those observations and turning them into systems that work is surprisingly intuitive.

Perhaps the most valuable thing I gained from medical school, combined with my own personality, is the desire to keep learning. I naturally gravitated toward systems thinking, and the path forward seems clear to me: become a true expert in whichever specialties I ultimately practice, while simultaneously becoming highly skilled at systems thinking.

As for systems thinking itself, I find it useful to create rules not only for the codebase but also for the testing harnesses and development processes around it. The goal is to build systems that can enforce quality automatically as the codebase grows, ensuring that standards scale without requiring constant manual oversight.

chopete3 13 hours ago|
Whenever I read articles like this one that appear to be very generic and give advice about dealing with AI, I remind myself that he software industry is like the construction industry.

It will never be organized, never fully optimized, and it will always be custom — because it has to cater to the reality of wildly different tastes, contexts, and localities. There may be some good tools, raw materials that show up once in a while.

linhns 32 minutes ago|
Spot on. This needs to be said more, and it’s why a lot of us do it.
More comments...