LLMs reward expertise

Posted by MaxMussio 4 days ago

LLMs reward expertise(www.seangoedecke.com)

1407 points | 569 commentspage 8

dr0idattack 4 days ago|

I like it. And if you don't know a codebase, like normal, spend some time learning in. Use the LLM to query it, create your own architecture diagrams. Get homey with it before making sweeping changes. Maybe make the first simple changes by hand.

jebarker 4 days ago||

A fascinating thing about the LLM/AI blogosphere and X is watching memetics in real time. An idea like this one propagates on the order of days until everyone that speaks publicly or in workplace meetings about AI is repeating it.

Arshad-Talpur 4 days ago||

I cant have an overall opinion but in my personal experience i have analysed that LLMs do reward concreteness

kwakubiney 4 days ago||

Might be a very noob question but in this era of LLMs, let me ask the reverse, how do you gain expertise? It seems this rewards people who had expertise pre LLMs, but what about people who don’t have that in a specific domain? What approaches are viable now in this current system?

michaelchisari 4 days ago||

Same way you get strong in an age of heavy machinery: Lift heavy weights yourself.

Skills will have to be built through artificial constraints. Pen & paper, reading books, not using AI, etc.

champagnepapi 4 days ago||

this 100%. Skills are gained via effort. Not too much effort that it discourages you, but enough so it's a challenge and engages you. Ofc it helps to have wonderful teachers, coaches, mentors, books, even ai to help guide you, but YOU must put in the effort. You don't get something for nothing.

lionkor 4 days ago|||

The same as it's been! Make things without using LLMs. Don't debug with them, don't use them to research things, just do it yourself. It'll be painful and that pain is learning.

lucb1e 4 days ago||

I'm not sure I understand the question. What would prevent you from doing what these people did now that LLMs are here?

kwakubiney 4 days ago||

Those people had no choice. In my opinion, it’s harder to grind through problems knowing very well an answer is a prompt away.

jselysianeagle 4 days ago||

But getting an answer is not the same thing as understanding why that is the correct answer, or going deeper and learning more about the subject.

IMHO, the people who genuinely desire to learn will trudge through whatever they need to in order to grow their understanding - be it through reading books, original research papers or what have you.

If, OTOH, all you seek is the answers and that alone is satisfying to you, then of course you simply will not be motivated to do it the old school way anyway. But that's hardly different now in the age of AI.

conartist6 4 days ago||

They may reward it but they don't build it, and therein lies the paradox

mintflow 4 days ago||

agreed on this.

recently i start to rewrite a core part of one of my iOS VPN app to rust, which previously use fd.io vpp as it's networking core, the original vpp port is 1.5 years ago manually by myself, given i know a lot about how the vpp does and how vpp coroutine and runtime scheduling works.

the rewrite is in good shape and solve many issues such as pre allocated memory heap using mmap apis and some scheduling issue of back2back tcp session terminated in the vpp host stack.

also by addressing the issus, i am now can easily integrated tailscale as a addon interface for moving in/out l3 packets between tailscale and the core.

All those i think cannot be done easily without domain knowledge about those networking and system stuffs.

amoorthy 4 days ago||

Agree so much with this! In domains I know well I get much better results then someone who doesn't know the domain because I know where to challenge the LLM. LLMs need to be pushed because otherwise their answers are typically average.

fny 4 days ago||

canine - dog = expertise

It's pretty obvious that for some questions a novice wont be able to drive the conversation towards an "answer". A novice may also not be able to understand an answer either. But there's a more subtle failure mode.

The vernacular used by an expert and novice to describe the exact same problem lead to different traversals of the information space.

For example, I recently asked ChatGPT a medical question using plain english. It gave me an imprecise vague response and told me to call 911. Repeated prodding did not fix this, so I asked the exact same question using medical jargon and in one shot I got what I wanted.

theturtletalks 4 days ago||

Domain knowledge will stand alone as the sole differentiator. Because LLM benefits can be reaped by almost anyone and it’s a force multiplier. Now those who have the strongest initial force will have a far bigger edge than before.

ehnto 4 days ago|

Real world domain knowledge and experience cuts through the chaff too. LLMs are going to have people reinventing the wheel and wasting tonnes of time on stuff that won't work out. If you're a domain expert you are going to be much more aware of how to focus effort in the right places, and what's actually needed or been tried before in your niche.

A lot of this domain knowledge is not in any training data, it's locked up in companies in the industry. I suspect it will get even more important to guard it.

theturtletalks 4 days ago|

Love this idea of reading prompts that lead to new discoveries and figuring out how the person got the LLM there. It truly is an art and I’m always reminded of “I, Robot” and the scene about “you must ask the right questions.”

More comments...