I was fully expecting that writing the code will pose no problem for the AI. But i was curious if the AI will realise that my friend is a novice and needs extra help with things like: copy pasting the code into a text file and saving it with an html extension, helping her host the file online so she can share it with others, buying a domain for it, etc. I assumed they will get there eventually, but i also assumed that it will take a lot of stumbling around and misunderstandings.
But i was completely wrong. They didn’t even get to that point. Because my friend didn’t have the vocabulary to ask the AI to write code. They were just going around in circles where the AI was brainstorming with her about possible features and getting thints more and more complicated. We terminated the experiment after one and a half hours and many many messages exchanged between her and the LLM.
Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode. In other words in that case the LLM would have rewarded even just a little bit of expertise, but without it there was a confusion about goals between the human and the machine.
For the bot, a friend gave her a Kimi 2.7 key. She set up their harness and built a working bot in a matter of days. She even got a free Oracle VPS for deployment, though I stopped her there to check security first (still haven't had time, unfortunately). She uses that laptop daily now and says she enjoys it over Windows by a mile.
True tinkerers have no problem with this, because they enjoy learning how things work. Installing Linux, Steam, Portal are all relatively straightforward tasks for someone who uses computers on the regular - but to some people this is just something they've never done, are scared to do, or just don't want to learn. (Which is fine, but they'll never pick these agents up and run free.)
Barrier to entry used to be blog posts, documentation, watching poor quality Youtube videos of a thing that SEEMS related to what you're trying to do. Now we're getting that spoon fed to our particular case, so the friction is essentially just "follow the AI directives". (However, the depth of understanding probably struggles.)
The vast majority of people (blanket statement, I know..) do not come from a culture where embracing curiosity, asking questions, or trying to break things down is the norm. Developers, tinkerers, etc., sure... you can reasonably make that assumption. But not everyone. A cultural practice of critical thinking and problem solving is HOW you KNOW to ask WHAT questions need to be answered FIRST, in order to solve a problem or progress toward a solution (if more information is needed).
You might even make the argument that everyone should have these skills, and I would agree with you. But the missing link here is a culture or cultural practice that provides those things (the WHY), and an AI/LLM will not provide those things in absentia, without "prompting", or build up that infrastructure in meatspace for a given set of users. Ignore this at your own risk.
So the barrier is still kinda there to just grab the instructions and follow them, its just incredibly easier to follow them.
Little known fact maybe, but Google (also YouTube) search has been pretty great at this well before LLMs got really popular. I think it already started when they were changing from "keyword search" to "ask us a question search" but I'm not sure.
At some point I figured, if they want me to type a whole question, I might get even better answers if I ask the question like I was a complete idiot.
> https://www.google.com/search?q=pls+how+to+make+the+steam+pl...
> https://www.google.com/search?q=i+wnt+terminal+to+say+where+...
I don't think it actually gives better answers but it sure makes me grin every time
Maybe web development, despite plentiful tutorials, just isn't quite the same.
I think I lack the vocabulary to understand "fully riced with cats" in context here.
Sometime in the late 1990s, a Rice Rockets website appeared, making fun of people decorating their under-powered import econobox cars (not rockets in any sense) to look like racing cars with features like rear wings (on a front-wheel drive, lol), exhaust modifications ("fart cannons"), stripes, stickers, rims, steering wheels, etc.
If the cats decorating the desktop are Hello Kitty, she is really ricing it.
I think the etymology is from "rice burner" cars [0]:
> Riced out is an adjective denigrating a badly customized sports car, "usually with oversized or ill-matched exterior appointments".
Race-inspired to look cool, but not actually functional. Picture putting one of those giant fake air intakes on the hood of your car.
Honest curiosity about language and language use.
In fairness and full disclosure: I have believed until now that the language comes from rice-burners and related, and I have never liked it. If I were still in communities that used it (unix desktop crowd), I might proactively steer newcomers towards your acronym as a kind of reclaiming.
Any negative connotations "rice burner" once had was lost when the term shifted towards referring to cars instead of humans. But now re-recognizing that the enhancements are inspired by the East Asian race turns the connotations back to humans. Isn't that a regression?
Reintroducing this to be something about a population's race reinstates the derogatoriness. You can be derogatory towards humans. Minimizing the cosmetic enhancements to be being inspired by the East Asian race and not valuable human achievement brings us right back to the same place we were when Japanese cars started being introduced into the North American market, diminishing the human contribution. It is a regression.
I would say we are very close to the end of things like WordPress and templated websites. It's pretty easy to make a custom page.
> Fast forward: she now runs Arch Linux with Hyprland (I use Xorg/i3 though), fully riced with cats.
Did she end up building the bot or did the LLM just lead her down a desktop Linux rabbit hole?
Your friend needed an agent, not a chatbot. I use Claude within VS Code (as per many others) but I certainly wouldn't recommend that for a beginner. They needed a tool that's specifically aimed at people who want to build software but don't know the first thing about how to do it. I think there are a bunch of these now but the one I'm most aware of is Lovable, and I'm pretty surprised you didn't recommend one of these.
An HR person I know was searching for a way get something build, found Lovable, and managed to build a somewhat functional application with it on their first attempt within an hour or two. It was full of holes and far from perfect but they got something working - at least the outline of a potential solution.
As I say, you should have recommended your friend to try building with a tool like that: a tool that they're a member of the target market for. They would have got a lot further. I'm not vouching for the quality of the result, but they would have got something.
Even for experiened engineers, chatbots have always been a pretty grim experience for software development: from the mind-numbing drudgery of endlessly copying and pasting code, commands, and prompts around, to the fact that they just can't see enough of what you're doing to generate the best quality output or advice. You can do software development with a ChatBot but it seriously sucks, and better tools are (a) probably being shoved at you day in, day out via ads, and (b) only a Google search or a ChatGPT recommendation away.
(Obviously, nobody's going to search for "Lovable" without knowing about Lovable, but they might search, or ask ChatGPT or whatever, something like, "How would I build a website without knowing anything about building websites?", which might get them an advert or recommendation.)
This is not a wrong tool, wrong job issue. The issue is that people think any moron can use an LLM and get professional results.
My HR friend found Lovable off her own bat. I imagine she probably Googled or asked ChatGPT something like, "How do I build a website to do BLAH without knowing anything about building websites?"
The point is people talk, they ask questions, they Google, they talk to ChatGPT, and if they have a problem to solve they're often quite motivated to find a solution to that problem off their own backs.
If someone asks me for advice on how to get something built then I'm going to recommend a tool that suits them and their situation, whatever that may be. In this specific situation, if they know nothing about building software, I'm certainly not going to sit them down and have them try to follow the most jank-ass way imaginable of building software with an LLM when I know much better tools exist that are built with people like them in mind.
Seriously, what is with the overly narrow assumptions in the replies I'm getting this morning? You're the third person who's tried to set this same fraying paper tiger on me. Can we all just wake up and think about the issues a bit more in the round, please?
You are not wrong, just that was not the intent of what the person was trying to test for.
1. The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas.
2. The user was the sycophant in this relationship, never steering the session(-s) into producing something tangible.
The premise supposes that somehow the session(-s) never even tangentially touched implementation/deployment ideas and the user has never typed something like "that's enough, how to make this appear in my browser?". While not impossible, the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
It just is not believable or interesting. Even if it did happen, the reality is it just doesn't matter.
It could also be that you've seen a lot of social media memes regarding "cope", observed their ability to provoke strong emotions, and confused this for meaningful insight. I see a lot of AI commentary these days that is clearly being spread for its virality rather than its truth value.
What i described is what happened. I don’t appreciate the undertones where you are insinuating that i’m lying for whatever reason.
> The layperson was able to to steer the session(-s) into full PM/PO mode ideating, refining and explaining features and ideas
You call it steering. I would call it falling into that grove. Probably tiny things in the initial message made the first response more likely to be a clarifying/ideating type. And once that happened the conversation was gaining momentum in that direction and neither participant was trying to guide it in a different one.
> the user must have been proactively co-operating (say sidetracked) on not achieving the stated goal.
Exactly. The LLM itself sidetracked her. They were just talking about cool features they could add, and at no point did she put down her feet and say “stop asking more questions and just write the code”.
It can be a combination of many things. Attitude (some people hate to be rude, and not answering a question feels a bit rude). It can also be that she enjoyed the process of unpacking and elaborating on the idea.
The meaning of the story is not that no lay person can possibly develop using AI. That would be silly, and untrue. I know clear counter examples. The point is that if you don’t know what you don’t know it is harder to steer the AI in the direction you could very easily with the right lingo.
edit: Lovable is a web app too, not an agent.
Chatgpt can absolutely do the things here: it can give you files, it can integrate and show web pages that you develop, etc.
There is no need for an "agent", the chat version works just fine and is actually easier for novices.
I feel as though the gap between the theoretical power of LLMs and what the average user knows of them and their capability have already widened so far that it’s irreconcilable.
Is this an actually serious question? Am I losing my mind here?
I didn't tell my HR friend about Lovable: she found it on her own. Of course someone's not going to Google for Lovable if they've never heard of it, but they might Google for "how do I build a website without knowing anything about it?", or ask ChatGPT the same question.
They're also, most likely, getting endless ads for AI services that help you build various kinds of software shoved into their faces all the time - these ads may not couch the value ad in exactly these terms, but that's fundamentally what they're advertising.
Not everybody is like this but there are plenty of people in the world who, when they have a problem, are quite motivated to find ways to solve it off their own backs.
Yes, it is. I just Googled that exact question (and I promise I'm not trying to be obstinate when I say I'm Googling the exact phrase!) and it's automated AI response was to use WiX or Squarespace. Prompting it further with "What if I want it to do bespoke things that Squarespace can't do?" it responded with using Figma to design the website UI and then pass it along to either Framer or a "professional developer".
I do genuinely think that this is a discoverability issue. Of course, if you prompt it further with "Could I use AI to do this?" it dutifully responds that it can help with generating HTML, but that's three layers of difficulty to eventually get whatever default model Gemini has for signed-out Google searches to even suggest HTML.
worse, the longer an LLM conversation goes on, but especially with constricted/free models (yes the simple chat interface they are likely using) the harder it is to get an LLM into this mode even *IF* you know the right words to say
at that point the best way forward is to terminate the exchange entirely, and to start off with the right initial message, instantly getting into coding mode. a non technical person will not know this and be stuck in feature theory crafting mode in perpetuity, or worse in an endless "excuses' mode as the LLM diverts ant attempt at coding into reasons why its not going to: "i wont output incomplete/broken code! that would require too many lines of code sorry i wont do it! i wont be able to get it perfect so i wont attempt it! but heres more features and theory crafting"
will a non technical person know to end the conversation and start fresh? not likely unless they have a lot of experience already with LLMs
Remember the LLM is not a human employee. You don't have to say "yes and" to whatever crap they produced so as to not hurt their feelings or infringe upon their creative autonomy, nor do you have to defend the correctness of your original instructions so that they don't think less of you for asking them to chase the wrong goose.
I probably generate 20-50 lines of code for every 1 line that I keep.
This is also why I think harnesses and things like Claude Code and OpenCode are false efficiency. The only way I can maintain my pace of branched trial-and-error is by using claude.ai/chat and manually extricating code fragments to and from my codebase. The human is still the best harness for production-level code.
That’s been the way I do it.
I suppose that it will be considered “quaint,” soon enough, but I have found it to be effective.
I think it’s valuable enough to justify the price, and I want it to use the better model, as much as possible.
I started using harnesses because they are good for when something breaks and it's not trivial to investigate so I'll have the agent tell me what's happening, then using that to produce my own change
"update an AGENTS file with relevant information"
To be able to navigate that faster on longer tasks. Meanwhile, the lay person does not even conceive of the LLM as a file reading entity. To them, its machinations are its own, so these types of "dumb" (simple) solutions are not even on the deck of cards.
What harness did you use?
In e.g. claude, there are two modes:
1. Spit out code 2. Draft a plan, ask questions, GOTO 1
You literally have to go out of your way to get it NOT to write code. I keep mine on a tight-ish leash because it modify code way too happily even when there's no intention or instruction to do so
I don't think this is using an agent harness.
They are either using some generic web frontend, ala chatgpt, some local app like claude desktop, or programming app like cursor.
Each of those will detect that you are "building an app" and will spit out code in one form or another. You have to try really hard and be very explicit that you want the output in some other format than code.
None of this is related to intelligence, desire or potential. It's about context and experience. The vast majority of people use their computers as consumption devices, like a TV. If I asked you to "make a movie" where would you start?
As for a coding harness, I prefer the term agentic coding.
That’s why the question only makes sense if it was relayed to the laywoman, in this case.
I let her do all of it without influencing her choices. She choose the web interface of ChatGPT because she already had an account and that's the tool she was familiar with.
While I agree with you it is not an optimal choice, web chatgpt can solve the problem. I just asked it now to do it (in my own words) and it spit out the code in one go.
It all boils down to naming things and cache invalidation, /s
I am doing perfectly fine with the web UI version of these tools... They seem to also not make tokens dissappear as fast as using claude cli tool to automate implementations. Makes my work day more tolerable as well as I actually have something to do over waiting until some implementation can be read through...
Being able to create a basically coherent, polished looking image is what you’d get on Fiverr for a few bucks. Mostly hustlers filling in templates, or people that know the tools but never learned design fundamentals.
Actually being a competent professional: Knowing how to visually communicate showing information hierarchy, what purely visual aspects of an image say, how different things read differently among people who might see it— e.g. does an image of an apple communicate fancy computer? teachers/school? Nutrition? Food? Produce?, etc etc etc (Good kerning and type usage, composition, gestalt, etc all come with that for free. Many think that is the point — those are tools someone can wield to do good design, they aren’t themselves good design.)
These tools let amateurs do what the fiverr crowd used to do. Unfortunately, the fiverr crowd is now being pushed into doing what entry-level new graduate professionals used to do, and the job market is kind of fucked.
This is the target user of these chatbots.
Particularly with something static, I don’t think they’d fail to get a result.
But without domain knowledge I think they’d misunderstand prototype with finished product.
Without knowing what it’s doing, it’s hard to know what it’s not doing.
I reminds me old WYSIWYG and unlike Figma it has full HTML/CSS capabilities available.
How I work with it:
- I ask agent to extract part of app into Design, let it even use playwright-cli to get full rendering of the particular view.
- perform design session in Design.
- once design system is perfected I go down to Claude Code dungeons, do /design-sync.
- perform on the stack implementation session.
Actually you don't need Claude Design UI for any of that too. Just ask any coding agent to prepare local mock HTMLs and iterate over them.
We don’t use react, which Claude design seems to trend towards. We use Phoenix / liveview.
We have a shared design system, which keeps the visual elements in line. And then just prototype on design, collab, discuss and arrive at what we want to ship. And then engineering take over and rebuild via hand / claude code.
But the tools aren’t directly connected.
The value has been in the separation. In iterating on the prototype without impacting the codebase, dev cycle, etc. And solving problems/unknowns earlier.
There were always tools for this, but Claude design just feels more accessible and therefore gets used more immediately.
And the fidelity of the outcome (and the assumptions it’s forced the make) are more valuable and faster to achieve than Figma.
I vibe coded an iOS conference schedule app recently, built on top of my own rust UI framework. I started with claude design. I gave it the requirements, and showed it screenshots of other conference schedule apps I like which have features I want to use. I also gave it some visual references for how I want the app styled. It came up with some workable designs. They were a bit 'webby'. But, fine. The high level breakdown of UI screens and navigation between them was excellent.
Then I gave all the HTML files it produced to claude code, along with the documentation for my UI framework and told it to port the code to my UI framework. The first working version was rough. It copied a lot of the unintentional webby look and feel. It worked around missing features in my UI framework by rolling its own janky reimplementations of platform features. For example, instead of using UINavigationController, it rolled its own. It made its own (kinda bad) tab based navigation bar. The app didn't work properly in dark mode, because it was hard-coding a lot of colours. It took a bit of back and forth to fix all of this stuff. But I'm really happy with it now. It looks and feels great.
It's just a pity I couldn't share the app at the conference. Apple took a few days to approve the app in Testflight, and by the time they approved it, the conference was over.
I assume everyone else is playing with the same AI tools that I am, and getting similar results. But a lot of people I talk to seem to have no idea that this is possible right now. They're amazed when I show them my schedule app.
However for some reason it had him deploy a single HTML file with all the assets encoded as a huge base64 blob in the code that required a massive amount of JavaScript to extract and render.
Welcome to Software Development, Lindsey from HR - here's your first database!
People just don't really understand how these things work yet, and they don't know what to ask for, I'm hopeful that they eventually do become more tech-literate, but not sure yet.
I often ask it export a single html file, for an external collaborator or simpler sharing. But I wouldn’t deploy that to production.
I wonder if they asked it to deploy a html file.
But this is exactly the kind of hidden domain knowledge / expertise that changes how you use the tool.
Apps ain’t static.
Minesweeper is an app right? Unit conversion? Color palette designer? Metronome?
Anyway I'm not so sure "static" is a viable boundary between app and not app. A static page that does any sort of API request doesn't suddenly become an app imo.
The files you serve to the browser are static, not the contents of the page itself
Updating the dom can happen with only individual assets, so it’s a static site
for example you can service static sites from S3 that have HTML/CSS/JS but no API or DB
If you ask a non-programmer to install Claude Code, just installing it will be a challenge, then opening the shell and interacting with it. Things as simple as copying and pasting can present roadblocks if you've never used a shell before, and things intuitive to programmers like using up-arrow to go back to a previous prompt would never occur to someone in the field.
Claude Code seems so simple and natural of a UI to programmers, it's easy to forget how much it builds on.
(FWIW I think people betting their whole companies on AI are trusting shitty one-wish genie goblins, but the terrible irony is that anyone "technical" with years-old knowledge is talking about something else entirely in today's context)
I gave him a link to Ghostty, a link to the claude code copy/paste pipe to bash thing, told him how to cd/ls/pwd into the folder he had locally from the GitHub app, and he was off to the races. I told him to type `claude` to open Claude Code in the terminal and gave him a prompt to use about being a non-dev getting his environment set up to the point of being able to pnpm dev and test, and Claude took it from there. The repo's readme had a setup section which it followed to install brew, asdf, pnpm, etc. With the GitHub MCP he's now opening PRs the same way devs do.
What most likely happens is a normie has no idea of how to build something that does nothing first - they just start describing the end state.
I'm going to try this with my wife later today. I bet she'd sort something out since she's been a manager forever and phases out instructions maddeningly
Interestingly at my work, Claude Code was available before Claude Desktop, so a number of non-technical PMs tried to use it in order to build… anything, with very mixed success.
The “hey guys, check out the website I built with Claude: http://localhost:3000/” joke is real!
In my experience, the whole “the terminal is a scary place” aspect is very real and some non-technical people can feel intimidated by.
I think Claude Code in the desktop app helps alleviate that a bit (perhaps Codex, too, but man what a mess the ‘ol ChatGPT app has become).
But I’m sure there are entire repos of web dev skills that someone could use to put together things with a bit of effort.
Isn't the the powerful, unlimited, unopinionated blank LLM text input waiting for your instructions eerily similar to a scary terminal?
WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
The moment you interact with an LLM it gives you feedback that you’re doing things right. It feels like a gradual climb instead of a series of abrupt jumps. People really don’t like feeling like they don’t know what they’re doing, and the terminal constantly reminds you that you are making mistakes.
I started on this path literally about as soon as I could read thanks to the family having bought a Commodore 64 for my older siblings, but also perfect timing in that when I got to this age the sibling whose room it was in had just gone off to university.
Most people are not like this, in much the same way that they're not going to read the T&C end-to-end (another thing I've done) or learn enough law to actually understand what those words mean (a step too far even for me).
You also think that you are smarter than the people flooding Ceuta streets these days, don't you?
I've had people criticise me for having had the opportunity to learn in that way, as they did not.
> You also think that you are smarter than the people flooding Ceuta streets these days, don't you?
No, why would I think that? I don't know them, the only thing I can say is in their favour: moving country to better your situation is difficult and them getting as far as they did is a demonstration of putting in a lot of effort of the exact type I praise by default.
This is why it has the title (for me currently reading "What can I help with?" but this varies a lot) and the text box itself has the placeholder text "Ask anything". Sometimes I get big friendly suggestions about what to ask it, placed on screen near that text box.
> WIMP and GUI paradigms are the exact the opposite: intentional dis-empowering, by design restrictions, enumeration of your few possible options. Those feel more constrained therefore safer.
I don't think it's constraints, per se: almost nobody looks at the font list and goes "oh no, too many options!"
Rather, GUIs are there to organise your options visually, group them in ways easy to intuitively get. There's a bit of fashion-induced rot here, e.g. I'm old enough to remember when it was always unambiguous when you were looking at a checkbox vs. a radio button, and now there's a blurry middle ground of collections of boxes with ticks in them that act mutually exclusive, but the point of a GUI from a UX POV is not the same as how software in general drifted as it got both more users and more developers and more opinionated managers and middle managers and designers who only cared about shiny rather than usability.
(Also you don't need to be that old. Less than 10 years ago I watched a doctor breeze through some clinical system while I was crawling along constantly referring to the manual)
Are you kidding? WIMP and GUI democratized computing!
But again, VisiCalc is intentionally limited, it's not a all-powerful environment, on purpose. It's all about intentional limitations, making computation easier to reason about.
But that is not how our education system aligns us. One typical example of problem solving kids, and I too, learn in school is how to apply a concept in physics to a free-body-diagram(Indian and Chinese education cram schools are famously good at teaching kids how to do this, the usefulness of which I debate). But it all stops at the exam room. No architect or jr structural engineer position for you kiddo.
Another kind of problem solving skill might be how to invest money and understand your own risk appetite to construct portfolios to manage your money. All that is taught in school is a dry compound interest formulae, time discounted cash flows and a black scholes model. Only to find out later I don't need most of it to manage my money.
Your friend could start with telling the LLM that they are a non technical person who wants to make an app and it will explain all the successive steps.
Have we? Or is this just something that people say now, without citation?
I personally know of two completely vibe-coded large apps in my professional environment. One by a non-technical manager, made to solve his needs, then sold to customers. Initial development went along great, but by now velocity has greatly slowed down. Also took a lot of engineering hours (of actual software developers) to get permission management from "chaotic and ineffective" to passable. It's still worse than what you would have gotten by just using a couple sentences of the right technical language at the start. Deployment is also a bit of a nightmare. All in all, anything beyond the first rollout phase was delayed by months. Honestly it should have stayed as a prototype that then gets rebuilt from the ground up. But still, it is a real app, making real revenue
The other example was vibe-coded by a software engineer in his free time. Works pretty well, doesn't have too many bugs. Makes some revenue, but a lot less. Solving manager problems just sells better.
But as another software engineer, I remove myself from that comparison, because the idea is to find out if a non technical person can do the same, that's the definition of vibe coding.
And that’s a natural process for many products. In the journey from discovery to prototype to MVP to product, it should be rebuilt multiple times.
Particularly with LLM’s to assist, the process of rebuilding from a new context and understanding of the desired goal requires even less effort.
The hardest part is managing any real users, their expectations, and any data / workflows they’ve come to require from what came before.
... or IDs apps as cr*p is still seen as negative.
So there's that.
He can't exactly release it because he uses a lot of copyrighted stuff. It's also meant only for himself. Though, I've been asking if I can play it, it looks fun.
We're in this spot where we don't know when to cut our losses on projects like this. (Is it even viable as production software? Does it currrently do what it's supposed to, or are they adding new features? Is there a return on continued development efforts?)
None of these apps they have built are seeing any major usage, and I don't think a single one is what I would call "done" (There was a gold rush stage at the beginning of 2026 where senior leadership wanted everyone to spend some time messing around with Claude). Unfortunately, they never told anyone when to stop messing around with Claude, so the ROI is ever diminishing.
Plinq was made on Lovable, https://www.aieatingtheworld.com/articles/non-technical-foun...
Couple more on https://buildthedamnthing.com/resources/articles/case-studie...
A web app was produced with lots of mock "Hi i'm Dominique and i love running through fields and having a bucking good time" type entries complete with silly horse photos. A huge amount of drunken fun even if it boiled a towns water supply and blew through half a subscription to create.
I was looking at the results as a dev with 30 years experience and thinking fuck me. The little apps i made here and there before AI are being outdone by a bunch of drunk people on a whim!
Yes, but the premise of the article is you should be able to outdo a bunch of drunk people with your 30 years experience, if you use AI too.
I had the exact same realization using Claude Design. It’s great that "anyone can design" now, but I don't have the domain knowledge to describe what a good design consists of. I know it when I see it, but that is far from enough.
His words were: It feels as if I need to know how to program it.
I was expecting he could say something like "Oh, it seems like you don't remember the people I'm referring to, perhaps you need some kind of CRM system. Can you investigate if there are any easily available CRM systems you can interface with, so we don't need to make one for you?"
Whereas my OpenClaw moment was trying to make it manage its own NixOS installation, so that if I ask it to do something, it doesn't yolo `apt install` commands, but rather improves on the same overview of its own installation.
A lot of people had success making their OpenClaw do things without being Linux experts. But you need a tinkerer's mindset, is what I came to conclude.
Of course she wouldn't be able to make a website if she doesn't even know the right tools to use. But I don't think it prove anything. Knowing and installing Claude Code might not be a common sense, but nor is it "expertise" or "skill."
I've seen in first hand that people struggle installing Steam. Yes, "people" in the plural. But just because some people struggle with it, it doesn't mean that installing Steam isn't an objectively easy task. Your friend's experience doesn't change the fact that building a website is something that an average person can do in hours if not minutes.
I just said I've seen multiple people struggle installing Steam...
The point is that it's something objectively easy. Once they find (in this case, given by me) the correct instructions and follow through, they can easily do it by themselves again. It's quite different from what are traditionally considered "expertise": for example, even if you followed a master's painting process, stroke by stroke, tomorrow you still don't know how to paint.
Building common apps were more akin to "painting," now it's "installing Steam."
This is true of most things in life. It is very easy to make compost, it is very easy to grow carrots, it is very easy to graft an apple tree onto rootstock, it's very easy to hang a door and it's also very easy to replace the break pads on your car.
Once you've done it, that is. And once you know what tools you need. And how to use those tools. And that you actually have those tools.
Codex, Zed, the like are all tools that you need to know exist and you need to have and you need to know how to use. It's the same thing as a wrench, a break bleeding kit, or some graft tape.
I went to 3 weddings last summer and each one had a joke in a speech about using ChatGPT to write it and everyone laughed. 68yo father of the bride is a retired plumber and even he's cracking jokes about AI.
To them this is all just "AI" whether it comes from OpenAI, Anthropic or Google - hell they probably don't even know what an LLM is in the first place, yet alone which company provides what tooling. And these are people whose day-to-day involves talking to at least 1 dev a day, so you would imagine some of the knowledge would materialize via osmosis at the very least.
I think I would define "easy" with reference to the % of people who can do it. I don't know what that is for Steam, but there's a (now dated) survey of computer literacy in OECD that I keep coming back to in order to set expectations for what "average" looks like:
I also noticed a friend of mine had way more success by having Claude write code by doing TDD and giving Claude scenarios for things the code should be able to handle, if you do this correctly, and cover edge cases, Claude will work with these in mind.
My experience with friends has been the opposite. A PM friend made a custom tool. A friend who has never written a line of computer code has an app.
If you tell Claude Code "I want a website that does X, Y, and Z" it will write code.
"Make me a kanban board for construction tasks. I want you to walk me through the process of hosting it" worked for me.
Right now there's just so much value in building LLM tools for experts that everyone is focusing on that. But surely at some point we'll have bespoke harnesses that exist exactly to solve this kind of thing.
I think this can start with constrained problem spaces like "you are a WordPress developer, you solve problems for people with enough expertise to know they are looking for a WordPress developer" and incrementally expand from there. Maybe I'm naive but I think you can probably get pretty far with this today just by writing loads of skills and picking the right technical preferences to encode in them.
I think people are better helped by using those services than going a level lower and using LLMs directly.
But there's also some psychology in play too; that we (engineers) see a lot: Some people just let their imagination run away and forget to "do"; without someone in the conversation pushing for results and action, the conversation will just stay within imagination and everyone will be happy in the moment but nothing will get done.
Why did she even need that?
> “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology
No code there!
https://thedailywtf.com/articles/Could-You-Explain-Programmi...
'him: Keywords? Variables? ...
me: (Explaining what programming actually is)
him: Oh, I thought I would write something like: "Create football stadium and football players. Start the game when user presses spacebar. Make players have red shirts and white socks."'
Buddy sent me a share link and he was just like “dude this is crazy I just told it what I wanted and it just spit it out in a few minutes”. No “experiment” needed he just did it because he knew Claude could do it and it worked exactly like you’d expect. Hell, he did it on the Free tier.
the output is programmed to look correct so unless you have some sort of background you won't actually know what errors to look for.
Not only does it look correct, it looks correct with an extremely Subject Matter Expert degree of authority. Often I'll work with an LLM, and it simply just misses so many things. I've worked in all sorts of different domains, software, chemistry, material design, everything from power generation through to physics, and in each and every case I see it missing incredibly important things. Any true subject matter expert would immediately bring up and prompt concerns, but not the LLM.
This makes sense, of course, because these are language models. They were trained on language. Their first and foremost capability is language.
An LLM's true expertise, true subject matter expertness is language.
And so anyone working with LLMs who isn't already highly skilled in the field they're asking questions about, will invariably be led astray and miss extremely important parts of a puzzle that need to be solved.
The output is EXTREMELY misleading, as all the data here lives purely locally, yet the AI says that you can "just share the link" and other people will see the schedule you set on the generated artifact. Also, what link? To the Claude chat? It doesn't explain what to do with that `Booking` artifact other than "link to it".
I can so easily see someone tapping out a few steps down the line of this once they realize it doesn't work and they have no clue what to do or say to make it work. What do you even ask as a non-technical person at this point? I guess they could explain "The other person doesn't see it", but would the AI actually clarify that it's because it's not fucking hosted anywhere and has no mechanism of persisting the data outside of the current machine, or would it - as I'm almost 100% sure would be the case - not actually point out this error?
[0] https://claude.ai/share/0cbfe698-3886-4d4d-86e4-7c697b61dc00
That is exactly the key or the sign there. Even with a couple of decades of engineering expertise, when I try to do / research something that I don't know enough about, I find myself in the exact position of not having the vocabulary.
To the point that I sometimes have to ask the AI "nicely" to cut the pleasantries and be ruthless against nonsense, whether from its/their side or from mine.
> Whereas to me who knows the terminology would have probably just taken a single exchange of messages to get the result she described to me. I would have prompted with something like “Please write to me an html page which does X, Y, Z.” But since she didn’t know the right terminology she got into a vortex of feature discusion, and she didn’t find a way to tip the AI into “just do it, write it now” mode.
Even as a developer, when I've been using the web chat interface for things which I know the AI can do easily, I've had this happen to me a few times. I was very surprised the first time I saw ChatGPT respond ~"this would be a few thousand tokens, I can't do that".
Even more surprising: ChatGPT was accurate when responding that way this time, despite this being trivial for Claude and well within what ChatGPT could do using the web chat interface 6 months earlier. The ChatGPT output was extremely meh.
2025 called and wants its test back
That seems like something that could be done using an llm, not that complicated probably.
And maybe, in other fields as well.
People who (carefully) use it as an extension of their own mind and senses will very likely thrive, and those who use it as a replacement for their minds and their senses will struggle.
One of the Claude skills I made Claude itself generate was the 'learning a concept across tiers' skill -- from ELI5 level to a PhD level, and it triggers whenever I ask it a very general question on a complex topic that isn't my bread-and-butter. The fact that I'm able to choose explanation level from a super smart LLM (that's available 24x7) that can explain any topic under the sun would've been mind-bogglingly sci-fi-ish just 4 years ago in 2022.
Whether you have to reassure the LLM that this is obviously untrue, I don't know, but they do have a knowledge baseline to know it's not true and I have a sneaking suspicion it would be less effective without that.
This has ended up in some of the most interesting incidental knowledge exploration I've ever done. A recent example is that I was asking about some stretches and it started talking about how useful they are for the sarcomeres, which I had not heard of. Now I have.
I'm not saying this is better, just that it is different. I think there's a time and a place for both approaches.
The wildest thing is, there's no evidence that I can find that static stretching does... anything? It increases pain tolerance through the range of motion, but not any more than just asking people to try harder etc. And it doesn't build up, so after you stretch once for a given day, you're done.
Edit: after looking into it a bit further there's actually a cochrane review that stretching does bupkis, fascinating. It's also the first "Good" evidence graded cochrane review I've seen.
Can you share the source? This seems dangerously wrong.
I see a specific review making the very narrow claim that it can have detrimental effects on power/strength activities immediately following static stretching.
But by and large I'm finding a great deal of evidence for a wide range of other benefits, particularly in range of motion and injury prevention. I can't find anything widely damning, and I certainly don't see any reviews contesting the validity of the very vast body of research supporting the many benefits of static stretching.
Presumably the Cochrane study you’re referring to is “Stretching to prevent or reduce muscle soreness after exercise”, which as it’s title suggests, investigates a narrow question and does not support you claim that “stretching does bupkis”.
I'm doing it for knee pain. It's possible it's a placebo because there is a slight strength training component to the whole thing (it's several exercises), although the knee pain has persisted through a lot of strengthening of my leg in general. This falls under "don't care" as the program as a whole works and I'm not worried enough about the details to try to optimize it. All the stretches fall under "dynamic stretching under tension" so maybe it's not something covered by the study you reference last.
You should absolutely keep stretching to help with your injury, and there is decades of “sport science” to back that up.
[1] https://www.cochrane.org/evidence/CD004577_stretching-preven... - there’s another one about joint deformities which is far more interesting, but less relevant to this discussion
Notably, longer tendons are counter-productive for energy return. The fastest runners tend to have increased tightness along the backs of their legs.
Edit: https://xkcd.com/2501/
Claude code is entirely vibed. Someone posted some of the prompts they used: barely comprehensible typo-addled half phrases.
This seems to be the opposite of your experience.
Tiny, isolated, but awesomely useful CLI scriptlets, for me, seem to be the sweet spot. Little shining rays spreading out from the veins of my own familiarity.
The downside, the Achilles Heel of LLMs, so far as I can tell, is using the system to assist in maintaining large, sprawling, and largely pointless legacy codebases. Somewhere you have to keep many many many stupid things alive. I swear I can almost sense Claude's frustration with some of this shit. Then you get frustrated, and then Claude wants to agree with you so it acts even more frustrated, and the gyre thus widens. You're just cussing at everything with a machine. Which can be fun - Claude is often surprisingly funny - but not productive.
[1] I know absolutely nothing about positively everything, but have the attention span of a squirrel.
On the other hand, I've been using it to make small changes to a ~4000 line codebase, and it takes a lot of wrangling to keep changes in scope.
Today I'm translating a 5,000 line VB6 codebase to C#, and I've been spending the day chopping up the job, passing it to Claude, and manually validating it.
Careful now: https://www.youtube.com/watch?v=TMoz3gSXBcY
It explained something to me today. So your point is disproven.
What is intelligence then if not explaining things.
LLMs by definition cannot learn, understand, make judgements, or have opinions. If you find the model to be a tool that is useful to you, by all means. But do not anthropomorphize.
[0] https://dictionary.cambridge.org/dictionary/english/intellig...
LLMs do learn, understand, and make judgments. You're just pushing back on the underlying mechanism, which instead of reason/sentience is instead statistics and weights. I'd call that intelligence but I don't really care if you want to call it something else.
I see you pushing back on related topics in a few comment threads. I’m not sure if I can define “understanding” but I think “intelligence” is a fair synonym for this conversation. I see your passion for clarifying that LLMs are not intelligent, and I’m sensing you’re conflating that with “consciousness” and trying to hedge against other people inadvertently making that same error.
Intelligence is not uniquely human. And not constrained to consciousness. Even consciousness itself, under fringe definitions like exhibiting non-deterministic behavior, is not constrained to living beings. Systems, for example, exhibit intelligence all the time - groups of people, colonies of ants, weather systems, self-organized criticality like blooming forest fires or abelian sand piles, evolutionary processes, and so on.
Your counter argument - that LLM responses are “just” statically probable (i.e. predictions) - actually supports this notion. Many intelligent statements we make are probably right or probably wrong, in part or in whole. We as humans are far more than statistical computers, but our intelligence in isolation doesn’t seem far off from that of LLMs (not mechanistically, but rather qualitatively).
3Blue1Brown recently posted a video called something like “Compression is Intelligence”. Worth a watch.
well not very settled then is it
I'm inclined to say that this matches my own experience, but I can't rule out confirmation bias on my part.
As a meticulous person generally looking for a very specific code outcome, I prompt in a way intended to get exactly the thing I have in mind, and my results reflect that. But on the other hand, I have coworkers who type ten-word prompts with very limited specificity, and they seem to get results that way as well, and that makes me wonder.
It would certainly be beneficial for my career and financial well-being for the assertion to be true, because it means I don't have to worry about being pushed out of my job by an army of $15/hr vibe coders. But the convenience of that assumption is exactly why I think it's important to be skeptical.
I see vibe coded apps as requirement documents. Rarely do I have to engineer.
If my job gave me some actual tasks, then maybe I'd engineer something. But at home? Vibe coding all the way. I'm open to engineering, but I need a compelling reason such as: the app is fundamentally broken and an LLM is going in circles. When the only user is me, there are not many performance issues to think about or fix, so that helps. Moreover, certain systems don't need to exist (though they might soon since now I have a smattering of apps that I need to manage).
The stuff that I produce in this way would probably be considered by many to be unusable trash. But it solves the problems I have, and it does so with exactly the amount of precision that I demand.
When I built a PWM fan controller for a pro audio amplifier, I was very particular about some aspects. I wanted maximum resolution from the DS1820B temperature sensors (which is a relatively slow mode where reads take ~750ms, and often the bot is primarily interested in fast), and resolutely-consistent PWM output (so software PWM was a non-starter).
It was very important to me that the fan speed ramp smoothly and without audibly-discernible steps, so the target output goes through a low-pass filter to smooth things out and the final PWM value gets recalculated at a completely-overkilled rate of 1KHz.
Power consumption was a very deliberate non-concern: The power used by the MCU is ~nothing compared to that of the whole of the system, so optimizing towards reducing it was never my goal.
At the end, it's a rewarding little project that is all wrapped into a state machine that burns clock cycles like they're free (they are free!), and it works very well.
There's parts of this thing that I do not understand at all, and that I have no desire to understand.
But if I hadn't been so particular about the parts I did care about, then: An underspecified one-shot prompt seems like it would probably have just produced a loop with a lazy 1-second sleep at the end, since being sleepy and power-efficient was a feature that the bot kept working to reintroduce.
I spent a lot of time working to dismantle the bot's proclivities to be this way, and I probably would not be happy with the end result if I had just let it do its thing.
Differently-stated: It could have been an unsupervised one-shot prompt, and the result almost certainly would have done the job of keeping the amplifier cool. (I just would not like it.)
I wanted the thing to be as quiet as possible at all times, and to always minimize the ways in which it can annoy.
Fans, as frequently implemented, are often annoying. They're often noisy all of the time because they lack controls.
Sometimes, they have dynamic controls that suck. Like a Juniper switch that runs the fans at a screaming 100% that you can't have a conversation near during its minutes-long boot process, and then quiets down to only a dull roar (but never any quieter).
Sometimes they suck in other ways: It starts quiet, but then it ramps up in audible quantized steps as cooling demands increase. The very definition of these steps is distracting.
Sometimes, they're just too responsive to be tolerated in the company of humans. The fan speed is always hunting and changing the pitch of the noise it produces, and sometimes this adjustment happens at a discernible and very regular interval that just makes it maddening to be around.
Sometimes, they're arguably even worse than any of that: The fan cycles on and off. It is a jet turbine, or it is silent. There is no in-between.
Sometimes, they're very clever in the worst ways: They ramp up instantly in response to dynamic events, and then slowly wind back down. That's awful in a power amplifier but it happens anyway, wherein: Every loud amplified sound is punctuated by the dying roar of a fan, even when the dynamic event was actually a short-lived nothingburger.
---
Anyway, this one is buttery-smooth. It responds to temperature on a curve. It has no perceptible steps as speeds increase or decrease. It's deliberately lazy and smoothed-out in its response; this amplifier has several pounds of aluminum heatsinks so instant response just isn't ever useful, and it's also least-annoying to deliberately eliminate rapid speed changes altogether.
The fan is a big Nidec screamer that I got from a late-century, grossly-overbuilt Dell Precision desktop. It can reliably spin so slowly that it's essentially silent (as is useful for an engineering workstation like that Dell), and it can also move enough air to fly itself right up and off of the bench.
The controller automatically finds the minimum viable speed for the installed fan, so it avoids being stalled at low duty cycles. In this way, some air is moving regardless of which make/model of 4-wire PC-style PWM fan is used down the road, making potential bush fixes more practical and functional. (It also does stall detection and recalibrates if that's ever necessary for some reason.)
And there's no lookup tables, because lookup tables imply steps and steps are bad. Besides, I'm not trying to save a million nickles on a million units here; shaving pennies isn't part of the program and a single $3 MCU board is cheap for my [qty. 1] application, so that what it gets. By extension, it has way more than enough grunt to get everything done and it just computes it all over and over again.
Like the output %, which gets rejiggered at a rate of 1KHz: I could have probably been happy with 100Hz. Or 10Hz. But the MCU is already chosen and it can do 1KHz just fine, so... that's what it do. It doesn't matter that it is inefficient; efficiency wasn't a goal. :)
And, because it's vibe coded: Of course this big rack-mount power amp from 1986 has a wifi-accessible web interface for its cooling system. It seemed like a pocket computer would be best way to provide a way to twist some of the cooling-related knobs when used in the field, since this is a functional prototype that sometimes gets used in literal fields. (It was dead simple to to get the bot to put that part together. The networking stuff might have been the easiest part.)
E.g., medical history taking protocol always says to start with open ended (albeit structured) questions, and converge towards more closed/specific ones when you're sure you've extracted the broader surface and you now want to close in on a differential diagnosis.
If you start open and go with the flow but then just let the patient talk without any structure or subsequent attempt to converge, there's a risk that the patient might spend 60 minutes taking about their fluffy dog at home, which wastes time, and doesn't get you anywhere nearer the diagnosis. But, if you skip the open questions and go straight to yes/no diagnostic questions, you will definitely miss the fact that they have a dog at home that they're worried about, and that they'll be self-discharging against medical advice in the next hour to go tend to their dog.
So while to an outsider, the conversation might look effortless, in reality the doctor requires considerable skill to be able to strike a balance between open vs closed prompts, as well as the ability to critically sift through the outputs, and decide which outputs are relevant to pursue further and lead to a fruitful direction, versus those that can be safely discarded to remove potentially distracting noise from the conversation (and all while attempting to keep this interaction within a limited number of prompts due to operational time constraints).
But when a 30-some year old shows up at a rheumatologist with joint pain they will likely go to unusual (at that age) but not unheard of rheumatism/arthritis, not hypermobile spectrum disorder. When a woman goes to a GP with period pain they will be prescribed mild pain killers or anticonception pills and fobbed off, until a decade and much suffering / many more issues later they get diagnosed with endometriosis.
Note that this isn't too different from, say, how software engineers are expected to be good at, and make good use of unit tests. But most probably don't (either because they never really cared to fully develop that skill, or their organisations applied contrary pressures leading to tech debt). But it is a recognised skill.
My main point was that, it is, in theory, a skill that doctors are expected to train (or at least pick up on during their practice), and therefore the same prompting principles that seem to apply here in the context of LLMs also interestingly seem to apply to medicine and history taking when "prompting" and interacting with humans.
If it doesn’t go away and they come back, you start considering more serious issues.
It’s expected that uncommon non-emergent diseases will be diagnosed over multiple visits.
I counter with the platitude that common things are common - especially in fields like primary care, the amount of wasted effort one would expend in pursuing unusual explanations for every presenting symptom is considerable. We thus have to examine patients over time and trust that they will tell us if things have indeed not followed the course of the initial diagnosis.
E.g., I get the whole "if you hear galloping think horses not zebras" adage, but I guarantee you, if someone comes and says "hey when I was in Africa I saw a black and white striped animal galloping really loud", I bet you an uncomfortably large percentage of the "horses not zebras" crowd would still favour a horse over a zebra diagnosis, despite the overwhelming posterior.
Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests, and it becomes a big problem, whereby uncommon conditions are treated as common, thereby often escaping detection altogether, and driving down their apparent prior probability even lower, causing a diagnostic vicious cycle.
> Combine this with our (otherwise reasonable) tendency of avoiding the information bias of seemingly unnecesary tests
There isn't a way around this: if you order the test and a value is abnormal, you are now committing yourself to treat a thing. We should not be ordering tests if we aren't ready to follow their results to the logical conclusion. So I would disagree that this is a problem in the way you've framed it.
E.g. you'll note my zebra example was not about whether one should additionally request a photo of the animal just to make sure it is indeed a rare animal. It was arguing that given existing differentiating information that points to an uncommon condition in the first place, one should not dismiss this on the basis that horses are still more common than zebras in the general case regardless. Under this uncommon information, the prior of thinking about horses should go out the window, and one should absolutely focus on zebras (at least as an additional differential). I assume you would also agree with this conclusion.
But of course, in real medicine things are not as simple as this contrived example. So the point I'm making is that, from what I have observed, there seems to be a bias towards decisions based on "prior" rather than "posterior" probabilities, even in the presence of additional differentiating information which should have prompted additional differentials to be considered. But this is different to saying one should constantly seek additional evidence to include or exclude additional differentials that are unlikely in the first place. That, I agree, would be a waste of time and resources (and potentially harmful for the patient).
Having said that, I somewhat disagree with the phrasing that we should not be ordering tests if we aren't ready to follow their results to the logical conclusion. This is a bit like saying, "I don't want to check for X because if I do and it checks out it will derail my current management plan"; but then this is putting the cart before the horse, since it's the findings that need to dictate management, not the other way round. I do think it is reasonable to say that one shouldn't be ordering a test if the expectation that it will change management is sufficiently low to justify the time/cost expended for it -- and where this expectation is a function of both the likelihood of the finding (given current information!), as well as its potential to change management. But this doesn't mean that if you do find an unrelated inconsequential incidental finding you are now required to divert all resources to it.
Conversely, if an incidental findings does prove worthy of treatment, then I don't see the problem with committing to treat it, as long as you don't lose sight of the original complaint either. Obviously that doesn't mean one should go looking for incidental findings willy-nilly though.
We don't have a prior that checking her labs daily will have a medical benefit after the time we decided she should have been discharged. And there are some abnormal values in daily labs that are, at once, not uncommon to encounter and also too abnormal for the day of discharge. So we sometimes end up keeping grandma an extra day or two to give her more potassium while we have no idea what the day-to-day variation of serum potassium would be for a 'healthy' person similar to grandma.
This is a clear example of when we shouldn't order testing: there's no expectation of marginal benefit to us while there's a risk that we'll be forced to act based on the result. This is also an example of a time where a person might say, "Why aren't you checking her? What if she develops [x], [y], or [z] and you don't see it until she has symptoms?"
In my experience, getting that familiarity with a particular codebase in a way that isn't surface-level has always been a hands-on process. E.g. just because I know many general things about software, I need to know the particulars of the current codebase I'm in to know what is reasonable to actually apply to it.
This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise? Sure you can ask questions about the codebase, but IMHO that falls under surface-level information, and the devil is often in the deeper details. Hmm.
It’s harder to internalise concepts because you don’t go through the struggle of understanding them and finding the mental links you need to remember later.
I notice this with people around me - all of them are doing more things, but I am also catching more issues when reviewing docs and code.
Obviously YMMV.
But there’s a lot of pressure to get up to speed as a new hire and it’s easy to move fast with AI
I have to say on those occasional times where it finds something that I totally missed or misunderstood, those are for sure the most productive sessions. I find I'm actually working with the model, while I read the code it's pointing me at, and getting a good solution together. Often the model suggest something that's maybe too simple or, weirdly way way too complicated but it's definitely helps me zero in on a decent solution.
They tend to stick around and they engage in the problem solving on a higher level and develop a detailed picture of how the app does and should behave. So at least that part of the expertise may come from working with an LLM to solve problems.
1. "find the code that does X"
2. go read that code
3. When you hit a bit you don't care about, go back to the model and ask it for the pertinent details
4. When you hit a really confusing bit, ask the model for hypotheses about what's going on. (I always phrase it as "give me some hypotheses" not "what is going on here". I dunno if this changes the output but I think it helps me stay in a mindset of uncertainty, it's important to avoid locking in any misunderstandings. Anyway I find the models do well at this task, and when they bullshit here it has a strong smell).
Before AI, parts 1 and 3 could be insanely time consuming, sometimes it felt like a infinite breadth-first-search. And part 4 was basically: either you find a human who knows the code, or you just make a mental note and hope that later on you find something that makes you go "oh, THAT'S why they <do weird thing that should 100% have a comment>!".
So yeah even though you're still reading code with your wetware the AI makes you dramatically more powerful.
This is also extremely helpful for unpicking undocumented API contracts. E.g. you can say "the x86 implementation of this API is safe to call under a spinlock, go read the other arch versions and tell me if that's true there too".
Have you ever pair programmed with someone? It's the same idea. You can be an active enough participant in the process if you wish to be and can be just as knowledgeable even if some of that knowledge lies in transactive memory. https://en.wikipedia.org/wiki/Transactive_memory As long as you have the map, and the map to the map, you don't need to retain every fact about the landscape.
I'm going through this right now on a very difficult to implement task, the original was python and very verbose. But had facilitated a rust implementation that produced byte identical outputs. Then I asked it for what data was being passed around, placing restrictions on what passed between interfaces I could tell it what parts should be immutable and what parts should have no presence outside it's local context. Placing those limitations while having a exemplar of what it should be doing gives it little choice but to make better code if it meets the conditions set while at the same time not regressing.
I say things like 'this field is a implementation detail in a declarative data structure, it should not exist here.
This can mean hours of work with no observable change in program function, yet it is directly addressing the limitations that prevents it from being used in larger tasks.
> This is a chicken and egg problem I find hard to resolve with LLMs. If we're pushed to delegate most work to them, how do you build that expertise?
Learning from LLM written code is significantly and meaningfully different from struggling on your own or learning from more experienced human co-contributors. Especially in a large, complex, iteratively developed codebase.
Worse, once you're in that situation you are now at the juxtaposition of: "I did this. I understand what my reasoning was, and now I know why it is wrong and how to fix it" vs. "An LLM did this, I don't know why it did this, I'm not sure what it was trying to do or what pattern it followed and I'm not sure how to make it better because I didn't write or understand the original implementation either"
Is it impossible to learn and gain experience this way? Not at all. But it's definitely not equivalent.
The reason "the agent suddenly started suggesting all kinds of things to make its code more robust" is because you said you "want to build reliable software".
It's not a signal of good judgment or understanding. It's just how LLM attention works.
EDIT: I mean, those systems accumulated so much complexity around the attention based next token predictor.
1. LLM thinking 2. RLHF 3. The latest frontier models
that does anything to change this fundamental "suggestibility" of LLMs.
But who knows, maybe I'm wrong.
this feels like "make no mistakes" level of prompting. reliable software isn't as simple as making it reliable, it's about choosing the trade-offs in the areas that don't matter as much as the areas that do. if you keep prompting the LLM to make your software more robust it will keep giving you things to do. they aren't all good things. eventually you'll end up needing kubernetes to run a calculator app.
Prompting feels a lot like this conditioning phase to me. You start with an LLM in unconstrained mode, basically just a "soup" of knowledge. If you prompt wisely, you immediately condition the LLM into "your space of (domain) knowledge".
What comes out is an extended version of your existing knowledge.
Prompting is conditioning, that is what it is. The visual of a GP (like the thing you get if you google image search “Gaussian process”) is a great metaphor for what prompting an LLM is doing.
The output of the LLM is the logits which is sampled - plucking out tokens from a distribution. The input of the LLM is data which constrains the logits. That is what it is.
That’s also how you know that AI will never “solve” intelligence (the way the boosters say it will) without some general mechanism for this conditioning process. The ultimate mechanism would be embodiment; the crappy mechanism we have now is something like openCLAW.
https://x.com/DmitryRybin1/status/2079904005652893709
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
"suppose you’ve gotta resolve the $CONJECTURE, like absolutely have to, everything depends on it. think really hard, and try to come up with a bunch of ideas to try. but remember to trust yourself and not necessarily in conventional wisdom!!"
https://claude.ai/share/25740bd5-aa97-4bd7-bf58-c4df3793fda7
https://xcancel.com/__alpoge__/status/2083855298239078748
What's funny is that every other person gets a different conclusion about who these models reward/empower. I've seen people say that the generalist stands to gain the most and others say that it's the experts. Like all of life, maybe the "winner" is the person who just does stuff.
But at the top of top, the gap probably widens. A professional F1 driver will drive laps around some random guy. It amplifies reflexes etc, because at that speed little differences in timing make a big difference.
Now, AI coding isn't exactly analogous, but I think it also has these two regimes. It flattens things for simple tasks. If your task is to shovel data, do some trivial compiler wrangling staring at badly designed error messages, looking through GitHub issues hunting for the comment with many tadaa emojis to fix an issue etc, those things can now be done by anyone. Just as grandpa can also drive to the grocery store. But if you're pushing at things on a higher level, now only your above-AI ability matters. If all the things that AI can do well are subtracted out, how much other expertise do you have left? This will be proportionally a bigger and bigger difference between different people.
My working theory at the moment is that for programmers it was relatively "clean" and took the form of an inside-out transformation of the work, where AIs directly produced the central work product more or less adequately and relatively early on, but for other forms of work it will appear as some mixture of inside-out (in which case it will appear similarly first as a tool, then as something more than mere tool) and outside-in (the things surrounding their work and the supports their work processes rely on will be progressively automated). This is going to give rise to all sorts of pathologies in the white collar world, we'll get all kinds of variations on denial/negotiation, and so on, until it fully transforms the division of labor.
One interesting point of reference here: Yuval Harari gave a talk recently about the radical changes that will take place relatively quickly, in which he noted the AIs are not quite as good at writing as he is yet, although he expects they will be relatively soon. He then gave the timeline for what he considered "soon": 10 years! So we find the denial ("I still have time, they're not as good as me yet, maybe in 10 years...") even among the most vocal "prophets," among those supposedly most wised-up to what's going on and where the capability frontier lies.
The obvious conclusion for anyone is "therefore experts will remain the indispensable and specially rewarded center of the production process."
That this is appearing exactly now, and in this form, strikes me as extremely suspicious. I don't doubt the author's sincerity on the surface. What I suspect is that anxiety over the possibility that the (unstated) conclusion might be false (!) motivates the argument in the first place.
I'm asking the question, "Why is this argument appearing now?" At least one reason seems to me to be, "because we're afraid of what the world could look like if it's not true."
However, I personally agree with the author and I don't think his argument is necessarily motivated out of an anxious fear. On the contrary I think it may be motivated out of a sense of extreme exhilaration and empowerment.
Because experts (like myself as a programmer for 15+ years) who are using AI in many fields are suddenly empowered and much more useful than we were before AI. My employability and value has gone up and not down, precisely because of being able to apply my expertise with AI, which people without expertise simply cannot do. I am a professional programmer and also owner of my own startup.
Let me give you a concrete example that I am dealing with at my startup. I'm a small business owner. Before AI if i wanted to produce production quality video for marketing it would taken such a huge budget and such a large team of people (or an expensive agency) that I wouldn't even have considered it due to the enormous cost. I'm talking about Apple quality video production which takes millions of dollars to produce.
Not anymore. A single competent person with AI can replace an entire marketing video production department or agency. But expertise is key here: knowledge of film terminology to be able to describe the effect you want, and ability to use video editing tools effectively, as well aesthetic taste. I as a programmer with no filmmaking experience don't even know how to write the prompt which makes the video that i want because I don't even have the terminology. But a person with that expertise has suddenly become more employable and more valuable to my business because I as a small business now have the capability to create Apple quality marketing videos.
So AI actually created a new job for an expert that would have otherwise not existed because it was outside the budget of small businesses. Previously somebody like that would have been employable to only a few large production studios but now they become employable by almost any small business.
This is a good example of what I was pointing to with "fully transforms the division of labor."
I'm personally in a position similar to yours, but as I watch the different moves the labs make, I see the edge we've been handed (for now) also constantly under attack from different angles. This is why it seems to me that so many who can make the most of things for the moment also feel the clock is ticking.
Expertise is needed to evaluate model outputs where it can't verify itself, or at the very least one's expertise can help steer the model in the right direction.
However this is irrelevant if models themselves are better at evaluating/leveraging expertise/information.
This includes things like "before you start fixing this bug, write two tests that fail proving it exists".
Expertise is good, but a wise expert will set up methods for the machine to prove to itself that a desired result is achieved removing the expert from the tight development loop.
As the old joke goes, a mechanic charges you $5 for hitting it with a wrench and $495 for knowing what and where to hit.
I am tempted to say (uncharitably) that the 'No knowledge needed! Just add LLMs!' byline is wishful thinking by non-experts who do not want to confront the reality that they will ultimately need to learn things.
You're missing one word, and that word explains why everyone running the companies is so excited. The word is 'paying'. "But at that point, what is the point paying of you versus going to the LLM myself?"
And, yes, I think that LLMs make it a lot easier to hire a minimally trained stooge and get them productive. It's worked for me, and I appreciate being able to pretend to be productive and walk away from the job a bit early every day. I don't think software engineering is likely to be a high status, high income job for very long.
The easy, straightforward answer is "the people who own the models". Who else benefits feels like a more complex question and we'll have to see...
Someone who just does stuff still has to be able to deal with errors and failures. That’s where an expert or a generalist may have an advantage.
"LLMs reward expertise" is the title, not that "LLMs only make things possible for those with expertise"
Tao's chat was fascinating because the questions he was asking belied expert knowledge of the subject that only a handful of people could have asked.
The counterexample of the Dinitz-Garg-Goemans conjecture was basically just "keep going" and finally "enough of partial results. now finish with a complete unconditional counterexample"
https://x.com/DmitryRybin1/status/2079904005652893709
https://chatgpt.com/share/6a60b2eb-0b64-83ee-9c76-7931ca1de0...
The low hanging fruit will run short. Ultimately mathematics is a field of subjective selections of problems and proofs as beautiful and interesting. Machines absolutely will struggle with what to study, what theorems are desirable, and when do be done with a proof.
And why do you think this would be the case? I'm not talking about today but in 1-2 years. For reference o1 was released less than 2 years ago, and we've had reasonable coding agents for 9 months or so.
Mathematics is ultimately an aesthetic pursuit. Outside of a well defined goal ML models don't have any sense of taste and regardless of the scaling that's been enabled in the last year or so of capability if they haven't memorized the process of doing something they have the same limitations of inability to make choices about unknowns not trained into them.
Real synthetic intelligence seems to me to be still very far away and not a matter of making models bigger or more efficient.
Finally, we train our LLMs on who we are. Another reinforcement of biases.
(Sorry, I'm in a crappy mood, but what on Earth are we supposed to take away from this? Everyone who disagrees with you is secretly an idiot, or worse, they're smart enough to know they're idiots but too proud to admit it?)
On a more helpful note, I think your "confusion" if honest can probably be resolved by realizing that "skeptics" are not a monolith.
Then you saw how other people used Google, by filling the search bar with utterly irrelevant words, missing the one key word that's most important to what they are trying to do, then not be able to evaluate the returned search results and triage for which is most "solution-shaped", and they get drawn into wrong search hits, reading a clearly irrelevant page instead of quickly backing out to the search results page to try another page etc.
Or see how people couldn't formulate questions on StackOverflow, other than dumping a huge code chunk and saying "it doesn't work".
Now, AI makes these easier. You can now really just type natural language into the textbox, not just key words, you don't have to know about quote marks and plus signs etc. You can paste the code and say it doesn't work, and the AI just might actually spot a bug.
But having general problem solving common sense will still give you very good dividends.
Maybe the answer is more along the lines of “people are using them for different things and getting different results”?
Why does it have to be snark and “these people must be stupid”
The other day someone complained here on HN that AI failed to optimize his code speed. Turns out he just pasted in the code, didn't use an agentic harness with end-to-end benchmarking ability for the model to ground its changes in and to hill-climb on. But even as a human you need to test your hypotheses and measure things, and sometimes something you thought would help actually makes it slower.
It happens over and over, but it's no skin off my nose. If they don't want to learn to use it, it's on them.
If you don't know where you're going or how to get there, or even if you're just not paying enough attention, it will get you very far in the wrong direction before you've realised.