Posted by tchalla 3 hours ago
In other words, the hard part moves from recall (“How do I write this?”) to judgment (“Does this actually make sense?”)
People want the "Fluent Text Generator" to think for them because its marketing calls it AI. At the end is just a tensor collection, probabilistically backtracked adjusted, text generator. It does produces fluency in the answer, but the fact that it sounds correct, to the point of passing a compiler approval, has nothing to do with been well thought out.
The code is just wrong a lot. By denying the possibility of emergent phenomemona or model interiority you just hand the hypsters an easy point to score.
The code is just wrong, all the time. You're right about the part that matters and that we can measure.
Cant answer should we without having done a lot of it before. Cant learn unless you actually do it and see the consequences through.
AI is very useful when you dont need to invent anything new
I believe the human constraint made us find ways to layer and modularize systems so it is easier to check. Now the LLM produces a lot of code and abstraction have to be enforced. Something is missing.. (resonates with the latest Pocoo blog)
I still absolutely loathe talking to them and using them. But I feel less scared about losing my career these days. Work wants me to use it, I'm using it at work. In my hobby projects I still code by hand.
The thing that concerns me, however, is that just a year ago it was only useful for doing things like reminding me how to read in a file. Now it's at the level of a competent junior, who can almost do the whole task from start to finish but needs a little supervision.
All the AI companies are retaining the messages we're sending to their agents to train the next models. I wouldn't be surprised if the agents will be able to do what I'm doing in a year or two.
UPDATE: I'm getting a lot of replies to this thread and while I'm not trying to be argumentative here, I have to ask if a lot of them aren't rooted in denial... it's been a hard adjustment process for me too.
Some things are very easy to solve via code (see https://xkcd.com/1425/), and others are extremely difficult.
For LLM based AI, it can be trained to write functionally correct code easily - there's tons of pre-existing code and documentation that can let AI learn patterns (LAMP stack, messaging queue, etc) and syntax.
Judgement is much more difficult. For some things "Should I go with a message queue for this problem?", there's a known set of solutions as to what works best in a specific situation, sure, LLMs can answer that easily.
But when it comes to turning human goals into code that does the job correctly and is maintainable and stable?
That's a heck of a lot different. Without human guidance and having tons of context gained by years of experience dealing with the very nuanced and situational judgement calls needed, LLMs are going to fail at doing a decent job.
LLMs (and junior people) don't know what they don't know. I think it will be really difficult to get a LLM to work well at a senior / staff level where it's more about coordination, working with human beings, handling situations based on legacy code, knowing what will and won't be fragile, etc.
Note that comic is about a task that AI has already conclusively solved...
Well... Now we have next years models, and people are still saying: “Last years models were no good, but this year we can do anything; so what about next years models?” And I am pretty sure in July 2027 (if this whole thing hasn’t collapsed in on it self by that point [very likely]) you all will be saying: “Last years models were no good, but this year we can do anything; so what about next years models?”
You seem to want it to collapse, but that's not the argument you person you're responding to is making.
https://hn.algolia.com/?dateEnd=1753056000&dateRange=custom&...
In particular those of us rooting for the collapse of AI were complaining about this speech a year ago as well:
It's already 100x better at writing todo apps than the average senior developer but when I get it to try and achieve something unusual using a relatively obscure library it flails.
I do think there is a very common denialism that is rampant causing many people to believe on what is essentially faith that AI is going to get just good enough to be a perfect assistant to them, but then magically halt on progress right before it entirely replaces them.
And to be clear I believe LLMs are likely to plateau before they replace literally everyone working in software development, but I also think that a lot more than n% of developers believe they will be part of the n% who will survive unscathed, where n ends up being quite a small number.
I am less optimistic than you, but would love to be wrong.
Define tasks!
The most important aspects is that they’ve always been good at applying patterns, but have no discernment into why and the specific adjustments that needs to be done for a particular contexts. It’s rare that I can’t find discrepancy at the edges if several patterns have been used.
We already have good tools for patterns. It’s called code reusability. So frameworks, libraries, code generators, paradigms, snippets, and the mighty copy and paste (there’s a reason vim has like 26 registers).
Software development is all about managing complexity, which LLM are notoriously bad at. A little supervision won’t cut it.
That, combined with knowing what to build, and more importantly what NOT to build, I don’t think AI ever will take away from us.
So true. I was too blinded to not notice the joy of my hobby getting sucked away by not doing it by hand.
My concern is how much damage could be done by someone learning 15+ years ago vs someone learning now.
That particular ship sailed more than 15 years ago. Back then if you asked people trained on high level GC-based languages about low level details or memory allocation, they wouldn't know.
The future is using LLMs for what they are good for. What that is still being found out. I've had great experience with LLMs reviewing the code (matches with Primagean's ~50% accuracy at finding bugs, which is really good) and for explaining unfamiliar concepts to me.
I firmly believe that the code itself needs to be 100% organic, and if it's not and you're relying on LLMs to generate tons of code, you haven't built enough abstraction to make it unnecessary.
They are also GREAT at writing code based on standards and patterns established by an actual human engineers. They are STILL trash at anything meaningfully complex from the ground up. I will go through hours of planning where I have to kick the hell out of its plans in ways that I don't think someone without a ton of real world experience could do. Even in the fancy new Fable models that remains true. They are not creative and any engineering worth doing requires creativity.
I think that’s the real point the article’s making. Unless you already know the ropes, you can’t tell if a plan is actually any good. I use LLMs all the time for work, but only because I get the system well enough to spot the nonsense or catch when a plan duplicates something that already exists. They really do keep rewriting the same functions over and over in the code.
Over the last four days I've few shotted a 1:1 clone (and improvement) of Fal, OpenRouter, and Replicate in Rust. It's going to be open source.
I've lined up $300k of monthly commitments for it that land at launch. Might apply to YC with it.
Moving "organic" is too slow for the world today.
The ball is moving towards open source and this will include fine tunes and LoRAs, and even more fertile growth after that. There's an abundant future market in that direction.
Several billion dollar businesses started as open source clones of existing things. Some found unique business lines, others simply did a better job.
Now the sharpening of a coherent point, challenging one’s assumptions, and editorial decisions of what (not) to include are super important because they’re no longer a byproduct of the writing process.
Much of the cognitive work in writing emerges in the labor of writing prose and challenging assumptions against the model you build in your head as you go. I've found this process is inverted when working with an LLM: it in effect emits a provisional structure first, and I discover what I actually think by finding where its output is vague, overconfident, incomplete or outright false.
Accepting surface-level coherence as finished thought is the failure mode to avoid.
1. Throw some notes, PR links, etc. into copilot and ask for a first draft
2. Go back and forth for a while asking for revisions
3. Trash all of the AI-generated text and rewrite the whole thing myself from scratch, which I can do quickly in a typing frenzy because I now have a clear idea what I want, by way of seeing what I don't want.
I was thinking more when it comes to stuff other people send me (like a freelance writer) and “editing” is basically writing the first draft because no cognitive effort has yet been spent on the piece
LLMs lie in the unhappy-medium between an abstract machine we can reason-about versus a person we can instinctively model and simulate.
Instead it's a complicated machine that evades both reasoning and intuition.
The future of software development will belong to those who can think clearly at scale, maintain durable mental models amid rapid change, and integrate machine-generated output into human-directed intent.
Wasn't this always the case? Maybe I just take this for granted because I am dumb Army guy and this is the only lens through which we dumb Army people see the world.
Whenever the subject of AI comes up in connection to programming it feels like the conversation always misses the human element. When you look at this only in terms of human behavior I am not seeing anything new with AI.
Maybe, its because I write in JavaScript and maybe its different in other areas of programming. In JavaScript it has always been a race to the bottom. The product is never the goal. The goal is always hiring and regarding code as a commodity that is designed to fail elegantly and frequently. So, when I look at AI writing code for developers I can't help but ask: What's different? Isn't that why Angular and React became popular, because they abstract away writing code?
The human element is "What do I want?" You have ideas about how you need things to work. Some folks don't care much at all about how things work, just that they can shove a thing out the door and get paid with minimal effort. Sometimes their boss is completely in agreement. Sometimes things fall apart because nobody cared enough to bother, and sometimes even that doesn't matter, and everybody gets paid anyhow.
Sometimes you wish things mattered more, and sometimes they do.
A farmer who ordered a farmhand quickly discovered he was an extraordinarily efficient worker.
The first day, he put him on sawing logs, and the farmhand sawed more logs than anybody else, ever. It was fantastic — but the wood-cutting work was all done in one day.
So, the next day, the farmer put him onto mending fences. There were all kinds of broken fences around the farm. And, again, the farmhand had all the work done in one day.
So the farmer thought, “What am I going to do with this guy?”
The next day, he took the farmhand to a basement and said, “Look, here all the potatoes that have come in from this harvest. I want you to sort them into three groups: those we sell, those we use for seeding, and those we throw away.”
He left the farmhand to it. And at the end of the day, the laborer came back and said, “Well, that’s enough, mister, I quit.”
“Oh,” the farmer replied, “You can’t quit. I’ve never had such an excellent worker. I’ll raise your salary — I’ll do anything to keep you around me.”
The farmhand said, “No. It’s all right mending fences and chopping wood, but this potato business is decision after decision after decision.”
This is the world of a manager working with a development team hah. When writing the code yourself is it not the same? If you're writing the code by hand then you're still making the decisions and still having to plan and design. I mean someone/something has to...
Ai, as a cognitive technology, has the potential to climb that hierarchy.
Yes, current software developers need to make more decisions now. But that is just until the methodologies settle.
Then it is over.
:P
Code has always been cheap, even before AI/LLMs. It’s your ability to understand your client’s problem and come up with a good solution that’s what gets you paid.
Until the people wishing it to be true are thoroughly sated.
The reality is that this is the end of programming as a large scale well paying white collar profession as we knew it.
You can't take a rapidly developing field and pretend progress is going to freeze at its current development level so you can decide how to handle it.
It's a coping mechanism to deal with rapid change by pretending change is going to stoo right here right now and you can get a handle on it.
This is wrong, code is the concrete "truth" being executed, the rest (plans, prompts, agent instructions) are just temporary artifacts used to generate the code. What's left is the code alone.
LLMs don't have any semantics, they can't execute anything with 100% certainty. So far programming languages are the only langugues that can do that.
If someone says something happens and the code says something else, the code wins.
You're implementing quicksort. The actual algorithm is a conceptual piece of math. You write the code. But sometimes the code produces output that isn't sorted.
The code "wins" in that whatever the code says is what actually happens. But the code is wrong. It must be edited to match the algorithm. The _algorithm_ is the thing that is proven correct; the _algorithm_ is what we wanted to execute. The code is a representation of the algorithm.
So code is just a notation. But one that is executable. In your example, the code is working, it’s just not what you intended. The issue is not the code, the issue is that you don’t know code well enough to “speak” your algorithm.
And that is the main issue with a lot of LLM users. They are translating specs (which they do understand) to code (that they do not understand) and then relying on superficial outputs to see if they got it right. While also telling fluent speakers that they should follow the same workflow. At least with human languages, the other party may detect discrepancy and the fact that you’re not a native speaker. But a computer will execute things blindly.
The necessary work is more than just the deliverable. Even a simple patch implies an indefinite number of other approaches not-taken, and sometimes not-taken for good reasons.
Sometimes we explicitly document those, sometimes we trust that a human expert decided against them. Those other aspects of work still exist, even if they are getting skipped/mismanaged with LLMs.
Thus the push to use LLMs has felt a little bit like being a fluent French speaker being told that all of the best French writers are using google translate to write French translated from English now.