Posted by kiwieater 8 hours ago
My thoughts on vibe coding vs production code:
- vibe coding can 100% get you to a PoC/MVP probably 10x faster than pre LLMs
- This is partly b/c it is good at things I'm not good at (e.g. front end design)
- But then I need to go in and double check performance, correctness, information flow, security etc
- The LLM makes this easier but the improvement drops to about 2-3x b/c there is a lot of back and forth + me reading the code to confirm etc (yes, another LLM could do some of this but then that needs to get setup correctly etc)
- The back and forth part can be faster if e.g. you have scripts/programs that deterministically check outputs
- Testing workloads that take hours to run still take hours to run with either a human or LLM testing them out (aka that is still the bottleneck)
So overall, this is why I think we're getting wildly different reports on how effective vibe coding is. If you've never built a data pipeline and a LLM can spin one up in a few minutes, you think it's magic. But if you've spent years debugging complicated trading or compliance data pipelines you realize that the LLM is saving you some time but not 10x time.
Examples: AI really wants to use Project Panama (FFM) and while that can be significantly faster than traditional OO approaches it is almost never the best. And I'm not taking about using deprecated Unsafe calls, I'm talking about using primative arrays being better for Vector/SIMD operations on large sets of data. NIO being better than FFM + mmap for file reading.
You can use AI to build something that is sometimes better than what someone without domain specific knowledge would develop but the gap between that and the industry expected solution is much more than 100 hours.
That is clearly false. I’m only familiar with Opus, but it quite regularly tells me that, and/or decides it needs to do research before answering.
If I instruct it to answer regardless, it generally turns out that it indeed didn’t know.
> Please carefully review (whatever it is) and list out the parts that have the most risk and uncertainty. Also, for each major claim or assumption can you list a few questions that come to mind? Rank those questions and ambiguities as: minor, moderate, or critical.
> Afterwards, review the (plan / design / document / implementation) again thoroughly under this new light and present your analysis as well as your confidence about each aspect.
There's a million variations on patterns like this. It can work surprisingly well.
You can also inject 1-2 key insights to guide the process. E.g. "I don't think X is completely correct because of A and B. We need to look into that and also see how it affects the rest of (whatever you are working on)."
"Ok let's look at these issues 1 at a time. Can you walk me through each one and help me think through how to address it"
And then it will usually give a few options for what to do for each one as well as a recommendation. The recommendation is often fairly decent, in which case I can just say "sounds good". Or maybe provide a small bit of color like: "sounds good but make sure to consider X".
Often we will have a side discussion about that particular issue until I'm satisfied. This happen more when I'm doing design / architectural / planning sessions with the AI. It can be as short or as long as it needs. And then we move on to the next one.
My main goal with these strategies is to help the AI get the relevant knowledge and expertise from my brain with as little effort as possible on my part. :D
A few other tactics:
- You can address multiple at once: "Item 3, 4, and 7 sound good, but lets work through the others together."
- Defer a discussion or issue until later: "Let's come back to item 2 or possibly save for that for a later session".
- Save the review notes / analysis / design sketch to a markdown doc to use in a future session. Or just as a reference to remember why something was done a certain way when I'm coming back to it. Can be useful to give to the AI for future related work as well.
- Send the content to a sub-agent for a detailed review and then discuss with the main agent.
The only way to make LLMs useful for now is to restrain their hallucinations as much as possible with evals, and these evals need to be very clear about what are the goal you're optimizing for.
See karpathy's work on the autoresearch agent and how it carry experiments, it might be useful for what you're doing.
Man, I wish this was true. I know a bunch of non tech people who just trusts random shit that chatgpt made up.
I had an architect tell me "ask chatgpt" when I asked her the difference between two industrial standard measures :)
We had politicians share LLM crap, researchers doing papers with hallucinated citations..
It's not just tech people.
It took a lot of back-and-forths with her to convince her that the numbers she uses every day are "Arabic numerals". Even the author of the spec could barely convince her -- it took a meeting with the Arabic translators (several different ones) to finally do it. Think about that for a minute. People won't believe subject matter experts over an LLM.
We're cooked.
It would help if you briefly specified the AI you are using here. There are wildly different results between using, say, an 8B open-weights LLM and Claude Opus 4.6.
You have to optimize your memory usage patterns to fit in CPU cache as much as possible which is something typical Java develops don't consider. I have a background in assembly and C.
I'd say it's slightly harder since there is a little bit of abstraction but most of the time the JIT will produce code as good as C compilers. It's also an niche that often considers any application running on a general purpose CPU to be slow. If you want industry leading speed you start building custom FPGAs.
You want low deterministic latency with sharp tails.
If all you care about is throughput then deep pipelines + lots of threads will get you there at the cost of latency.
how exactly you are passing data? You can pass some primitives without allocating them on heap. You can use some tiny subset of Java+standard library to write high performance code, but why would you do this instead of using Rust or C++?
Then things like the jit, by default, doing run time profiling and adaptation.
It wasn't a matter of choosing Java for HFT, it was a matter of selecting a project the was a good fit for Java and my personal knowledge. I was a Java instructor for Sun for over a decade, I authored a chunk of their Java curriculum. I wrote many of the concurrency questions in the certification exams. It's in my wheelhouse :)
My C and assembly is rusty at this point so I believe I can hit my performance goals with Java sooner than if I developed in more bare metal languages.
I've worked at places where ~5us was considered the fast path and tails were acceptable.
In my current role it's less than a microsecond packet in, packet out (excluding time to cross the bus to the NIC).
But arguably it's not true HFT today unless you're using FPGA or ASIC somewhere in your stack.
I don't work for a firm so don't get to play with FPGAs. I'm also not co-located in an exchange and using microwave towers for networking. I might never even have access to kernel networking bypass hardware (still hopeful about this one). Hardware optimization in my case will likely top out at CPU isolation for the hot path thread and a hosting provider in close proximity to the exchanges.
The real goal is a combination of eliminating as much slippage as possible, making some lower timeframe strategies possible and also having best class back testing performance for parameter grid searching and strategy discovery. I expect to sit between industry leading firms and typical retail systematic traders.
So yeah there's really no HFT anymore, it's just order execution, and some algo trades want more or less latency which merits varying levels of technical squeezing latency out of systems.
Even non-influencers are trying to exaggerate their LLM skills as a way to get hired or raise their status on LinkedIn. I rarely read the LinkedIn social feed but when I check mine it’s now filled with claims from people about going from idea to shipped product in N days (with a note at the bottom that they’re looking for a new job or available to consult with your company). Many of these posts come from people who were all in on crypto companies a few years ago.
The world really is changing but there’s a wave of influencers and trend followers trying to stake out their claims as leaders on this new frontier. They should be ignored if you want any realistic information.
I also think these exaggerated posts are causing a lot of people to miss out on the real progress that is happening. They see these obviously false exaggerations and think the opposite must be true, that LLMs don’t provide any benefit at all. This is creating a counter-wave of LLM deniers who think it’s just a fad that will be going away shortly. They’re diminishing in numbers but every LLM thread on HN attracts a few people who want to believe it’s all just temporary and we’re going back to the old ways in a couple years.
This always seems to be the pattern. "I vibe coded my product and shipped it in 96 hours!" OK, what's the product? Why haven't I heard of it? Why can't it replace the current software I'm using? So, you're looking for work? Why is nobody buying it?
Where is the Quicken replacement that was vibecoded and shipping today? Where are the vibecoded AAA games that are going to kill Fortnite? Where is the vibecoded Photoshop alternative? Heck, where is the vibecoded replacement for exim3 that I can deploy on my self hosted E-mail server? Where are all of the actual shipping vibecoded products that millions of users are using?
https://www.reddit.com/r/selfhosted/comments/1rckopd/huntarr...
One redditor security reviews a vibe coded project
The maintainer, instead of listening to the security researcher and accepting feedback about his development process, instead:
1. Denied the problem
2. Censored discussion of the problem
3. Banned the people calling out the problem
...and then when the security issues were posted more publicly and got traction...
4. Made the subreddit private
5. Wiped and deleted his account
6. Wiped and deleted the GitHub repo
7. Took the project's web site off the web
Absolutely wild and unhinged behavior.
Claude Code and OpenClaw - they are vibecoded. And I believe more coming.
Also people are using CC for the cheap access to the model, otherwise they'd be using opencode.
I note that games are mostly art assets and things like level design, and players are already happy to instantly consign such products to the slop bin.
The whole thing is "market for lemons": app stores filling with dozens of indistinguishable clones of each product category will simply scare users off all of them.
This is ditto my observation. There seems to be a certain "type" of people like this. And it's not just people looking for work.
My guess is either they have super low critical thinking, a very cynical view of the world where lies and exaggeration are the only way to make it, or something more pathological (narcissism etc).
I was listening to an "expert" on a podcast earlier today up until the point where the interviewer asked how long his amazing new vibe-coded tooling has been in production, and the self-proclaimed expert replied "actually we have an all-hands meeting later today so I can brief the team and we will then start using the output..."
With all that said, today’s LLMs do seem so provide a little bit more value compared to the bit chain thing, for example OCR/.pdf parsing is I’d say a solved thing right now thanks to LLMs, which is nice.
I can't say how many times the LLM-proposed solution to a jittery behavior is adding retries. At this point we have to be even more careful with controlling the implementation of things in the hot path.
I have to say though, giving Amp/Claude Code the Grafana MCP + read-only kubectl has saved me days worth of debugging. So there's definitely trade-offs!
That says something about how much some people care about this.
The test cases themselves becomes the foci - the LLM usually can't get them right.
No it is not.
There os no amount of testing that can fix a flawed design
Consider the the following: Unit, Integration, System, UAT, Smoke, Sanity, Regression, API Testing, Performance, Load, Stress, Soak, Scalability, Reliability, Recovery, Volume Testing, White Box Testing, Mutation Testing, SAST, Code Coverage, Control Flow, Penetration Testing, Vulnerability Scanning, DAST, Compliance (GDPR/HIPAA), Usability, Accessibility (a11y), Localization (L10n), Internationalization (i18n), A/B Testing, Chaos Engineering, Fault Injection, Disaster Recovery, Negative Testing, Fuzzing, Monkey Testing, Ad-hoc, Guerilla Testing, Error Guessing, Snapshot Testing, Pixel-Perfect Testing, Compatibility Testing, Canary Testing, Installation Testing, Alpha/Beta Testing...
...and I'm certain I've missed dozens of other test approaches.
Everyone thinks LLMs are good at the things they are bad at. In many cases they are still just giving “plausible” code that you don’t have the experience to accurately judge.
I have a lot of frontend app dev experience. Even modern tools (Claude w/Opus 4.6 and a decent Claude.md) will slip in unmaintainable slop in frontend changes. I catch cases multiple times a day in code review.
Not contradicting your broader point. Indeed, I think if you’ve spent years working on any topic, you quickly realize Claude needs human guidance for production quality code in that domain.
There’s also a big disconnect in terms of SDLC/workflow in some places. If we take at face value that writing code is now 10x faster, what about the other parts of the SDLC? Is your testing/PR process ready for 10x the velocity or is it going to fall apart?
What % of your SDLC was actually writing code? Maybe time to market is now ~18% faster because coding was previously 20% of the duration.
Absolutely. Tight feedback loops are essential to coding agents and you can’t run pipelines locally.
So instead of the 10x coder doing it, the 1x coder does it, but then that factor of 3x becomes 0.3x.
A lot of people are OCD pedants about stuff that can be solved with a linter (but can’t be bothered to implement one) or just “LGTM” everything. Neither provide value or feedback to help develop other devs.
For me it is far more than 10x, but I consider noobs by saying 10x instead of 20x or more.
When an agent takes a shortcut early on, the next step doesn't know it was a shortcut. It just builds on whatever it was handed. And then the step after that does the same thing. So by hour 80 you're sitting there trying to fix what looks like a UI bug and you realize the actual problem is three layers back. You're not doing the "hard 20%." You're paying interest on shortcuts you didn't even know were taken. (As I type this I'm having flashbacks to helping my kid build lego sets.)
The author figured this out by accident. He stopped prompting and opened Figma to design what he actually wanted. That's the move. He broke the chain before the next stage could build on it. The 100 hours is what it costs when you don't do that.
I shipped a React Native app recently and probably 30% of the total dev time was wrapping every async call in try/catch with timeouts, handling permission denials gracefully, making sure corrupted AsyncStorage doesn't brick the app, and testing edge cases on old devices. None of that is the fun part. None of it shows up in a demo. But it's the difference between "works on my machine" and "works in production."
Vibecoding gets you to the demo. The gap is everything after that.
This is the exact kind of task that LLMs excel at
Edit: It's interesting how I am getting downvoted here when pangram confirms my suspicions that this is 100% AI generated.
The author accidentally proved it: the moment they stopped prompting and opened Figma to actually design what they wanted, Claude nailed the implementation. The bottleneck was NEVER the code generation, it was the thinking that had to happen BEFORE ever generating that code. It sounds like most of you offload the thinking to AFTER the complexity has arisen when the real pattern is frontloading the architectural thinking BEFORE a single line of code is generated.
Most of the 100-hour gap is architecture and design work that was always going to take time. AI is never going to eliminate that work if you want production grade software. But when harnessed correctly it can make you dramatically faster at the thinking itself, you just have to actually use it as a thinking partner and not just a code monkey.
The final solution ended up being something like: 1. Page includes new React report widget. 2. Widget imports generic overlay component and all canned reports, and lets user pick a report. 3. User picks report, widget sets that specific report component as a child of the overlay component, launches overlay. 4. Report component makes call to database with filters and business logic, passes generic set of inputs (report title, other specifics, report data) to a shared report display template.
My original plan was for the report display template to also be unique to each report file. But when the dust settled, they were so similar that it made sense to use a shared component. If a future report diverges significantly, we can just skip the shared component and create a one-off in the file.
I could have designed all this ahead of time, as I would need to do with an LLM. But it was 10x easier to just start coding it while keeping my ultimate scalability goals in mind.
- version 2 -- we realise we're solving a completely different problem to what is needed
- version 3 -- we build what is actually needed
Additionally, the author seems to build an app just for the sake of building an app / learning, not to solve any real serious business problem. Another "big" claim on LLM capabilities based on a solo toy project.
I'm doing a simple single line text editor, and designing some frame options. Which has a start end markers.
This was really hard to get the LLM to do right.. until just took a pen and paper, drew what I wanted, took a photo and gave it to the llm
You have to figure out what you want before the AI codes. The thinking BEFORE is the entire game.
Though I will also say that I use Claude for working out designs a lot. Literally hours sometimes with long periods of me thinking it through.
And I still get a ton more done and often use tech that I would never have approached before these glory days.
I know it's not the point of this article, but really?
It's bunk.
It could have been a web app, but with NFTs and Farcaster miniapps, you market to people who are willing and able to spend using their wallet instead of asking “normies” for credit card information for a 2 dollar custom image (that you could also prompt out of a free Gemini session).
With Farcaster, you also already have the profile picture of the user, one less hurdle again.
AI can be really attractive to bros but also be incredibly useful.
In other words, AI isn't a trend that's going to pass, it's permanently going to reshape the tech scene and economy in a way that cryptocoins and NFTs absolutely did not.
This exact wording was used for crypto. "It isn't a trend that's going to pass" and "It's going to reshape everything." Why are we sure of it now for AI (and that we're going to be right), when they were also sure of it before for crypto (and they ended up wrong)?
The AI people have the exact same feelings of absolute certainty as the crypto people had.
Probably zero grandmothers outside the west, and very few grandmothers within the west, know what NFT even stands for.
Are people just expecting there's going to be enough digital fools to make a market?
A movie can be duplicated indefinitely. There's no guarantee your song will be appreciated as art. I'm not sure why you say you can't print out an image and hang it in your living room; we do that all the time at home.
I've personally never dabbled in NFTs, but I don't think it's fair to ascribe the inherent conflict between information and scarcity uniquely to them.
There's some 80-20:ness to all programming, but with current state of the art coding models, the distribution is the most extreme it's ever been.
When we start selling the software, and asking people to pay for/depend upon our product, the rules change -substantially.
Whenever we take a class or see a demo, they always use carefully curated examples, to make whatever they are teaching, seem absurdly simple. That's what you are seeing, when folks demonstrate how "easy" some new tech is.
A couple of days ago, I visited a friend's office. He runs an Internet Tech company, that builds sites, does SEO, does hosting, provides miscellaneous tech services, etc.
He was going absolutely nuts with OpenClaw. He was demonstrating basically rewiring his entire company, with it. He was really excited.
On my way out, I quietly dropped by the desk of his #2; a competent, sober young lady that I respect a lot, and whispered "Make sure you back things up."
Honestly, seeing all the dumb code that it produces, calling this thing "intelligent" is rather generous...
I suppose if you are doing something that truly can be decided based on a test but, I just don't see it, at least for anything I do.
If I'm reviewing all the code, so far I'm still the bottleneck even with a single agent and I don't see an easy way to change that.
I needed it, I quickly build it myself for myself, and for myself only.
The average user just has no interest in building things.
There are well attended parent evenings in our school on that topic.
Thinking about it, we should turn these into vibe coding hackathons where we replace all the ad-ridden little games, learning tools, messengers we don't like with healthy alternatives.
That would be good news, but I doubt most people will do things like that.
You really believe that?
Unless LLMs can read minds, no one will bother to specify, even in plain english with the required level of detail. And that is assuming the user has the details in mind, which is also something pretty improbable...
What about this is new?
Sitting down with a child to teach them the very basics of javascript in an hour? Trivial.
Needing Claude to do it is kind of embarassing, if anything.
You can learn to bake good bread. It’s not _that_ hard. And it’ll probably taste better than store bought bread.
But it almost certainly won’t be cheaper. And it’ll take a more more time and effort.
Still, sometimes you might bake your own bread for kicks. But most of the time, you’ll just buy the bread someone else has already perfected.
I can have fresh bread anytime I want from a handful of nearby stores.
It was: "With sufficiently advanced vibe coding the need for certain type of product just vanishes."
If a product has 100 thousand users and 1% of them vibe codes an alternative for themselves, the product / business doesn't vanish. They still have 99 thousand of users.
That was the rebuttal, even if not presented as persuasively and intelligently as I just did.
So no, it's not the case of "both things being true". It's a case of: he was wrong.
One of the most valuable things about Slack is the ecosystem: apps, API support, etc. If you need to receive notifications from external apps (like PageDuty or Incident.io or something like that), good luck expecting them having a setup for your own version of the app. Yeah, some of them provide webhooks (not all of them), but in the end you have to maintain that too...
Also though, I feel like being attached to Confluence helped it because there is a lot less competition in the world of documentation wikis than there is in task management.
The things that are going away are tools that provide convenience on top of a workflow that's commoditized. Anything where the commercial offering provides convenience rather than capabilities over the open source offerings is gonna get toasted.
An example would be if the user only ever works with .jpg files, then you don't need to support any of the dozens of other formats an image program would support.
I cannot stress enough how many software users out there are only using 1-10% of a program's capability, yet they have to pay for a team of devs who maintain 100% of it.
They just want the software to do the few things they need it to do. AI labs are falling over themselves to remove the gate keeping regular people from using their computing device the way they want to use it. And the progress there in the last few years is nothing short of absolutely astounding.
Yet, all the astounding progress notwithstanding, I don't have a suite of bespoke tools replacing the ones I depend on. I cannot say "hey claude, make me a suite of bespoke software infrastructure monitoring and operational tooling tailored to my specific needs" and expect anything more than a giant headache and wasted time. So maybe we just need to wait? Or maybe it's just not actually real. My view is unless you show me a working demo it's vaporware. Show me that the problem is solved, don't tell me that it might be solved later sometime.
I could certainly imagine building myself some sort of dashboard. It would seem like a prime use case.
You want to hear about a problem solved? Recently I extended a tool that snaps high resolution images to a Pixel art grid, adding a GUI. I added features to remove the background, to slice individual assets out of it automatically, and to tile them in 9-slice mode.
Could I have realistically implemented the same bespoke tool before AI? No.
Let's say I emit roughly 1TB of telemetry data per day--logs, metrics, etc. That's roughly what you might expect from medium sized tech company or a specific department (say, security) at a large company. There is going to be a significant infrastructure investment to replicate datadog's function in my organization, even if I only use a small subset of their product. It's not just "building a dashboard" it's building all the infrastructure to collect, normalize, store, and retrieve the data to even be able to draw that dashboard.
The dashboard is the trivial part. The hard part is building, operating, and maintaining all the infrastructure. Claude doesn't do a very good job helping with this, and in some sense it actually hinders.
EDIT: I'm not saying you shouldn't take ownership of your telemetry data. I think that's a strategically (and potentially from a user's perspective) better end result. But it is a mistake to trivialize the effort of that undertaking. Claude is not going to vibeslop it for you.
I tried it works wells. I can do the same thing in my Linux machine, but even my 12 year old now can get perplexity to build him a tool to compare ram prices at different chinease vendors.
Could you do the same in eg. Photoshop? Maybe, but even if, you would need to learn how.
LLM's change the calculus of the above chart dramatically.
The difference I've noticed is that the act of actually typing out code made me backtrack a few times refining the possible solutions before even starting the integration tests, sometimes before even doing a compile.
When generating, the LLM never backtracked, even in the face of broken tests. It would proceed to continue band-aiding until everything passed. It would add special exceptions to general code instead of determining that the general rule should be refined or changed.
The reason that some devs are reporting 10x productivity is because a bunch of duct-taped, band-aided, instant-legacy code is acceptable. Others who dont see that level of productivity increase are spending time fixing the code to be something they can read.
Not sure yet if accepting the spaghetti is the right course. If future LLMs can understand this spaghetti then theres no point in good code. If we still need human coders, then the productivity increase is very small.
That is pretty bad..