Posted by gumby 4 hours ago
I think if there is a major change in code review culture, it's not going to happen first in established engineering teams. It'll happen in startups that adopt a no-human-review policy from jump and then maintain it as they mature. I'm sure that's happening, but it's likely invisible to most studies and surveys at the moment. If it proves successful, and such companies start shipping significantly faster, the rest of the industry will be forced to adopt.
Code review is where many of the design & code quality problems are observed/highlighted.
It's really the smaller firms which are able to learn quick adaptations that will win out in the long-run of this thing. Legacy firms with institutionalized bureaucracies are relying on gobbling up competitors and creating vertical monopolies while pushing out the worst slop - it's why GH goes down, why Meta products are buggy as hell now, and everything feels like it's unpolished MVPs these days.
I had it do a full audit. There were about 100k lines of tests, of which it determined about 60k were near useless. Many were overly mocked, some mocked to the point that they really weren't testing anything at all. It deleted those and wrote more expansive scenario tests to replace them, caught a lot of bugs on the way and fixed those too. Total time was about 2 hours.
I think humans need to review requirements, specifications, and test scenarios. And for some industries and uses they're still going to have to review every detail. But for a typical CRUD app, I think we're at a point where we can already set the process and quality bar higher than ever, in addition to moving faster.
The calculus on a lot of these old adages is changing.
And I find that, when it reasons through and encounters some of those, it just makes assumptions rather than asking the user.
Which could be summarised by a single line: "In other words, the primary cost of change increasingly becomes feedback delay, not development effort."
The more things change, the more they stay the same.
To be fair, when I was a kid we'd connect the editor to the server using an unencrypted ftp connection and modify php files in production.
This worked perfectly fine, and phpmyadmin made schema changes super easy.
These days we do tests, schema migrations, terraform, staging sites, traffic migration, monitoring, logging, revision control, etc.
... it's a lot of boring work, and for most CRUD apps, its entirely unnecessary :)
That said: I doubt we'll abandon these practices for things that matter.
Because while modifying php files in production can increase velocity, it can also get very "exciting" if you have a few co-workers on the same project :)
We need to take back writing requirements and specifications to humans.
Adopting high level BDD style testing of human written user stories (cucumber) is probably a good idea.
Wait, no, that's ACTUAL HUMAN pensions, because plenty of AI companies are still "investment grade"
https://www.forbes.com/sites/brandonkochkodin/2026/10/12/ai-...
This would be like if we had our first factory producing cars but each car was like a die roll in terms of getting a car, a washing machine, a pile of scrap metal, or a baby. And people just asserting over and over that this was the future of making cars, no matter how much scrap metal keeps coming out.
I’ve heard people complain that human code review doesn’t reliably catch them, either. And I have seen that with my own eyes. Just not universally. Teams that have a strong culture of shared ownership and collaborating on designs regularly catch these kinds of problems in code review. If they’re really good at it, they catch them even earlier. e.g., I know planning poker isn’t fashionable anymore, but when I’ve been on teams that take it seriously they tend to be pretty good at finding minimally expensive, correct designs pretty early in the process. Which makes both coding snd code review go faster because simpler designs are easier to manage. And design flaws that do make it to code review tend to be smaller and less expensive to fix. So those teams didn't tend to bottleneck on code review in the first place.
I think that this is the kind of thing that led to folks like DORA observing that AI coding is something of a multiplier for the effects of both good and bad software development practice. If you’ve got a cowboy coding culture, throwing AI into the mix will probably just lead to agentic cowboy coding.
I actually dislike traditional code review. In part this is because most of our code review tools are very cumbersome to use (GitHub's and GitLab's web interfaces in particular).
But also because humans aren't all that great at code review when it comes to correctness and security. Part of the problem is that you don't have access to the author's mental model for non-trivial changes and have to reconstruct it, otherwise your understanding of the program logic will be inadequate.
The problem is that purely agentic code review has orthogonal problems, and typically remains subpar when it comes to judging architecture and design considerations, compared to human reviewers. Both false positives and false negatives also remain a concern.
My best results so far have been with agent-assisted code review in a proper IDE, helping you to quickly construct a mental model and to analyze the underlying logic (in addition to having agents do a first pass for correctness issues).
I actually believe the opposite: exactly the fact that a reviewer had to develop a mental model independently is why code review is and remains valuable (and why people use different models when doing agentic reviews).
The biggest value of doing code reviews is in ensuring maintainability of the code (1. someone else can understand it, 2. it is readable and 3. can be evolved in the future) — everything else is downstream from that.
I don't think we actually disagree here. I also think it is valuable, but it's also what often makes it difficult.
> The biggest value of doing code reviews is in ensuring maintainability of the code (1. someone else can understand it, 2. it is readable and 3. can be evolved in the future) — everything else is downstream from that.
Agree very much with that (with the slight caveat that it is not always the biggest value, just an important one).
But do they? In my experience, at most workplaces it's simply a meat linter phase with the occasional useless comment about style (which should probably be handled by proper CI/CD).
Like if you don't even check out the code (generalized 'you'), then they can hardly build up that model. And even checked out, viewed in its proper context, Naur's famous paper comes to mind - the author who wrote it is the only one who has properly built the theory (which might have included alternative implementations that failed but have no visible artifact).
I'm not claiming that there are no places that go absolutely above and beyond with their review sessions, but I would argue it's really rare - and AIs are great at figuring out some of the context the PR description lacks.
Code review is not really about correctness and security. We have testing and other verification process for these. Code review is about idea discussion mostly and correcting a few stuff that may have slipped through the author's initial review.
Your PR wants to change the software in some way. My review is to comment on how you've implemented that change, like did you miss some areas where you needed to change, does your description is actually what is happening (either a cursory reading or through testing), or are you doing some unnecessary change.
IMO, the initial reading shouldn't take more than 10 minutes. More than that (so a big diff) it should have been discussed upfront, and the design agreed. If it's an important area, a more careful reading should be done about the tests and relevant stuff.
But generally, if a review takes more than 10 minutes, there's some failure of communication going on (reviewer not apt to understand the change, big diffs that should have been discussed outside of the PR, obtuse code,...)
I also agree that merge requests should be short and understandable in 10 minutes, but IME this happy state often does not obtain in practice. It shouldn't, but it often does, whether it's mission creep, whether it's turning out that a change is more complex than originally imagined, or whether it is unavoidably complex and cannot be broken down further.
Part of the code review process is to verify that the writer actually did build the agreed design too though.
My company's leadership is oscillating every few weeks on AI goals. Their most recent story is "we're expecting developer velocity to approximately double over the new 2-3 years as people get better at using AI". But of course that's completely non sequitur. People on my team spend about 10% of their time writing code and 90% doing design and analysis. So if AI doubles our code writing velocity, then it will make us 5% faster overall.
Microsoft study of 484 developers, published in 2025 says 11%.
Another Microsoft study where programmers had telemetry installed showed 21%
I guess at Microsoft coding is similar to building a house in that sense. Either that or all big orgs devolve into large amounts of time spent on politics.
The point is that while good to remember, and specifically Fred Brook’s “No Silver Bullet” article about it, there are second order effects.
If coding becomes 100x cheaper, what’s going to happen next? At least to some degree, design and planning was there since coding is expensive. (Not just coding, debugging also got squashed by AI and was an expensive part of the job)
This is why the original comment seems pretty spot on, organizations will learn how to skip more phases, and startups are probably a place where this will be happening faster. (Not clearly though, a lot of big tech are happy tokenmaxxing to try and not stay behind those startups when they come)
If the cost of emitting syntax collapses without adequately resolving the essential complexity then you have not eliminated the design phase and you're just externalizing your failure modes onto downstream teams and systems.
And how many billions or trillions have we wasted on people who didn't understand that (or did, and are just busy generating hype).
Ultimately, this (llm) technology (as it is being developed/used), is not useful to us...
Google had many engineers working on a) failed projects b) people being paid to sit around.
There seems to be this strange belief on here that making something more accessible to everyone will always result in better outcomes. I don’t see this being true of software.
I’m very pro-tech. But nobody can point me to a product built by someone outside of the large tech firms, made by by llm’s that is a great value add to one’s life. Many of those were already created - they’re on the apple App Store for example.
I’m absolutely confident coding speed is irrelevant. A person who has a real vision + strategy and hand written code will beat the team with no vision, strategy and code blasted out by llm’s.
Sure llm’s could augment the former - but you’re missing the point!!
Nobody writes code in notepad anymore, even though you could and might even like all the black text.
This tracks with the actual revenue impact of AI adoption:
https://www.bcg.com/publications/2026/the-formula-for-agenti...
Their very cheerful "high" revenue increase from AI adoption (that few companies in the index manage to achieve) sits at 4.5%. Most of those in the "value generating" minority are seeing sub-1% returns.
Which, of course, is non-zero, real, and all that, and matters when we're talking about a 4-5% increase of a big company's revenue, assuming the expenses are not catching up with that.
Which they absolutely are. The AI is subsidized by venture capital, and the subsidies are drying up (hence OpenAI upcoming funding round). None of the AI companies are profitable (much less self-sustaining), and AI prices are inevitably increasing.
But how many changes are actually serious?
I can have an agent mass-produce security bug fixes, zip through our low-priority backlog, etc.
IMO - we need to get out of our own way.
Building is cheap now. Agents are also really good at following checklists and running tests.
So instead of having individual engineers reviewing code, we need to
* Leave human review for the most impactful changes
* Get better at defining the quality bar
* Continually codify human concerns into rules or systems that can be checked against
A lot of this was touched on a few months ago with Spec-driven development becoming a hot topic when agentic coding started.
We should continue that with spec-driven review, testing, etc.
Quite a lot of our bugs in production have been in seemingly less impactful changes. How do you know what's impactful a priori?
I can't figure out what you're trying to say here. Are you saying that this is acceptable? Because I very much don't think it is. You'll have twice the bugs after you're done, and they'll be inhuman ones that you have little chance of catching.
> But how many changes are actually serious?
I mean, "security bug fixes" are not on your list of serious changes? What is?
I'm no booster but this is most of what I currently spend tokens on so it's maybe not as absurd as it sounds. Maybe those gains are possible and without having to sacrifice serious code reviews too.
I've spent literal decades where one of the most valuable practices has been code reviews, for many dimensions. It is one of the best ways to ensure quality production releases. It is one of the best ways to transfer skill to junior engineers. It increases tribal knowledge throughout the engineering org by exposing others to systems they may not be working in. The number of benefits to a traditional human-centric engineering team is hard to overstate.
But I'm starting to feel that instead of starting at one extreme (heavy handed reviews) and moving towards less review, it probably makes more sense to start at the other extreme (no review at all) and move towards a brand new practice tailored for the new agentic reality.
It is looking increasingly likely that we don't need to modify code review for the agentic era, we need some completely new practice that hasn't yet been invented.
But another question is "did we build what we planned, and did we build it to a high quality". Code review partially answered that question. Adversarial review is the closest we have to that in the automated LLM world, but I feel we need something better.
Doing a line by line review as a human is silly now.
If you're right (and you probably are), we're in for a wild ride further down the quality death spiral.
The era of user telemetry and conversion tracking already decoupled product design from customer experience. No need to wait for customer's to articulate their experience or desires or for discerning analysts to amalgamate and ponder their opinions. The numbers tell you everything you need to know! Just throw 3000 pounds of spaghetti at the wall every month and wait to see what sticks.
But now you really were bottlenecked on engineering.
Only 3000 pounds of spaghetti per month? And throwing more bodies at it barely even makes a difference? Oof.
Of course the solution is to just flush pureed spaghetti-slop through the hose as fast as we can. Brilliant!
What could go wrong? The users are sure to love it. We'll know because they keep clicking. That's how they tell us the truth, even as every forum fills with louder and louder complaints!
I've been hearing this for more than a year. But if these startups are all moving at superhuman speed, a month instead of a year from idea to a shipping product, where are the products? Every time I ask that, people talk about the amazing velocity gains they experience for the thing they are about to ship. It never does, unless we count throwaway hobby projects at GitHub - by that metric, AI is adding a lot of software. And I guess never-downloaded slop-games on mobile app stores.
But if you actually go shopping for consumer or enterprise software that's camera-ready, the landscape is basically the same as in 2022.
Pi.dev and Cursor's Tab completion are two of the most innovative and extraordinary products I know in AI era, and both were built by humans, at least in their "breakthrough" stages.
Why do you feel like that? Agents have got better but better how? Looking at the frontier model output, I’d say the same myopic approach that causes long term issues without frequent course correction is still there.
Our industry has spent the better part of the last two decades interviewing with brain teasers and leet code questions. Reading code is a skill that we just weren't assessing for, and lots of people aren't really good at it.
I would argue that code review is mostly theater, politics, bike shedding and pedantry.
There is a lot of work to get away from "coding" and "programing" and get down to the actual engineering that really needs to happen in the industry.
I’ve seen a few projects that heavily rely on ai with not very good (though senior) devs driving it - and its a mess. Not the code. The resulting product. Code is actually sound as ai is pretty good now.
All because ai does code reviews and dump decisions go into prod
If you take it off too early you end up with shit and will spend weeks fixing all these baked bad assumptions your project is built on top of. I spent weeks fixing some of this shit then the LLM revived a not insignificant portion of it in a later change because it got "confused" (its words) because some of the comments it previously made were left around.
i am pretty senior. I hate code reviews so much. They are so boring, easily the most tedious part of my job. Its such a chore. Who is clinging to line-by-line review. Ppl dont really give that much about their employers.
Code review isn’t a quality gate, it’s a way to maintain shared understanding of a codebase.
Throwing out code review is equivalent to throwing your hands up and saying you don’t need to understand anything about what your code is doing in its black box.
I don’t think anyone on this forum is half as smart as they think they are, especially in terms of systems engineering a black box solution. Myself included. Someone needs to understand what’s going on inside the Widget Factory until we have full Artificial General Intelligence.
I think that is what the C-suite is currently wanting out of our engineers. Prompt, don't write, maybe read a bit, don't stop, ship, ship, ship.
What exactly would you say it is you do around here?
Why? Because I started to run into way too many AI generated messes in codebases. The agents are really good at some things and complete ass at others.
It really depends on the context. A react button background color - meh - the critical part of the backend where a new 300ms delay will cost the company millions then yeah.
If it is so critical, why don't we have tests and benchmarks for it?
Oh right, because "Greg the super-coder" never wrote any of this down and it lives in his head so he can be the arbiter of change.
It was toxic before agentic coding, it's still toxic now.
Document your systems. Add tests and benchmarks for what is important.
We've always known this, we have a dozen or more textbooks written about it. But we always gave it a pass because "Greg the super-coder" was producing more value than he was hurting.
That's no longer the case. Your best coder is now Opus or Astra
No, that's our _fastest_ coder, which is a different thing than _best_
If a performance regression like this isn't caught by an automated testing suite, you have made a horrible mistake.
these might be 1% typically
One of those things made of code that we should neither be writing nor reviewing? Shouldn't I just assume that the LLM did that or whatever?
You can fix a lot of this stuff by shepherding the whole process but it can get quite tedious. I'm impressed by the number of people that are letting Jesus (one of these days an AI provider is going to name their AI that way :) ) take the wheel and I wonder how long they will manage to support the output. So far the score isn't very good. I've gotten good mileage out of these tools by letting them loose on very narrowly scoped pieces of code that are written in isolation. As soon as the codebase gets larger than 100K LOC or so managing the whole thing becomes an exercise in putting up barriers to avoid spillover from a tiny issue into a much larger one.
That's what's happening in software. We knew that coding is only a fraction of the overall work, and even completely automating it away won't increase productivity by an order of magnitude.
Yes, the job is changing, but only parts of it. High-level engineering best practices still apply even without the coding part. And there is no free lunch.
But, even if the new engine is the golden egg, now what? You're still competing against every other team that has the golden egg... And it's right back to the driver mod, vehicle setup, etc.
Everything else is a case of project decomposition, planning and scaling up - aka this is when having code written faster is great.
The SV approach of the past decade has been about experimentation, A/B testing and data science - it has its limits.
Same goes for software. 20 years ago I got into software development for the love and passion of solving problems and writing code. Now I've been forced to pivot to a glorified BA/Products Manager. It sucks.
That I agree with, but the messes are already catching up with some early adopters and I've been watching a couple of these up close with the roughly same level of interest that you'd bring to a road accident if you were part of the technical investigation team. It is more than a little interesting to me how the difference between the initial conditions seems to be a fairly good predictor of the outcome. Great teams are doing quite well with this new tech, others, not so much. And great teams were rare to begin with.
In this case, we're looking at a picture from 2025. The models involved in the main agent results were primarily Claude 3.5 through Sonnet 4.5, GPT-4.1 through GPT-5, and Gemini 2.5. And the paper's design means you're mostly looking at the very initial effects of deployment, before companies developed good review processes.
If we weren't still in a massive hype-and-investment bubble, the burden of proof would be on the LLM manufacturers to demonstrate what deficiencies that studies like this have revealed have been fixed in their latest products.
I mention this because I've been hearing "Yeah, well, that opinion/study is completely invalid because it used a six month old tool. Today's release of SOMNOS 6.9 (Pinnacle Snoozytime) has totally changed the game!" every six months for what feels like the past five years now.
Given my own conclusion that it's not possible for conventional academic research to accurately snapshot industrial adoption of this technology, I would expect this pattern to continue for a while until either academia becomes much faster or technology progress slows down (or both). This just isn't yet a situation where you can rely on published papers telling you what to think -- you still need to mix in a bit of your own instinct and experience and extrapolation to get an accurate up-to-date read, even if there's some prediction error as a result.
> I suspect it's just not really possible for academic research
But we were told everything changed in fall-winter of 2025 and that when we got “real capable” agents/models. Whatever happened to that?
Everything did change after late 2025, but there are tiers of usefulness. Late 2025/early 2026 was the first time agents became clearly useful for personal projects. That's distinct from immediately having economically valuable end-to-end autonomous workflows in teams of median talent, which many would suggest is the threshold we've just recently crossed.
You can, but it doesn't make it correct or, evidently, yield a logical argument. Do you notice that your logic is circular? The date doesn't matter if the passage of time has not affected the outcome and that's basically what I questioned, which you refuted without any evidence. So the question still stands. Are there any comparable stats you have access to that change the picture?
What we have today is not meaningfully better than what we had a year ago, so that isn't relevant.
For this reason alone, I don't expect code review to go away entirely in most contexts where the code "matters" - in the sense that there is substantial liability to it being mis-specified at the implementation level, not just technically wrong in implementation. I do expect it to evolve a lot and become dramatically AI-assisted.
The main protection ultimately is a human being willing to put their reputation on the line by saying "I understand this change and I approve it". That, inherently, means review of some kind.
I think there's a bit of a false premise around "line by line".
I don't need to syntactically parse every line to scan it for the the type of risks that matter here. You can look high level and see the pattern and shape of the code, the type of things it is doing. If it is doing stupid things ("Wait is it making a new connection to the database inside the loop here?"). It does take an experienced eye, but with that experienced eye it is a very rapid process.
I never have operated this way. Code reviews were a basic sanity check, a chance to implement best practices, and a chance to teach these things to juniors.
If I was concerned about something breaking, I'd duscuss test coverage.
Was I doing it wrong this whole time??
That said, I think I’ve done a tremendous amount of PREVENTING bugs from being introduced via code review. Caught all kinds of weird code patterns, duplication, wrong levels of abstraction, etc. Essentially, the kinds of places that bugs like to hide and grow in.
I still think that’s valuable, and it’s the kind of thing that models generally aren’t great at, to this day. It generally relies on having a large amount of context about the project and about what’s being solved, so if you wanted to do this via automated review, you maybe COULD, but it would probably get pretty expensive, especially on larger projects.
I like an ecological-framing, especially as systems get more complex.
Yes, I could hunt cockroaches all day, but it would be even better to have caught how the new wall was missing a metal screen over a low ventilation opening.
I find recent LLMs don't do such mistakes though, in my experience. And also LLMs will find such mistakes if asked to review, if not perfect then more predictable than humans.
It's a good way to catch thinking errors, which are the insidious errors.
In order to keep some of these extremely large contracts (using Samsung as an example here) for example, we had to revert certain changes that were global but now we have a "Samsung version" because we want to maintain a large presence in Asia and Samsung leaving would be a huge blow.
At this scale, code isn't even the bottleneck, it's customer wants, politics, etc.
“But all that extra code doesn’t translate directly into improved software output on the firm level. On the contrary, the resolution rate for Issues and Epics (i.e. wholesale software features) tracked by tools like Jira did not change in a statistically significant way after AI tools were introduced (the researchers also found no “compositional shift” in the size or complexity of those Jira-tracked issues across the AI introduction).”
I'm going to make a potentially bold claim here. This depends _heavily_ on what the threshold of "rarely" is, and whether we actually _love_ to make these processes.
In the case of people before process, as stated in the Agile Manifesto, the process is begat _from_ the people in many cases. That doesn't mean it shouldn't be questioned, but when the chimney sweeps that built the process are saying "I don't want to go back to things that are similar to the way that gave me mesothelioma", I don't think it's fair to write them off as the minority.
I'm literally building a black-box client + server emulator for one of my favorite old MMOs, remotely, from a coffe shop in SF as I type this, on my laptop at home via Claude remote-control.
Insane.