Posted by soheilpro 2 days ago
On our own 1.5M loc system, we've found and fixed thousands of bugs, security holes and circular dependencies in the last year. Code quality has increased, test coverage increased. We've also recently set our compiler to treat warnings as errors, and to be as strict as possible. It is absolutely amazing to see the gaps being filled, it would've been an impossible task for human devs and IDE's would've made it very uncomfortable. We've also ironed out all the bugs and failure modes in our infrastructure automation / monitoring. Our next project is to eliminate all 3rd party dependencies, and after that to rebuild our frontends completely using zero frameworks, not touching npm ever again after that.
So when I read this about postgres, I was very happy that they found and pursued this side quest. They can make the codebase bullet proof.
I could imagine scenarios where only well off developers and companies could afford that.
Then if you don't have the money, you are out of luck and risk having vulnerabilities in your code.
And it takes just one adversary to pay the bot & the bot will happily find an exploit for them in your code.
This worry this could lead to a serious power imbalance & furthe damage the free software community...
Would you mind elaborating?
Genuine question, because I don’t know: Wouldn’t it suffice to vendor your current packages and “only” stick to maintenance from there on out, instead of rewriting it all?
This should still achieve the goal of not needing to touch npm, but presumably with less effort/tokens, right? Or is it just that you would expect the result of that process to be less clear and maintainable than a full rewrite?
How's that going? Any lessons learnt you'd be willing to share?
Before we started this journey (about a year ago), we spent a ton of time cleaning up our codebase, splitting large files (10K+ lines) into files that are no more than 2K lines, things like that. Then we let claude scan our important files and summarize them, add comments for future claude sessions. So we've let it built up tribal knowledge per file and the reasons for things to exist. We tell it all about our operating environment, the country and legal framework etc, so every little piece helps it stay honest and on track.
So to start, do something simple like: Scan the files in this folder for database code and check for queries that loads the entire result set into memory and/or queries that loads all columns. Suggest which queries we can filter better or reduce columns being transferred. Once done, also check real db row counts, compare them with the queries in these files and check if they might benefit from an index or partitioning/sharding. Also check for queries that loads data but their results are not used. Don't make changes yet, just do a deep investigation and report back. Feel free to spin up muliple agents to batch the scans.
edit: it goes without saying you need a language with strict typing and a good compiler (we use dotnet). Doing this on python/javascript would be a nightmare.
This seems…good?
That it might disrupt the project's "traditional" release cadence a bit doesn't seem terribly bad in light of the alternative, which is shipping another major update with bugs which have apparently haunted the codebase for some time.
We will hopefully get over this wave and be able to move forward with much more secure software, but it's crazy right now.
Bottom line: it's a good thing. My software will be better after that. Even if it's currently not good for my ego (and my health). But I'm glad for all the tools that help me improve quality.
And sure, I can just ask LLMs to fix the bugs, which works well half of the time, and produces bad slop in the other half. This is where human expertise and judgement is still necessary to avoid letting our code bases drift into LLM spaghetti hell.
While I did fix most of the things because while they might not have been exploitable they were bugs waiting to happen, in all but one case I haven't been very impressed by any of the LARGE IMPORTANT SECURITY HOLES that were found.
But that doesn't matter for me. It shows me real bugs and less than 1% false positives. Some bugs are truly nasty and urgent, most are trivial and not exploitable at all, but they're bugs and so I'm fixing them.
Mythos isn't mythically superb for being the best LLM model, but only for not having these annoying guard rails that make all others randomly refuse to cooperate when you need them the most. Whoever invented guard rails - please stop! - they just slow the good guys down in the race.
> This has been stressing me out
> the bugs follow me in my sleep
Sounds like you need some vacation.
i would say the recent slew of vuln reports/breaches making the news rounds are doing work!
This is the silver lining i suppose!
LLMs finding unpatched vulnerabilities faster than they can be reasonably patched, bringing all other software development to a halt.
If you consider it to be productive work, it's a massive productivity boon happening right now. Of course, product managers might disagree...
Imagine that the whole city has to install iron doors and window grilles, because wooden doors and glass windows are not safe enough anymore.
That's a lot of work, but is it productive in the end? Probably not.
With AI it's the same. World was fine with lax security approach before. Today it is not fine. Tomorrow we might need to prove every code line before shipping, will that be more productive? Hell, no.
> That's a lot of work, but is it productive in the end? Probably not.
Your imaginary scenario defines the solution to be pointless. If that same city was subjected to weekly riots smashing windows & doors, and people regularly died, stopping "productive" work to install things that keep people alive and productive? That's productive.
One way to think about it is that production allows you to pay for stuff. Not necessarily directly but by way of trading. It is the output you use to buy things like security.
Consider an apple farm. What does the apple farm produce to pay its bills? Apples. They could also dedicate resources to pears, &c, and that would add to their production -- to the goods they produce, the goods with which they (indirectly) purchase tractors, fence posts, &c. They could also dedicate resources to security -- more fences, &c -- and this may even be prudent. Maybe thieves are getting taller. This doesn't really add to production, though; at best, it maintains production at the status quo ante (assuming the fence is effective).
The software is better in the end but not all of it will be productive.
If that same city was subjected to weekly riots
That's productive.
The point it, this was not happening before. LLMs made cheap cyber attack possible. But it didn't make the implementation of fixes correspondingly cheap. So now everyone needs to spend extra money/effort to install grills to cover where LLMs fall short.
For postgres, this absolutely makes sense though as long as people are willing to shell out the profits from LLM use to support postgres development. That is one way this could workout for everyone..
Then they have a pikachu face when someone smashes their car window to steal something worth $50 they left on front seat because “who would do that?”.
What is even better thief will most likely pawn this for $10 but for him it is effortless $10.
Obviously this doesn't work on the internet since most hacking comes from overseas out of our control. Which makes me wonder if there is a possible future where we just block packets coming from countries that don't lock up the hackers.
Not obvious to me. I am generally a fan of Cyberspace manifesto and the free Internet, I think it's great people can reach outside their ingroup and talk to strangers and ostensible enemies.
But, that doesn't mean we should give up physical security, the guarantee that the connecting party comes from a certain physical location (state, company). I think allowing connections from anywhere by default is fundamentally flawed security architecture, for certain things.
It becomes productive only once your business stops for days because somebody brought it to a halt by breaking into the servers and deleting them.
The algorithm is: if the threat model is valid, there are vulnerabilities and you either have some software everybody knows about or a gang of hackers specifically targeting you, you should patch.
The part of the industry that profits from security theater will want you to patch no matter what, even if your business goes down in flames due to downtimes and new bugs introduced by patched components. They won't remind you to assess the threat model first.
I don't think that's true.
In medicine, science often doesn't know how specific drug really is working. They just do a lot of tests and if these tests are positive enough, they ship it. I think it's good analogy. Biology is very complicated and it's out of our grasp to truly understand how various parts of it interact with each other on so many levels. But if something helps, it's stupid not to use it just because we don't fully understand it. Sometimes it bites. Sometimes it bites 30 years later. But we still follow that approach.
Software is similar. The complexity of the systems we build is enormous, when we're taking in account all the moving parts. I just read that someone emits radiowaves toggling memory bit and ex-filtrating information. How can you account for that? Probably you can't. But we still need software for good things.
You have a perverse view of product ivity
Some people will much appreciate the holes in Postgres that are being plugged. Many more will be bored with patch releases until new features rollout.
A software affecianado appreciates patches, and might do well to teach laypeople the benefits of a good defense.
Also, in the article they mention a significant number of patches for older versions as well.
It's not that the LLM code is that great, but that many overestimated the human code and the human review.
I think comparing the resources they spent at it would be a more apt comparison.
AKA if you tasked 150 mathematics departments to only focus about that problem, and stop doing things like teaching and researching anything else, you might get close to the amount of resources that these companies put into solving the problem to get a press release.
Also: most tests are positive tests. Tests rarely observe the negative space: that which is there but shouldn't be there.
looks at particledb
I wouldn't be shocked if in 1-2 years non-security vetted software released into the wild will be hacked on arrival, like Windows XP without service packs used to be.
Human-written code isn’t inherently buggy, and LLM-written code isn’t inherently bug free. What is irresponsible is shipping code you don’t understand, and that will only increase with LLM-written code. A major reason why so much of the world’s software sucks is precisely that we think of it in terms of what’s “economical” and not what’s good or right. What we need is to slow the fuck down and stop treating every facet of life like a transaction. Software is filled to the brim with dude bros who “move fast and break things” and actively exploit people as their business model and somehow are called “the brightest minds of our generation”. Our priorities aren’t right.
And even if the impossible occurs and LLMs hack everything that's connected to the Internet, so what? That would be inconvenient but it wouldn't cause doom. People would muddle through and eventually fix the critical problems, just like they always do.
What makes you think that AI trained on slop is suddenly going to produce quality code?
But any kind of mistake, any kind of ambiguity and you'll be in for a wild ride. It usually takes a while for the rot to set in. I've had to revert - more than once - to many commits prior to undo a bunch of work that had me completely puzzled as to what the reason that particular fork was taken to begin with. Usually I end up figuring it out but definitely not always. In some cases this kind of wild modification fed back on itself until the situation was beyond rescue. I'm a bit quicker with such reverts now than in the past, for the first year or so I would try to save what could be saved, now I just restore and try again.
But if you're not sharp when evaluating what gets done there is a good chance that you'll miss it and then it can pop up way later.
What I've noticed as correlation (but I'm not sure if that is the root cause) is that non-local effects and stack effects are particularly hard as is everything that requires a working model of the world.
edit: this comment seems to make people uncomfortable.
Harness, prompt etc etc are all peripheral IMO, though people like to think their super long prompt, custom harness and MakeNoMistakes.md skills are what contributes to their special skill with AIs, but really models are just improving.
You can just trust the AI to generate a perfect answer I suppose (and why are humans slowing it down at all if so?), but if you don't expect that yet you have to put in the work.
You would think so but medicine has faced this paradox since the development of modern testing technology.
You would naively think more testing is always better. More sophisticated practitioners then talk about Type I vs Type II errors and the problem of false positives.
But even if you were able to come up with tests that had 100% accuracy, there are times when you don't want to run that test on someone. The key distinction is that people only ever die of one thing and having a tumor that doesn't kill them is often better than risking the surgery to remove the tumor.
Similarly, there are some bugs you don't want to find since the harm from the people it affects is smaller than the opportunity cost of fixing it. Ignorance of the bug is better than knowing about it and having to make an affirmative decision not to fix it.
Logically, a bug that prevents someone from doing something and a missing feature that prevents someone from doing something are equivalent, they shouldn't be thought of as two separate activities but should be evaluated via the same framework. The list of all defects in a piece of software is infinite so it becomes impossible to erase all defects. It's about prioritizing with finite resources.
> You can't predict the harm from a small bug when AI can chain complex exploits and when your software is used in mission critical applications as a data store.
Yes, any bug can theoretically be part of an exploit chain, just like any tumor detected can theoretically be the tumor that kills you. This is why the detection paradox exists. Once you become aware of the tumor, you face a duty to try and mitigate it. But we know from autopsies that people die with all sorts of tumors that didn't kill them.
Similarly, if we bothered to do autopsies of software products after sunsetting to discover a comprehensive list of CVEs, we'd discover a ton of bugs we were better off not knowing about because they ended up not making an impact but we would have diverted resources if we knew about them.
This is not an argument that we should remain totally ignorant of all bugs, but a refutation of the "common sense" belief that more knowledge is always better.
I guess it really depends on the severity of the defect. Would it allow some really significant exploit? -> Important to fix.
Does the defect only prohibit one very esoteric use-case which basically never comes up (and was thus never reported by a human user and only now that we have an infinitely patient machine checking every completely unlikely code path is it detected) -> more productive use of your time to develop a new feature which would be used by more people.
Firstly, the cost of fixing a bug once you know it exists is usually minimal compared to the cost of discovering that bug. This is the inverse of medicine, where the cost of testing (at least in terms of health outcomes for the patient) is typically minimal compared to the cost of treatment.
Secondly, if you have an asymptomatic illness, this does not typically make you more vulnerable (hence why it's asymptomatic). Like you say, you typically only die of one thing, so if you know the patient won't die of a particular diagnosis because something else is more likely to kill them, then it might not be worth making that diagnosis. But for software, you can be simultaneously vulnerable to lots of things at once, and any of these vulnerabilities can be the one to "kill" you. Some vulnerabilities may be easier to find and exploit (or do more damage) but in principle any active vulnerability can be exploited.
Thirdly, having multiple vulnerabilities makes software more vulnerable overall, because those vulnerabilities can potentially be chained together to create more serious exploits than each vulnerability would have represented on its own. To a certain extent this can be true in medicine as well, but generally it isn't.
Fourthly, we can't cure death (yet?) but we can cure all the vulnerabilities that we've found so far. There is no known upper bound on the lifespan of a piece of software, and we know that older code tends to have fewer bugs in that newer code, presumably because those bugs have been fixed over time. This is different from health issues, where a given condition may be chronic or terminal and therefore at a certain point there's nothing you can do to fix it.
Finally, ultimately in healthcare it's your body, and your decision was to what you do with it (assuming you can find a doctor willing to do what you want). If you have a condition and you want it cured, even if it is completely benign and will have no impact on your life whatsoever, then you can do that. (Again, assuming you have the resources and the condition is curable). But a software vulnerability does not affect you as the developer of the software, it instead affects your users/clients/customers. Therefore there are different dynamics in play. If I sell you a tool and I know it could be vulnerable, but I don't try and find out what those vulnerabilities are, then you're ultimately the one on the hook when all your data gets deleted or leaked.
Boy, I wish!
> Like you say, you typically only die of one thing, so if you know the patient won't die of a particular diagnosis because something else is more likely to kill them, then it might not be worth making that diagnosis.
You can't know someone didn't die of X until after they died, that's the entire point of this dilemma.
The key thing you forgot, which makes this approach non-sensible, is the omniscience, without which you have no clue whether your posture towards a new bug-finding tool is good or not, since you don't know whether it will mostly find those bugs that are not worth fixing and the time wasted on fixing them is detrimental.
Similarly, how is ignorance better than informed decision when ignorance also includes missing critical security bugs?
This isn't an argument against all automated bug detection, just that it needs to exceed a threshold beyond just "It has found at least one real bug not found via other means" which is the seductive threshold people want it to be.
That very much depends on got you define “bug” and how you specify your product requirements. Also, you have to assume the specs are correct.
We want software that remains robust in the long term, so that even when people are able to inspect the source code, it's still solid, rather than hiding the source code and hoping people don't find holes everywhere.
"We found a bunch of ways to make our product safer and better. We need more time to ship a higher quality product."
PICKET SIGNS
No, they were not. The problems they faced were absolutely real, but the solutions they chose to attempt were stupid and counterproductive. Which is why they lost so hard, and it was not even a contest.
Socialism movement that refused to use machines and factories would inevitably lose to capitalists that embraced them.
Socialism movement that refuses to use AI will inevitably lose to capitalists that embrace it.
> In the end, I don’t think that the delay in shipping PostgreSQL 19 is particularly concerning. It is not a long delay in the grand scheme of things, and it emerges directly from the project’s concern for quality.
So unless it gets outlawed, I don't see people opting out of it
The article also concludes that overall, yes, finding more bugs is a good thing. Basically saying it’s the LLMs “fault” that the quality is improving, alongside a slight delay to the release.
[0] Write skew observed under serializable isolation: https://www.postgresql.org/message-id/flat/CA+COZaBo+ZpKgMvx...
[1] Serializable anomaly: duplicate primary keys: https://www.postgresql.org/message-id/flat/CA+COZaBOiiRPmEfX...
[2] SERIALIZABLE can commit write skew after SSI summarization: https://www.postgresql.org/message-id/flat/CA+COZaCtK=UQbeQw...
[3] Fix temporal foreign key validation of pre-existing rows https://www.postgresql.org/message-id/flat/CA+COZaChq_ukm2o7...
[4] Lost lock during toasting allows FK violation https://www.postgresql.org/message-id/flat/CA+COZaC-2-GQabq+...
[5] Partition pruning can incorrectly exclude a RANGE default partition https://www.postgresql.org/message-id/flat/CA+COZaDXrfTaBjLE...
[6] timestamp vs. timestamptz comparisons inconsistently ordered under DST : https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
BTW, the Postgres community is also quite protective of their culture, so I expect this welcoming nature of the community to persist for a while.
question is, how long do you think this persists before there's only a long tail of minor exploits left?
SQLite has gone into a frenzy of bug fixing this past year, changing long stable processes to ease things up: opening a specific report channel just for these kinds of bugs, adding features to Fossil to help manage things on their end (and that's just the visible stuff).
And with all that, the latest release has unfortunately been the buggiest I've experienced so far.
I skipped the .0 because of a crash, and only released .1 after giving up waiting for a .2 that never came, with a patch to fix a serious performance regression.
I spent a month supposedly shipping old bugs no one had a problem with, rather than shipping new bugs that instantly crashed things or caused measurable regressions.
That's on a software with stellar testing, and a good engineering team known to favour stability over the new shiny.
I'm tempted to say "no, it isn't". But I don't have numbers to back that up
That era is over, compilers are used for almost all software, eliminating entire categories of bugs.
(And hope performance stays in the same ballpark as the old code.)
I suspect both old codebases and new ones that have a very intentional architecture are both dangerous to let loose.
The agent doing the fixing must have not only an incredible map of the project and how it is put together, but the ability to reach into annotated examples in the past and refer to them as it goes about a fix.
I’m seeing that frontier models will not follow intended patterns or know when to deviate if their primary charge is “fix this bug.”
It is perhaps a better idea to have agents’ primary job be “maintain this codebase” and one way that they do it is to fix bugs, and “oh by the way here is a bug.”
(Performance can be part of 'correct', and can be just as hard to actually get right in specification!)
Very curious about the details and why it’s unfixable. Is it just because timezones are such a mess it’s logically impossible to satisfy every constraint?
[0] https://www.postgresql.org/message-id/flat/c22286ad6268347c6...
There are probably real world cases where dropping the TZ/offset provides meaningful performance/storage savings, but it hasn't been the typical case in my current sector.
At least they are UTC and not EST or PST...
This seems rather extreme. The pathological case only seems to make a given timestamp ~1h bigger, so you could still use an index scan, just with a bigger input followed by a filter that removes the timestamps that shouldn't be in the query.
Step 5: Do you fix bugs before writing new code?
(now i feel old)
I've worked for CEOs and managers that would insist on ridiculous ideas. Like that time a CEO (let's call him Cedric) came in with a printed powerpoint slide that had a screenshot of the website with a "login form" drawn on top of it.
Cedric: "I need you guys to put this onto that scrunt board thing and make this before friday".
Me (tech lead at that time): "But what features would users need login for? Where or whom needs the profile data? We'd need to work this out in detail and plan it. Certainly not even this month".
Cedric: "I don't know, it's for you to figure out". and then "It can't be hard, it's just two squares one for a username one for a password" (he forgot about the submit-button, the third "square", he didn't even fathom the idea that something has to lie "behind" the "squares").
I recall this anecdote whenever I feel anxious about whether soon "stakeholders" can just chat to an AI agent and have it build features, rather than hire me to do so. Sure, a lot of them will be able to. But there's always Cedrics. Those who not only fail to communicate their ideas in a way that even an AI agent would understand, they lack any understanding of what tradeoffs of their requested features are.
The Cedric's would be happy with an AI agent that complies and starts churning away. An agent that blindly accepts any request and builds feature upon feature. Making their product worse in every sense: unusable, unmaintainable, debt-ridden, security-nightmares.
Their agent will build features that are plain illegal. Or features that users hate, or confuse them. It will oblige requests that drive users away, or changes parts that users/integrations rely on. Their AI agent will gladly add a "form" to extract all PII from their database, but on request, will remove the authorization from it. Because "you are absolutely right. If we don't link to the page from the menu, no-one can reach it".
I'm not making the last one up. Another "Cedric" alike recently hired me to look at his vibe-coded app because his go-to developer gave up and dropped the project. It had loads of such extractors, dashboards and overviews. Online. Unauthenticated. His claude, however, did add the pages to the sitemap, so a customer found their private data on google and contacted "Cedric", after which he asked me.
No, AI is getting smarter, but as long as the "Cedrics" remain as stupid as ever, I don't fear for my job.
“Find the issues in this code” vs “write this feature” are different problems.
The first one is easier because the feedback loop is llm to code/test and trivially scalable.
The second one is llm to human which is going to be slower because it can’t read your mind.
The tighter and more deterministic the loop the more llms excel.
make a bunch of "build the feature" vs "heres a detailed spec" prompts, and see which one does better.
if you take out the AI boogieman, and instead go with "next token prediction" then "people are doing it wrong" is much more reasonable
For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly.
NP-Hard, even[1]!
[1] https://www.i-programmer.info/news/112-theory/3830-physics-i...
AI’s only part of this was helping review things and finding bugs. More bugs, bugs that would have been found later, whatever.
Postgres did not have a slop issue and it’s wrong for people to insinuate it.
any kind of pattern matching / transformation is going to be an ideal task for LLMs
A pattern I'm seeing w/ companies and projects not used to continuous delivery (or at least shorter release cycles) is that they are slipping in schedules more now.
I think in the current environment it makes a lot of sense to re-think how often releases get cut and how to handle security related updates.
When you do only one or a few releases per year everything is 'higher' stakes and more bureaucratic.