Posted by taubek 20 hours ago
Otherwise he's just going to replicate all the legacy bugs and questionable design decisions.
Right? ...
[1] https://github.com/enterprisequalitycoding/fizzbuzzenterpris...
Anyone who has used the built in date picker knows why everyone rolls their own.
It's a bunch of text files in a repo with massive churn (all new version 5!), but somehow even the technical debt 'skill' (i.e. text file) has technical debt itself:
https://github.com/DietrichGebert/ponytail/blob/main/skills/...
> ...every ponytail shortcut is marked with a shortcut: comment ... (older code may say ponytail:).
It doesn’t matter if it’s satire. Developers seem to be so vulnerable to all these AI promises of ‘automating’ everything.
BTW it’s terrible. It gave two very hollow suggestions that did not solve the real problem. Hour wasted, angry colleagues, no code change.
Legacy code is typically a euphemism for old unmaintained code, no one knows how it works so it can be edited, and it probably doesn't align with current work practices.
Not to say a rewrite is necessary right, but avoiding rewrites of working code is how we ended up with more COBOL code than COBOL programmers in 2026.
I think people are assuming the worst but AIs most helpful contribution beyond churning out code is explaining the code base.
Who knows if they will do a good or bad job that's down to the engineers and not the tools imo.
Once AI came along, I became like Captain Picard (RIP) and would just sit back and let the AI write all unit tests. "Try harder, make sure coverage is 100%". It was very exhilarating to sit back and watch the models churn away, creating 100s of lines of unit tests. Of course, I checked them all manually before submission ;-)
That said, there's also research pointing to LLMs encoding existing bugs as test requirements: https://arxiv.org/abs/2607.05139
That CS PHD researcher, Konstantinou, in particular, has written a lot on this: https://arxiv.org/search/cs?searchtype=author&query=Konstant...
The issue with using "the code as the spec" is that humans can't read 100K lines of code in a day, can't hold the entire ball of unstructured mud in their brains all at once, and generally require hierarchical "structure".
Modern AIs just don't have these issues! Their context windows are big enough to encompass an entire codebase, so they can "see everything", all at once.
Each line of new code they write is compared against every relevant line of code in the original, which they treat as a spec.
My main issue in my experience is that that AIs tended to reproduce the legacy behaviour "bug for bug", which was not my intention.
Similarly, the AIs were hamstrung by external constraints: I typically prefer to leave the DB schema unchanged, which meant the torturous workarounds were needed for design errors in some stored procedures, etc.
The older the code, the bigger the issues with baked-in assumptions about transactions, stateful sessions, data types, string formats, etc.
This trips up AIs just as much as humans. You can't trivially maintain backwards compatibility and modernise the codebase! Those are conflicting goals, no matter what.
Surely we have more code than programmers for every language in existence?
Except for the part where the concept isn't even well-defined, this sounds like a complaint that the world contains more bricks than bricklayers. Of course it does.
-Works according to a defined specification
-Within the bounds of that specification, has a lot of code written to optimize speed, cost, latency, etc - all measureable goals
-Written in c, with all it's weakness
-Given limitless engineering work, some could be ported to better hardware
-the new system could be automatically compared to the old system for functional correctness
All those factors fit well with agent based development.
Btw, many of them are true in general to embedded systems.
Then consider that not all vendors implement the standards identically, because the standards can be a bit woolly and in some cases followed rather than lead live implementations.
There will be a bunch of code that looks useless, but is actually there because of specific interoperability scenarios.
I wonder whether AI is any good at looking at that code,running the system, and documenting those edge cases?
Lots of this stuff should work in theory, you only find out that it doesn't in practice when you try it on real hardware. And there are thousands of bits of hardware.
This whole thing sounds like a deluded vanity project birthed in an ivory tower.
But yes, fact it uses a bunch of very well documented standards with multiple implementations gonna help massively
The whole design where you need to get a magical list of IDs to decode the metrics and the list is not even shipped with device is entirely asinine
There are also tight time budgets (basically SLAs) that must be met along with memory considerations (both in SW and on the NPU/HBM), not to mention how the device needs to respond to load and various failure scenarios.
if any hardware does not confirm to specs, thats on them and its actually a reason to pull out and replace legacy hw
Your rewritten codebase will presumably look much cleaner and more maintainable that what it's aiming to replace, but it's easy to miss the purpose/necessity of all of the old "cruft" which need not be just poor coding but rather later enhancements not anticipated by the original design whose implementation was therefore crufty.
Does anyone here work at Cisco, and have any insight as to why a rewrite is considered necessary and might be expected to succeed?
A rewrite has historically been a huge mistake in many situations, some of which I've personally had the agony of witnessing firsthand, powerless to stop it. But AI completely changes the calculus because it doesn't waste significant developer time or energy crawling back to the previous working state, it just does it while you go get coffee. And this I've found to be a huge benefit, that I can continue making my codebases nicer to work with and better documented/integrated with bugs stamped out for free. There are upsides to a rewrite, and they're now much more likely to be worth it as the downsides are automated away.
AI has no magic powers to figure out what the “API/etc contracts” are in a nasty old codebase. This problem is fundamentally hard and not always well defined.
AI is also (at least for now) not consistently good at writing tests that test the behavior that matters. Recent models seem to be getting slightly less bad, but again, this is a hard problem.
I’ve personally used AI to rewrite gnarly code where the goal is to have exactly identical results to the old code. GPT-5.6 Sol could do this correctly with some extra help guiding it to rest the right things, and I haven’t tried this particular task on a newer model.
But ancient code like in the article is probably an enormous buggy mess full of accidental behaviors and implementing protocols in ways that do not fully match the spec. Some, but not all, of the mismatches will be cases where the spec is wrong. Sure, an AI can probably generate a new implementation with a similar set of bugs, but is this what is actually wanted? Also, depending on the gnarly set of bugs, a clean implementation may be unable to reproduce the bugs, leading to a messy implementation instead.
With the current sota models code is free.
But there is pretty big risk of false positives and negatives; some "bugs" are design decisions that were not documented well/at all, and inevitably some new ones will appear when it reasoned something wrong.
An LLM doesn't have enough context to know what you so and don't want. If you give it call logs for every function that was called and how, maybe, but a seasoned engineer will have more context around what the company needs.
Honestly though, it's pretty clear the AI rewrites work, we have a ton of high profile examples. It's just a holdover attitude about rewrites from before AI. In reality, AI is going to make the code better when it rewrites
Such as?
You don't get to pick what parts of the API you're porting during something like that, it has to work the same or some of the ecosystem around it will break.
Obviously this fails if this is a large org with many teams and microservices, but for a leaner org this isn’t really a problem.
I realize this thread is about Cisco so my comment doesn’t apply here haha.
And AI is really good at all the stages here.
AI makes these projects significantly more feasible, at least in my domain. We have some critical-path software I wouldn't touch with such an approach, but the vast majority of our code is internal and/or supporting tooling.
For those cases, burning it down and rewriting via AI tooling has proven far more productive than anything we have attempted in the past. This is for old technical debt laden stuff everyone has been afraid to touch for well over a decade.
It's not perfect, but it moves the needle where before a lot of such projects were stuck on the backburner for years.
It certainly introduces new bugs and such, but at least this way forward momentum is happening vs. just being stuck in decision/analysis paralysis forever.
This also assumes some very senior SME folks are driving the process. I would not expect similar results with junior or even mid-level folks in these specific domains.
One of the reasons to avoid rewriting has long been that the time spent rewriting code that works is time not spent writing code to meet new business needs. In other words, it's usually a huge opportunity cost, rarely balanced out by sufficient gains to be net positive.
I do think AI does challenge this prior, as the time needed to do a rewrite shrinks dramatically. It's not zero, of course, but it now becomes worth asking if it's worth doing, rather than defaulting to avoiding rewrites in nearly all cases.
Until one day you leave and the next guy comes and says “what is all this shit”
A critical security firmware update once brought down redundant pair of firewalls we used because the bug didn’t manifest until a counter buffer overflowed months after application. Three hours into the outage the tech power cycles the thing and our services came back to life.
Cisco is only doing this because some other alternative is more expensive. They had more than a decade to get started.
We had to turn off a bunch of on-by-default features in the system I'm deploying now because IOS matches too aggressively and decides that arbitrary UDP datagrams are DHCP requests or PTP packets that need rewriting (and then dropping when they turn out to not be DHCP/PTP). I have to think that an AI rewrite with a today-good model would recognize the lax checks and do them better.
Idiocracy (2006): https://en.wikipedia.org/wiki/Idiocracy
yeah, i think they'll survive whatever the outcome
The main takeaway though is we had one person, in a single month, do the equivalent of at least a couple years of work for a small team. We view bugs and performance issues as a currently inevitable part of the process and we are dealing with them with models just like we do with humans on every other project already. Overall though it is representing an absolutely massive speedup. If you work in an environment where the models make egregiously more mistakes than your human counterparts then I am quite jealous.