Posted by evakhoury 11 hours ago
This is exactly how you learn to create better abstractions and write clear code that future you will understand.
I do the former for fun. The latter to provide for my family.
There is a reason old men take on hobbies like woodworking and fixing old cars and other stuff that has been replaced by technology.
(I swapped the title for the subtitle earlier because I thought it was more informative. What I missed was the flamebaity effect that "the old way" would have. Obvious in hindsight!)
You mean the way that the majority of code is still written by professionals?
Why would you think that? The landscape is fast-moving. Prompting tricks and "AI skills" of yesterday are already dated and sometimes actively counterproductive. The explicit goal of the companies working on the tech is to lower the barriers to entry and make it easier to use, building harnesses and doing refinement that align LLMs to an intuitive mode of interaction.
Do you think they'll fail? Do you think we've plateaued in terms of what using a computer looks like and your learnings for wrangling the agents of this year will be relevant for whatever the new hotness is next year? It's a strong claim that demands similarly strong argument to support.
How? I just open multiple terminal panes, use git tree, and then basically it’s good old software dev practices. What am I missing?
Claude Opus is going to give zero fucks about your attempts to manage it.
It is hard indeed. I find it really quite exhausting.
Personally, I feel like I have always been a very competent programmer. I'm embracing the new way of working, but it seems like quite a different skillset. I somewhat believe that it will be relevant for a long time, because there is an incredibly large gap in outcomes between members of my team using AI. I've had good results so far, but I'm keen to improve.
For the good stuff, there’s no alternative but to know and to have taste. Llms change nothing.
Citation needed.
> There were 2 or 3 bugs that stumped me, and after 20 min or so of debugging I asked Claude for some advice. But most of the debugging was by hand!
Twenty whole minutes. Us old-timers (I am 39) are chortling.
I am not trying to knock the author specifically. But he was doing this for education, not for work. He should have spent more like 6 hours before desperately reaching for the LLM. I imagine after 1 hour he would have figured it out on his own.
Though a lot of the time this is more an inefficiency of the documentation and Google rather than something only LLMs could do.
* Ask someone to come over and look
* Come back the next day, work on something else
* Add comment # KNOWN-ISSUE: ...., and move on and forget about it.
But year spent days on a bug at work before ha ha!
This is a tried and true way of working on puzzles and other hard problems.
I generally have 2-4 important things in flight, so I find myself doing this a lot when I get stuck.
Just a note that, for chronic procrastinators, having 2 to 4 important things going on is a trigger & they'd rather not complete anything.
I wonder, for such folks, if SoTA LLMs help with procrastination?
If anyone remembers middleschool mathematics this is the coding example of the teacher making you write out the equations in their longest form instead of shortcutting. It's done this way because it shows you your exact train of thought and where you went wrong. That sticks in your head. You understand the problem by understanding yourself. Giving up after twenty minutes instead of stopping, clearing your active cognitive load, and then coming back erases your ability to understand that train of thought.
For a comparison it's like being in first person view in a videogame, and the only thing you have is the ability to look behind you, versus being able to bring up a map that has an overhead view. In first person you're likely to lose where exactly you went to get where you are, while with the overhead view map you can orient your traveled route according to landmarks and distance.
So, the short of it is that this is a great insightful comment that I can back up with my own experience in making a game from scratch over the last 4+ years.
If you want to solve the problem quickly then just use the resources you have, if you want to become someone who can solve problems quickly then you need to spend hundreds of hours banging your head against a wall.
2) There are different levels of debugging. Are your eyes going to glaze over searching volumes of logs for the needle in a haystack with awk/grep/find? Fire up the LLM immediately; don't wait at all. Do the fixes seem to just be bouncing the bugs around your codebase? There is probably a conceptual fault and you should be thinking and talking to other people rather than an AI.
3) Debugging requires you to do a brain inload of a model of what you are trying to fix and then correct that model gradually with experiments until you isolate the bug. That takes time, discipline and practice. If you never practice, you won't be able to fix the problem when the LLM can't.
4) The LLM will often give you a very, very suboptimal solution when a really good one is right around the corner. However, you have to have the technical knowledge to identify that what the LLM handed you was suboptimal AND know the right magic technical words to push it down the right path. "Bad AI. No biscuit." on every response is NOT enough to make an LLM correct itself properly; it will always try to "correct" itself even if it makes things worse.
This can be set as far as 1h of being stuck. Can also be 5 minutes. But by default it is 30 seconds.
My inner kid was screaming "that's cheating!" :-D but on second thought it is a very cool feature for us busy adults, however it's sad the extremes that gamedevs have to go in order to appease the short-term mindless consumers of today's tik-toks.
But more seriously, where's the joy of generating long-standing memories of being stuck for a while on a puzzle that will make you remember that scene for 30 years? An iconic experience that separates this genre from just being an animated movie with more steps.
I couldn't imagine "Monkey Island II but every 30 seconds we push you forward". Gimme that monkey wrench.
TFA and this comment just made me have this thought about today's pace of consumption, work, and even gaming.
Having a tool that instantly searches through the first 50 pages of google and comes up with a reasonable solution is just speeding up what I would have done manually anyways.
Would I have learned more about (and around) the system I‘m building? Absolutely. I just prefer making my system work over anything else, so I don’t mind losing that.
Just so many confusing things go wrong in real-world software, and it is asinine to think that Mythos finding a ton of convoluted memory errors in legacy native code means we've solved debugging. People should pay more attention to the conclusion of "Claude builds a C compiler" - eventually it wasn't able to make further progress, the code was too convoluted and the AI wasn't smart enough. What if that happens at your company in 2027, and all the devs are too atrophied to solve the problem themselves?
I don't think we're "doomed" like some anti-AI folks. But I think a lot of companies - potentially even Anthropic! - are going to collapse very quickly under LLM-assisted technical debt.
The euphoria I felt after fixing bugs that I stayed up late working on is like nothing else.
If you cant fix the bug just slop some code over it so its more hidden.
This is all gonna be fascinating in 5-10 years.
But for juniors, it's invaluable experience. And as a field we're already seeing problems resulting from the new generations of juniors being taught with modern web development, whose complexity is very obstructing of debugging.
I worked on a project that depended on an open source but deprecated/unmaintained Linux kernel module that we used for customers running RHEL[1]. There were a number of serious bugs causing panics that we encountered, but only for certain customers with high VFS workloads. I spent days to a week+ on each one, reading kernel code, writing userland utilities to repro the problem, and finally committing fixes to the module. I was the only one on the team up to the task.
We couldn't tell the customers to upgrade, we couldn't write an alternative module in a reasonable timeframe, and they paid us a lot of money, so I did what I had to do.
I'm sure there are lots of other examples like this out there.
[1] Known for its use of ancient kernels with 10000 patches hand-picked by Red Hat. At least at the time (5-10 years ago).
The time wasted thinking our craft matters more than solving real world problems?
The amount of ceremony we're giving bugs here is insane.
Paraphrasing some of y'all,
> "I don't have to spend a day stepping through with a debugger hoping to repro"
THAT IS NOT A PROBLEM!
We're turning sand into magic, making the universe come alive. It's as if we just got electricity and the internet and some of us are still reminiscing about whale blubber smells and chemical extraction of kerosene.
The job is to deliver value. Not miss how hard it used to be and how much time we wasted finding obscure cache invalidation bugs.
Only algorithms and data structures are pure. Your business logic does not deserve the same reverence. It will not live forever - it's ephemeral, to solve a problem for now. In a hundred years, we'll have all new code. So stop worrying and embrace the tools and the speed up.
This is both a strawman and a false dichotomy.
Too many of our engineering conversations are dominated by veneration of the old. Let me be hyperbolic so that I can interrupt your train of thought and say this:
We're starting to live in the future.
Let go of your old assumptions. Maybe they still matter, but it's also likely some of them will change.
The old ways of doing things should be put under scrutiny.
In ten years we might be writing in new languages that are better suited for LLMs to manipulate. Frameworks and libraries and languages we use today might get tossed out the door.
All energy devoted to the old way of doing things is perhaps malinvested into a temporary state of affairs. Don't over-index on that.
But just today a bug was reported by a customer (we are still in testing not a production bug). I implemented this project myself from an empty git repo and an empty AWS account including 3 weeks of pre implementation discovery.
I reproduced the issue and through the problem at Claude with nothing but two pieces of information - the ID of the event showing the bug and the description.
It worked backwards looking at the event stream in the database, looking at the code that stored the event stream, looking at the code that generated the event stream (separate Lambda), looking at the actual config table and found the root cause in 3 minutes.
After looking at the code locally, it even looked at the cached artifacts of my build and verified that what was deployed was the same thing that I had locally (same lambda deployment version in AWS as my artifacts). I had it document the debug steps it took in an md file.
Why make life harder on myself? Even if it were something I was doing as a hobby, I have a wife who I want to spend time with, I’m a gym rat and I’m learning Spanish. Why would I waste 6 hours doing something that a computer could do for me in 5 minutes?
Assuming he has a day job and gets off at 6, he would be spending all of his off time chasing down a bug that he could be using doing something else.
If you’re experienced as you are, you’re not learning the same way a junior assigned this might learn from it.
I also used Codex and asked questions about how the codebase worked to refresh my own memory. Why wouldn’t a junior developer do the same?
I mentioned that I had Codex describe in detail how it debugged it. It walked through each query it did, the lines of code it looked at and the IAC. It jogged my memory about code I wrote a year ago and after being on other projects
Just because it worked this time doesn’t mean it always will.
If you need further explanation of why you might want to spend more time resolving a bug to learn about the systems you’re tasked with maintaining then I’m at a loss sorry.
But he was doing this for education, not for work.