It's worth asking ourselves "why does the x amount matter?" I get the desire to define KPIs to estimate productivity gains. But this all seems to be—rather rapidly—leading to an increasingly dehumanized reality (both figuratively and literally) so we can...produce more software? I love building software, and I enjoy using LLMs to help me do it, but there's just this weird vibe I can't quite shake about how we're trying to quantify all of this.

I'm not sure about 1x, 2x or 10x increase as these metrics are about code written but that isn't a productivity metric (something pretty much every half-decent programmer would agree with before LLMs - remember stories about Bill Gates saying that more LoCs being good for software is like more weight is good for airplanes or Bill Atkinson's story about adding -2000 LoCs to improve QuickDraw?).

But they can certainly help "get you going" faster in that if you're stuck on something (for whatever reason - including "that feels too much drudgery") or have issues starting something, you can have an LLM take a stab at it and it'll produce "something". Sometimes it is enough by itself, but more often than not it'll need tweaks (either directly or having the LLM do it). I got to make a bunch of things i couldn't convince myself to do - e.g. an image viewer that doesn't suck (based on my arbitrary judgement), a game database, a script to convert a git repository into static html pages that kinda look like GitHub, etc.

They can also help find (and sometimes fix) bugs or other "code smell" issues. They're not that great for exact results (without tool calling -and knowledge on how to use them effectively- at least) but when it comes to fuzzy / vague stuff like "check out this code <code dump here> can you spot any issues?" they always tend to find some stuff (even if it is hallucinations :-P but sometimes they find actual issues too or whatever hallucination they come up with reveals some actual issues with the code that you didn't spot by yourself). I've been dabbling with Rust recently and asked Qwen 3.6 35B-A3B to judge my code and it wrote "6.5/10, will compile but looks like C in Rust" :-P.

The article says:

> Never write READMEs, docstrings, or comments. I will write those myself later. And yes, I really mean this.

And sure, LLMs aren't that great about those (i do let them leave whatever comments they want though and remove them later myself - i think those comments help during the generation/prediction - basically how they "think", kinda like the reasoning phase), but they can be very good at things like "here is the code, here is the documentation for it, spot discrepancies" (i had Devstral Small 2 do this and it hallucinated a few discrepancies but also found real stuff i missed in the docs).

I've tried to use Devstral Small 2 for some API docs but found it'd sometimes make assumptions about what function do or how. One approach that might work, but i haven't tried yet, is to write the "guide" myself, then have the LLM write the function docs using both the guide and the code as reference. The reason i think this will work is because it got things 95% correct just having access to the function code alone (and without the rest of the codebase), so the "guide" would help it reach 99%. I do not expect it to get to 100% so a manual edit pass will need to be done anyway (and i have an idea for a tool to assist in the manual edit pass for that - a tool that i'll probably get an LLM to write - BTW good luck coming up if all that stuff would increase or decrease productivity for an actual product and not some random stuff i'm toying with :-P).

One other thing i've also found LLMs useful recently is to have them use the stuff you make and see how they try to use it. I have an old project, a GUI toolkit i've been hacking on every now and then since 2011 or so, though it was never a priority. Yesterday i decided to dump all the header files to Qwen 35B-A3B (i use 4bit quantization that gives me a 256k context - it isn't particularly smart but it is neat to not have to micromanage context size like i have to do with Devstral Small 2 or Qwen 27B where both of them aren't very usable speedwise at anything above 32k context sizes).

Then i asked it to just make a few small programs and it did[0] (the shot shows a paint app, a calendar, a todo list and a unit converter). Pretty much every program found bugs in the library :-P and gave me ideas on how to improve things.

In general i get the impression that LLMs aren't great at architecting things but if you do the architecture yourself and write them a framework to use, they should do a fine job at it.

[0] http://runtimeterror.com/pages/iv/images/d50c436990db203a07f...

Wouldn't trade it though. Feel like I can overall do more with less time and energy.

At the end of the day, AI is making me work more (good thing). If you count that as productivity, then sure.

I did get wins for sure (and I did save time), and I'm sure most people wouldn't even spend five minutes cleaning up, but this is definitely one place where people talk about different things regarding whatever x speedup they get.

What you are using it for? What language, what framework, how many LOC, what domain, how much docs can the LLM read, etc etc.

What are you optimizing for? Cost? Human knowing how your code works? Getting something out the door?

How Good You Are at Prompting the LLM for THAT specific set of parameters?

I find AI can save me like 5x time on something, and in other cases it's wasted 5x time. Overtime I'm hopefully learning how to use it better.

I found that latest codes don't write comments in code by default. And when they do, they write stupid shit like "This was code that did X, it was now removed".

You have to explcitly prompt them to write comments in code. They are still useful for you, the user. But are arguably useful for the model, too, given how many of them (especially Claude) only reads small chunks of files. So I'd rather have code comments than it reproducing a picture from incomplete data.

> Never write READMEs, docstrings, or comments. I will write those myself later. And yes, I really mean this.

This is quite validating as I came to the exact same conclusion myself. We’re required to use an LLM for every task at work that touches code†, and I was really struggling to get Claude to stop with the long waffly comments that reiterate the next few lines of code in 3x as many characters, making contextless references to subtasks in whatever harness du jour we’re using this week.

No amount of examples or explanation of what I wanted would make it stop. And then I realised of course, I’m asking something which has no concept of meaning (or, indeed, anything) to only add meaningful comments. More fool me I guess.

Of course, it’s ultimately pointless given all of my colleagues are regularly opening PRs with more comments than code anyway. 80% of my code review responses these days are just increasingly exasperated “pointless comment, please remove”.

† This is just as infantilising as it sounds, by the way

Before I get there I have to figure out how to reliably audit plan adherence. The problem is that when the specs are in natural language, as they are, you need a fallible LLM to verify it.