Posted by HieronymusBosch 4 days ago
1: https://www.joelonsoftware.com/2000/04/10/controlling-your-e...
There is some movement in this direction--there is a new 'gh' subcommand called repo read-file for example, that lets agents view a file without cloning a repo. And I made something called venetianblinds that shows equidistant samples of a file. In combo they work pretty well:
gh repo read-file sqlite3.c --repo clibs/sqlite --output sqlite3. c && npx github:firasd/venetianblinds sqlite3.c
--- sample 2/20 char 283427 line 5855 col 53 range 283367:283487
le].
**
** ^Closing a BLOB shall cause the current transaction to commit
** if there are no other BLOBs, no pending prep
^Bash is a great control surface anyway for LLMs as it is wordy and powerful.
Another venetianblinds survey, of the Paul Graham 'What I Worked On' article that's used in many LlamaIndex examples:
npx github:firasd/venetianblinds pgworkedon.txt
--- sample 1/20 char 0 line 1 col 1 range 0:60
Before college the two main things I worked on, outside of s
--- sample 2/20 char 3946 line 19 col 237 range 3886:4006
d an intelligent computer called Mike, and a PBS documentary that showed Terry Winograd using SHRDLU. I haven't tried reHaving a scripting language as a tool is powerfull, and can help remove unnecessary stuff from the LLM's context.
This is ridiculously complex to just type a few paragraphs. Nobody in their right mind would invest this amount of yak shaving... and yet I do so because I bet, rather confidently, that in few years, heck few decades, all those tools will be different (or maybe not, I still use Vim on my server, desktop but even mobile phone) but the lessons will remain practical.
IMHO the trust comes from trust yes but also more directly plain ownership.
For example, I use Codex and Claude Code by default, but when I need to look at the API surface, read tests, etc I have those tools setup to open Zed. Zed is also rapidly evolving in the other direction, where it's closer to the tools that are opening it. It won't be long, I think, until I can continue my prompt from inside Zed.
This makes it unreliable and preferences are things I need to assume are treated as exactly that, preferences, not hard settings.
It is an area where it is more like working with an unreliable human than I would prefer.
> "Developers become attached to tools like Vim, Emacs, or an IDE because years of experience make those tools predictable extensions of their thinking. The attachment is less about features and more about accumulated trust, muscle memory, and a workflow built around known boundaries.
> AI coding agents disrupt that trust because they are fast but probabilistic, opaque, constantly changing, and capable of producing more code than humans can realistically review. This shifts the bottleneck from writing code to specifying, reviewing, validating, and operating it safely."
(Note the > is paraphrasing)
Trust is a big problem I'm having with these tools so far. What I've been running into a lot is, I'll get the equivalent 40 hours of work done in 8 hours, and I'm like, wow, that really was quick. Then I'll start using the application I'm making more directly (a tool for writing), and I'll start to see that it's broken all over the place in very surprising ways (ie, updating this menu item broke something on the other side of the app, etc.). So then I spend another 40 hours of real wall clock time kind of fixing everything that was broken, and at the end of those two weeks I'm like, did I actually go much faster or was that all kind of a wash? Because if I'm not going faster in overall terms, then the loss of deep understanding of the code base might not be worth it if my pace is the same.
I'm sure someone is going to be like "BRUH AUTOMATED TESTS" or "BRUH MODEL CHOICE". I have a LOT of automated tests, and I don't like fussing with models so I pretty much use Opus on high reasoning for most things (or the equivalent from other providers). Code review also doesn't help that much, for much of the same reason it doesn't tend to help find bugs in human written code either.. you're reading the happy path usually.
Anyway I wouldn't say these tools aren't useful, but, I'm deeply skeptical of all the productivity claims because I think people just look at one dimension of it while ignoring all the other important dimensions. Yeah you can generate a lot of crap fast, but most of it is not shippable and making it shippable does take time.
Also, is there ways for AI to help prevent the loss of deep understanding of your code base without you having to know every line of code deeply?
Am I giving the agent enough context? Well, I'm giving it as much as I can. Each submodule has an AGENTS.md, I have the agents add gotchas and instructions for some feature work when I discover where an agent went wrong, the codebase has a lot of comments along the lines of "if you edit this section, you need to also edit XYZ", and I lean on the type system as much as I can to make wrong-code not compile. It has access to playwright for driving the UI if it wants to. (Weirdly, I've found that Claude is really inconsistent about using these tools -- even though the instructions make it clear that it's allowed and encouraged. I think if your workflow differs from the models training and thusly you have to tell it so in AGENTS.md/CLAUDE.md, then it's very inconsistent about following those instructions. For instance, I don't want Claude to commit and I don't want it to sign commit messages, and it still does that all the time even though it's my like #1 directive of "don't mess with my git history")
There are some things though that are very hard for it to test. I'm exporting essentially a programming language to three game engine runtimes. They all have automated tests, but, I think people that have worked in video games know that games are very hard to automate testing on. This isn't really the fault of the agent I would say, just the nature of the problem, but it is worth noting.
I guess this is a long winded way of saying, even with LLMs tech debt is a thing you have to manage, and I think managing tech debt becomes even more important when you're dealing with LLMs, not less important.
He says "40 hours" because he feels like he's managed to do 40 hours worth of work in this time, but then has to spend another "40 hours" (actually: 1 day) just going around kicking tyres.
Obviously the implication is that it's a net gain of some kind, but he's unsure if he caught everything.
(sorry to reiterate the GP, but I feel like you missed the important nuance that it's not a real 40 hours of time).
Alice: "in the end, AI doesn't make me any faster because it still takes 80 hours to do 80 hours of work once I fix it"
Bob (AI booster): "actually you might've been holding it wrong, did you try XYZ?"
Carol (AI double-booster): "Bob, actually Alice means it took 16 hours to do 80 hours of work. So it did work for her."
Alice: "no I fucking didn't"
And I can hear your objection now: correct, it's probably not worth all that for a throwaway, but the effort per output goes down as the infra builds up and you end up with a program that can reliably expand.
I'll also just point out my philosophy for using LLMs for this project, which is that I'm not trying to go as fast as I can. (I want to go at a good pace, but this isn't an experiment to just finish something over a weekend). The 150k LOC have come about since February, with some mix of me writing code and LLMs, so on average I'm probably bringing in about 800 LOC per day, which I imagine a lot of vibers would find to be glacial. To me that's the sustainable rate of what I can do when you factor in that I need to test drive every feature, make sure it doesn't conflict with another feature, check for bugs, check that the code looks reasonable, and debugging. (I also think that rate limit is specific to this project: I could see easier to test things going much faster, and harder to test things going slower)
My own stance is that there's never any reason to go fast on anything. Communication has always been the bottleneck. Whether it's about gathering requirements or understanding the purpose of a badly written code, any speed improvements I get has always been a small percentage of the overall progress.
What has helped more is my understanding of the platform and some theoretical knowledge. Because one I get the information, I can quickly derive a solution in my mind. And that solution has always been easy and fast to implement, at least the happy path. 90% of the time taken in coding is always about handling all the edge cases, aka fixing bugs. And writing tests so that you're not easily introducing more bugs.
I update my toolset all the time. It always results in discomfort and backlash but people don't understand that the goal isn't their perception or trust. It's about skill, ability, and execution. This idea probably won't get me promoted but it will get me paid. I am not attached to tools because I learned the hard way that they will always find a way to take them from me.
Zero trust is a better alternative for people like me. In terms of cybersecurity, being attached to a tool is crutch because fatal flaws in every design are frequently found. As it relates to agentic AI, I never select the "Yes, trust the AI and let Claude execute arbitrary commands in a non-sandboxed environment" option. However, I frequently utilize agents, but I'm not going to have the "Jesus, take the wheel" moment with them right now. That being said, AI is a very helpful tool that helps me create boilerplate code, brainstorm ideas, and review my work. I also anticipate when AI can, in fact, take the wheel and I'm looking forward to it.
Parallel to this, I also know that developers often disagree, and I'm not casting judgement on anyone for being attached. If it's Turing-complete, then I have the background to complete the task. In these scenarios, I just adopt whatever tools work best in team building because, in my own words, I'm not too attached to the way I do things.