Posted by poisonfountain 18 hours ago
Look at prompt engineering, and how quickly it became a hot thing. Does everyone know to steer their AI well? There's only so much a harness can do for you once you start attempting to one shot with a single sentence of 4 words.
As others said, "write a Rust compiler make no mistakes" can only work if you overfit a harness to that single prompt. Nobody is going to do that.
So the part you mentioned about the knowledge you accumulated around how to know that "trade-offs between implementations" and "idempotency to prevent double-charges" is just moving to the domain of the english language and tokenizers. One could argue here that this is far more interesting as it requires you to explore deeper into how we communicate and describe the world around us. Reminds me of physics and math.
I think there's an optimism lenses to it if you can grasp it as an opportunity rather than an inevitable doomsday apocalypse.
If you’re not a good engineer and you don’t have the domain knowledge, your token costs will be very high for whatever gets shipped, because you won’t be able to provide the context necessary to prompt machine efficiently.
Claude will still very often hallucinate bugs, explanations, domain requirements, that have no basis in reality. It will offer fixes and improvements that are pretty standard but not optimal. This is correctable if you catch it, but you need to review every line of code and comment, because in addition to being obviously wrong, it is often very subtle in the wrongness. For every bit of “slop” there is almost microslop, the places where it just kind of confidently guesses… and doesn’t tell you… but sometimes is correct anyway.
The “problem” is there’s less low hanging fruit. You have to know a lot to add value beyond being a middleman gating the slop. You have to really pay attention to the details to find some of the errors that it’s making.
That said, Opus 4.8 and Codex 5.5 both can write code that is higher quality than your average engineer. They are not quite there yet in terms of code re-use, but I think that's a solvable problem.
Ownership and responsibility are the new currency for the engineering staff. Willingness to implement these tools and then own the consequences of their use is what leadership is looking for. They want their cake while they eat cake, and they will keep those around who enable something approaching that experience. Owning the side effects of LLM use is more challenging than our own natural output because of the radical volume increase and unfamiliarity with low level details. However, I argue it is still possible. It has always been significantly more expedient to poke holes in someone (something) else's work than it is to perform that same work. And, the executives know this. They leverage this capability too.
The relationship between the business and the development team has been tenuous at best. I've rarely seen a technology team that was properly subservient to the business that ultimately signed their paychecks. I every case I have personally experienced, it is was like a hostage situation where the business owners are in constant terror of the technology people screwing them over in some infinitely nuanced way they or their lawyers could never understand. Many business owners are looking at this technology as a way out of the hostage situation. They noticed a window that was left unlocked. They are going for it right now. Whether or not they will succeed in their escape is a separate matter. Whether or not them being held hostage was justified is also a separate matter. It really helps to keep these things in their own lanes.
It's still funny that 4 years into this mania the models can hallucinate basic ground truths, humans are increasingly not reviewing the output, and misusing LLMs where simple automation would suffice.
My wife does project management and works with a lot of tech leads. They came to her with a project plan deck, and she started questioning some weird dates.
The LLM was able to pull artifacts out of their issuer tracker, but it just.. hallucinated some of the dates in the process of creating a project plan deck out of the underlying data. These guys didn't care to review and notice, and who knows what else it hallucinated content wise. They were happy to send this project plan multiple levels up the food chain with hallucinated unreviewed dates.
5 years ago they would have just written a script and had none of this mess.
Instead of directly: do this.
Preferably I would interweave code and AI queries where some function waits on prompt result too I think?? To avoid too big context hallucinations
I mean that would work for my use cases.
At least what I learned is that the less AI itself does in the context is the better so to say as critical LLM mistakes are approaching 100% of probability over time.
There are a lot of non-tech people using these products in this manner.
Along these lines my friend is CTO at a non-tech firm and theres vibe coding happening in one department on a project that is going to churn $1M of tokens. Head of that department told him it's OK because instead of paying a SWE annual salary, they'll just pay $1M of tokens once forever.
People don't know what they don't know about software, SDLC, support, maintenance, etc. If code was something you write once and never think about again, most tech orgs could be 75% smaller.