Posted by youngbrioche 4 days ago
I'm staunchly pro-AI as a technology, but I do think the bubble is going to pop in the next year or two just because the business value won't materialize for most companies fast enough.
AI content has a look and feel people sense immediately.
It’s amazing to see how quickly things shifted from “wow this is so cool, AI is going to change everything” to folks calling out “you lazy bum, this just looks like some slop you threw together with AI… let’s get some real thinking please.”
We are firmly heading into “trough of disillusionment” territory on the hype cycle.
The more I use AI, the more I see mistakes. I've noticed others see these same mistakes, correct them, then when queried say "Oh, it gets it right all of the time!". No, having to point out "you got this wrong, re-write that last bit" isn't "getting it right". And it's not that the code is wrong overtly, it's subtle. Not using a function correctly, not passing something through it should (and the default happens to just work -- during testing), and more. LLMs are great at subtle bugs.
So moving forward with this isolation you mention, ensures that maybe the guy in the company, the 'answer guy' about a thing, never actually appears. Maybe, he doesn't even get to know his own code well enough to be the answer guy.
And so when an LLM writes a weird routine, instead of being able to say "No, re-write that last bit", you'll have to shrug and say "the code looks fine, right?", because you, and the answer guy, if he exists, don't know the code well enough to see the subtle mistakes.
Can we get some enabling legislation? A UN resolution perhaps?
The “get an immediate agent answer then a human expert’s fast-follow” is I think a great idea for many domains - imagine if you could get legal advice this way; the agent will have already explained the basics and the human expert just has to provide corrections - way less typing by humans.
Also, the corrections are now documented and could become future grounding for the agent.
They won't just need to understand what problem the requestor has (or thinks they have) but also validate that the "immediate" feedback wasn't subtly horribly wrong.
So, like what already happens when my boss asks claude something and I have to pick up the pieces. Except now it's everything he slops about the topic, not just the ones we discuss later?
> a great idea for many domains
I completely agree. This is a great idea. If you don't do something with it I'm stealing it. ;-)
AI can get a pretty good picture, near instantly, whenever you need it.
It’s not just competent-sounding, it is reasonably competent, and certainly very useful for tasks like that.
Gone are the days of mandatory corporate "synergy" and after-work bar gatherings to promote "team building."
AI is showing people in the tech industry that they're just interchangeable cogs. AI is bringing the offshored Indian work environment to Silicon Valley.
Not a problem if the hired "AI" now does that job. /i
Our mental models of developments like the industrial revolution, literacy, printing or suchlike tend to be a lot more straightforward than how things play out in practice.
When a bottleneck is eliminated... you tend to shortly find the next bottleneck.
Meanwhile, there is an underlying assumption everyone seems to make that "more software, more value" is the basic reality. But... I'm skeptical.
To do lists, wishlists, buglists and road maps may be full of stuff but...
Visa or Salesforce have already exploited all their immediate "more software, more money" opportunities.
The ones in a position to easily leverage AI are upstarts. They're starting with nothing. No code. No features. No software. With Ai, presumably, they can produce more software and make value.
Also... I think overextended market rationalism leads people to see everything as an industrial revolution...which irl is much more of an exception.
The networked personal computing revokution put a pc one every desk. It digitized everything. Do we have way better administration for less cost? Not really. Most administrations have grown.
Did law fundamentally change dues to dugital efficiency? No. Not really.
If you work on a terrible enterprise codebase... it's very possible that software quality/quantity isn't actually that important to your organization.
It's possible capitalism will drive all enterprise to terrible codebases.
While AI tools have been provided pretty quickly (over a year ago, I initially used gemini cli, then copilot once it added anthropic models) the management is absolutely clueless about it.
The top wants agents. Every team is asked few times a week "what autonomous agents will you build next" and answers the current AI lacks agency required not to mess up critical long running tasks and generate even more work are falling on deaf ears.
(also ideas such as, why don't we setup a wiki page were teams can post their repetitive tasks and we can use AI to script them, are considered "not fast enough" - just build it... but we are the automation team, we automated everything we do years ago :-)
Middle managers on the other hand suddenly started giving juniors senior's work and asking seniors "tell them (juniors) how to prompt it".
Seriously? How about I prompt it myself instead? Oh, but it makes a shit load of architectural errors and boobie traps the junior will fail to find... So now instead of a cursory glance I have to spend an hour reviewing a small PR from them.
And any questions about "why are you creating a new X for this instead of extending the existing one?" are met with blank panicked stares...
The essence of this BS is contained in my description of the recent "Copilot Review" incident.
We sometimes merge the same Github workflow files (10 line files) to dozens of repos, we have to obtain approvals for the PRs from a bunch of teams working in different timezones, but the merge has to be done everywhere at once and it has to be coordinated with other work.
As we were on a day of such task some "helpful hand" enabled Copilot PR reviews for the whole org.
Copilot helpfully opened 7 or 8 discussions on each PR giving us such precious advice as "your concurrency group uses the commit sha as a differentiating factor, this will allow multiple runs to proceed concurrently" to which one is tempted to answer "no shit sherlock".
We suddenly had almost 200 conversations to "resolve" an hour before the merge and a bunch of approvers didn't give their approvals because "there is a discussion".
Thankfully we had copilot that wrote us a script in 5 minutes to resolve the problem caused by itself...
Maybe our next overnight agent can go over all our open PRs and close Copilot Review conversations with appropriate messages?