Posted by florianherrengt 9 hours ago
We've seen this game before, with the Dot Bomb and 9/11. A lot of you have 6 figure jobs that will simply be gone next year after the election, never to return.
That winter lasted about 5 years until 2007 when the iPhone and social media sites like Facebook went mainstream. This winter won't end. Or more accurately, it is the end.
Maybe we should stop debating whether we're in a crisis and start looking at the 5-10 year endgame of the Singularity. We're entering the eventuality that many of us have predicted since the arrival of the internet 30+ years ago, that prices will get lower but nobody will have any money to buy anything anyway. The feeling that we're doing something wrong with our lives will continue to intensify. We'll try so hard, and get so far, but in the end, it doesn't even matter.
I don't know about the rest of you, but I'm turning my attention away from trying to play the job game with AI. I'm looking instead at how we can provide time/money/resources to the middle class after capitalism can no longer provide them.
A conservative friend of mine just read the Communist Manifesto by Karl Marx, which I didn't see coming, especially since he beat me to it. In the neurodivergent spirit of not being able to do anything alone, what if we all read it?
https://oll.libertyfund.org/pages/marx-manifesto
https://www.marxists.org/archive/marx/works/download/pdf/Man...
I'm predicting that it stops just short of what full automation can provide. Loosely the evolution of economics goes: hunter-gatherer -> agrarian- > feudalist -> capitalist -> socialist -> communist. And now solarpunk (or something).
The working class is looking forward to the next chapter, but the capitalist class is looking backward to neofeudalism.
Those two directions can't coexist, so what does the future hold? Well, it's always darkest before dawn:
"It's easier to imagine the end of the world than the end of capitalism." - Mark Fisher.
https://www.youtube.com/watch?v=aCgkLICTskQ Mark Fisher: The Slow Cancellation of the Future
https://www.scribd.com/document/835022535/Mark-Fisher-The-Sl... (transcript)
https://goodscienceproject.org/articles/the-slow-cancellatio...
https://mediationsjournal.org/articles/end-of-world
If the endgame of capitalism is that it inevitably eats itself, then the game now isn't how to get more money and do nothing with it, but instead how to start doing something with no money.
I should add that even if I'm exploring anti-capitalist futures, that doesn't make me socialist. I don't like the 6 hour daily work requirement of many socialist societies. I think we can do better than that and get closer to 0, so that we can shrug off wage slavery and finally get real work done.
Whether people will buy AI-generated software is a different question. The problem I’m describing already exists inside businesses with paying customers.
I’m talking about established teams working on products that already have users and make money, where AI lets individual engineers introduce changes faster than the rest of the team can properly understand and review them.
I don’t think it’s “just a tool”, it’s something that directly compete with the role of humans in a system, by design
Is a very sophisticated computer driven adaptive control system making decisions for the user? Arguably yes. If your car has a bunch of assist tech you are not driving. You are commanding a nonlinear control system.
AI today is quantitatively far beyond these systems but internally it is still just a giant decision table. It’s just a differentiable table that can be automatically programmed by back propagation at what would be unimaginable scale for manual implementation.
As soon as you put a machine between you and the target action that is more complex than what you can hold in your head and that has multiple layers or feedback loops, you are ceding fine grained direct control in favor of a simpler abstract control surface.
I will change my mind a little if I start seeing evidence of genuine volition, but as of now I suspect that’s a property of life not intelligence. One interesting result of AI when viewed as an experiment is: I think it proves that life and intelligence are separate phenomena and that consciousness and intelligence may actually be unrelated.
I have the hypothesis that consciousness (and therefore true volition) is a property of life. You have to be metabolic and tied in some intimate way to thermodynamics and the arrow of time. Solid state electronics doesn’t have this property. It’s unrelated to intelligence. A bacterium may be conscious.
I'm not the parent, but personally, no, I don't believe so.
I always appreciate the "it's like past changes" thought when things change, but in my opinion, this one simply isn't. It's notably different both quantitatively and, critically, qualitatively, than all previous increments in computers/software. This is much more lateral than punch cards -> assembly -> compilers -> frameworks.
Also, regardless of that thought, the "it's like past changes" argument relies on a false implication: That all changes that survive are "good" in the broad sense. Many changes often persist simply because they are inevitable - not because they are good. They have both good and bad parts, and it's subjective as to which side it leans towards.
Though I personally don’t find the topic of consciousness interesting, it’s not something we are able to define and is distracting from all the other aspects of LLMs that should be discussed or evaluated for what they are.
You’re deliberately constraining the output so changes stay small, human-reviewable and reversible. That’s good engineering culture.
It isn’t all or nothing. I use AI heavily but that doesn’t mean I have to pretend there aren’t serious problems with how it’s being used.
So many A car is simply a faster horse arguments being made here.
The “You must review every line of code” camp are going to be in for a seriously bad time when Claude/Grok/etc starts writing machine code, and they absolutely will and you will look like a crazy person the same way as those who said we can’t trust compilers back in the 1970s.
Breaking up code into small reviewable pieces is obsolete advice from legacy software engineering. We will probably be reviewing prompts or functional simulations as a form of review. Which brings me to the point that a Senior engineer in 5 years will look absolutely nothing like a Senior engineer in 2019.
I'm working on this right now, will have a release probably in a week or two. Not machine code but it's essentially a custom assembler/compiler pipeline (close enough) that an LLM can operate via the CLI (you can do it too, it's just way slower) and it emits native asm optimized chunks that can then be inserted back into cpp (working on a rust et al impl). The results are _obscenely_ good. It's outright outperforming gcc vis-a-vis by a MONSTROUS margin, and it's not even close. As in, gcc can do abour 3-4 cyc/byte (arbitrary workloads on a wide variety of tests) and my tool does it in 0.5 cyc/byte if not better. It outright emits kernels that run at a flat 4 IPC for more or less _anything_. You can then have an LLM convert it back into high level C++ intrinsics (not always, gcc doesn't always expose all the levers needed to do so) if you wish. The one downside is that you need to provide fairly accurate cpu arch maps (ie agner fog instruction port/latency tables) for the tool to work right, which is what i'm focusing on right now.
If you don’t understand the system, don’t know what the code is doing and you’re mostly prompting an LLM to make the decisions and implementation for you, what exactly is your contribution?
The ability to operate the tool isn’t much of a moat if everyone else has access to the same tool.
I don’t review assembly produced by a compiler because the compiler isn’t deciding what my system should do. It’s translating a program whose semantics were already specified. More importantly, that translation is deterministic.
A compiler takes a human-specified program and translates it into another representation while preserving its semantics.
If in five years I can give an agent a complete specification and reliably verify the resulting machine code against it, then sure, reviewing code may become obsolete and I’d happily stop doing it.
Also, keeping changes small isn’t just about making individual lines readable. It limits blast radius, makes behaviour easier to reason about, isolates mistakes, makes changes easier to revert and so many other things. None of those properties suddenly become obsolete because code generation got faster.
I ask because I've pushed some of this to the limit in my own testing and when I get the right building blocks / primitives in place in the repo, I'm able to get plans written in a way where I don't just describe what I want but specifically mention outcomes, verifications, constraints, etc. that utlimately describe the shape of my system and my intent.
My plan is my understanding of my system and the changes I'm making, and the execution of that plan (e.g. with subagents), verified against my spec, is the code itself.
With the correct building blocks, I've found that frontier models can write code in fairly predictable ways that lets me maintain my understanding and confidence in pretty large PRs.
I think perhaps the quoted part of your post seems to imply it's not possible / isn't phrased with that nuance?
If you genuinely understand the resulting system, then that's fine. That’s not the behaviour I’m criticising.