Posted by bkd9 3 hours ago
> when technological improvements that increase the efficiency of a resource's use lead to a rise, rather than a fall, in total consumption of that resource.
[1] - https://en.wikipedia.org/wiki/Jevons_paradox
Las Vegas replaced the expensive incandescent lighting on the strip with cheaper to run LED equivalents. But the costs didn't come down because they were able to add more lights and larger displays.
I think the same will happen with tokens. As the cost of tokens comes down, these models will just consume more tokens.
People are a gas; they expand the fill the space they're in. If you give someone a big house, they'll fill it with crap. If you make food cheap, they'll eat too much, even when it harms their health.
Making things cheaper usually just makes making them more prevalent. It's why computers are not faster than 20 years ago. They're merely more capable -- developers quickly and aggressively fill up (and overflow) all that added capability until you're back in the same place you used to be.
Compare the AMD Ryzen 7 9800X3D to the AMD A12-9800, you go from 4 cores to 8, and you double the power consumption. But it's not twice as fast. It's 10 times faster. Depending on the exact metric, it could be as little as twice as fast or as much as 1000 times faster, depending on the exact operations.
On top of that memory throughput between DDR4 and DDR6 is about 2.5 times faster as well.
Oh and this isn't 20 years apart, this is 7 years apart. 20 years will see almost an exponentially larger gap even still.
(...or over 40 years ago)
Obesity is nearly everywhere. In the old days, "prosperous" might have been a euphemism for "fat," as only the rich could afford to be fat. Now even people who are "food insecure" are often quite fat. Some of this of course is due to food quality, but ultimately there are no starving fat people.
You're right that obesity is not necessarily linked to overeating . How is this relevant to your original point? Isn't it a complete contradiction to what you were saying before?
The original post was hyperbolic to the point of nonsense and now it seems we're on a completely different topic.
This can be true, but I think some people are confused. The vast majority of the time, it is linked to overeating. There are some edge cases where there is something else going on metabolically, but most people do over eat.
I anticipate we will see something similar with intelligence. There is probably headroom to consume 100x as much intelligence in R&D. But that isn't most of the economy. Will run of the mill service jobs increase their use of intelligence by enough to offset the effect of cheaper prices? I think that's the real question.
True high-intelligence outputs (Maxwell's equations, the Fourier Transform, quantum theory) are fundamentally transformative in ways that mid-high-competence (starting a generic B2B SaaS, making another CRUD app) aren't.
Which is why we've assumed we're already pretty far down the path to AI, but we really aren't. Solving random Erdős problems isn't the same as opening up a completely new kind of math/science with game changing practical applications.
I don't think you can get to that level with more compute and more tokens. I think it's going to take new higher level knowledge representations and new kinds of training to get there.
And the token count and compute may turn out to be lower than what we're using now.
The lightbulb thing seems different though? Or at least it is a specific subset. Lights in Las Vegas are sort of an advertisement, right? In the sense that having the brightest or most interesting (or whatever) lights draw attention to your show, casino, hotel, whatever. It’s kind of a zero sum game in that the different shops are competing for the finite attention of a more-or-less set number of tourist. I think part of the Jevons paradox is that society generally finds more useful applications of the newly cheap thing. If the thing’s only purpose is to compete better in a competition with a set prize (all of the tourists’ money), that’s constrained in some way.
Intelligence is weird though. I guess we could eventually hit the point where, I dunno, maybe there’s some information theoretic bound where we process all of our signals as cleverly as possible and aren’t bound by intelligence anymore. Obviously we’re nowhere near that. It would be a very alien environment.
That sounds a race to the bottom for AI companies profits.
I do not think that LEDs are high profit margin items.
But we never scaled intelligence like this. The industrieal revolution created for the people at that time quite a huge issue / it was disruptive.
What will hapen to us though?
Related:
- Parkinson's law: "Work expands to fill the available time." https://en.wikipedia.org/w/index.php?title=Parkinson%27s_Law
- Lewis–Mogridge position: "Traffic expands to meet the available road space." https://en.wikipedia.org/wiki/Lewis%E2%80%93Mogridge_positio...
And I pretty much just plain agree, this is exactly what will happen.
I don't think there's anything wrong with it (in isolation) either, though I do already find myself pointing out that we're misusing LLMs at work sometimes (most notably, a recent mini project could have been a jinja template - and it did become one thanks to me pushing back on this). Abundance is one thing, waste and misuse is another.
I love how in our day "reading everything" means "the computer reads it for me".
I expect soon the computer will be able to go on bicycle rides, and spend time with my wife.
Sure you can speed things up with parallel work under subagents, but as with parallelizing traditional computational tasks, there are diminishing gains.
I keep hearing people saying just change the way you work to trust long-running agents and multi-task more, because they’re too slow to work with interactively for many use cases. I think that’s painful in a world where we expect humans to still heavily guide and interact with agents for their day-to-day work.
This just demonstrates how much we already take for granted the LLMs that we have now. If you compare it to what we had before (hand the task off to a junior dev and wait for them to complete the work) then it doesn't seem slow at all.
What's especially bewildering to me is that translated back to raw bandwidth, even 15000 tok/sec is just like what, 75 KB/s? Extremely meager amounts of data, moving mountains.
It's already kinda funny seeing LLMs throw out effort estimates in wall time terms. It's always some "hours, days, weeks" tier thing, when in reality, it's gone and done in minutes.
Robots right now generally move at glacial speeds. You might have seen robots doing flips in semi controlled environments but watch how slowly they open doors etc. processing time is a major bottleneck.
You can do backflips with pretty much just visual sensors for your environment, a good IMU for your spatial orientation, and some feedback on the position of a small number of really beefy joints and the force exerted on them. Folding laundry and opening doors is much more difficult, and trying to compensate with mostly vision requires going slow enough that things have time to move over appreciable distances before you take the next adjustment
Still slow compared to humans, but Chinese robots will be as successful as Chinese EVs, phones and solar panels.
The most recent video which actually impressed me was a demonstration from Gemini Robotics 2, where a robot was shown autonomously removing the bag from a trash can and folding the loops closed in real time.
I don't follow robotics advances closely so it's possible I'm just ignorant, do you know any autonomous robotics demonstrations of useful activities that you would suggest checking out?
Right now all three of those are at abnormally high levels. Competition will come for all three.
For a lot of tasks, even small models have saturated them a while ago, and then going cheaper and faster is just pure gains.
For coding I also prefer to do it interactive/realtime, micro-prompting, surgical edits, which the small models can handle just fine.
And then at the top, the real question is consistency. Not "can they do it" but "reliably enough that you don't need to constantly double check everything." (In my experience, not quite there yet, although it's getting way better.)
Entirely feasible that by 2031, Fable 5 (or greater) intelligence level models will run cool on smart phones, if not sooner.
Maybe it'll take 10 years or 20 years. <5 years is not long enough for manufacturing to catch up.
Not much of a comment on the phone stuff but I'd caution against suggesting technology will never be good enough to do X. Maybe it'll be horrendously wasteful but it might happen.
(Actually talking to it, it was about as coherent as you'd expect, i.e. 3/10)
The floor for "actually usable model" keeps dropping though. (Seems to be about 27B right now?)
textiles had jevons paradox, and many more textile workers were employed even when textile machines were being created, until we saturated the demand for cheap clothing in the world and then textile workers were kaput (same for farming, and horses)
software is currently undergoing jevons paradox, but it's very unknown how high the ceiling of demand for software is. web dev might be doomed, but software in general i think is probably limitless
Intelligence is also probably unbounded (atm software and intelligence are very closely tied together). its very possible token spend rides up the curve forever.