Posted by milkshakes 8 hours ago
I'm sure they must do some of this type of work, right?
What I had hoped for was something more ambitious: using AI as an arbiter in economic, political, and social debates, one capable of weighing evidence, exposing trade-offs, and helping us make decisions that produce better outcomes over the medium and long term, even when those decisions conflict with powerful private interests.
I suspect, however, that this is not a particularly urgent goal for the people funding and directing these systems, many of whom live far removed from scarcity and its consequences.
I agree, some of the problems are more difficult. I don't think that's the case for all of them. And, besides, these companies could be demonstrating how to approach problems and where their users could spend tokens to help with these problems.
Should not these companies try to work on these problems _because_ they are difficult?
We already know how to solve all of these issues. What we lack is collective political will.
And yeah, the lack of collective political will sucks. It would be naïve, however, to think that there is no value in ensuring longevity in our current and future infrastructure. And improving it to sustain the population giving these companies their value is an obvious win.
Little sacrifice actually
> Solving the issue would be doing it without sacrifice
So... you are expecting magic?
LLMs cannot create resources out of thin air.
This is division of labor, and it's a good thing. I'm sure OpenAI employees, who are very well paid, donate some money from their salaries to others working on the areas you're citing: probably more than you think, I say that from having attended some EA parties back in the day.
But that isn't my point: my point is that a company which makes brushless motors should put most of its time and money into solving the "make and sell brushless motors" problem, and if they or their investors feel like they need to do more for the world, give money to the people who have the time and ability to do things about that. There are a lot of quality-of-life improvements which need brushless motors.
Next question is how useful their product (OpenAI, I mean) is to more focused do-good-in-the-world professionals. I'm sure that varies quite a bit. For getting the homeless off the street? I conjecture, not very useful. For 'complete the transition off fossil fuels'? Extremely useful, no one in that field knows how to do their job without AI in summer 2026. I'm certain of this.
AI development is likely to be more compute bottlenecked than solving math problems since validation of any algorithmic improvement would likely require significant compute. But you could imagine that at this point it could be economical for a frontier lab to task 10,000 agents to work non-stop on finding novel algorithmic improvements then validating the top 50 out of 1,000 candidates on a GPT-2 sized network.
I would suggest RSI is now very close. The singularity could be less than 6 months away. I'm not saying I'd put a high probability on that, but I'd give it at least 20%, and I'd double that if looking 12 months out.
I know I'm just a crazy man shouting at the clouds, but please take to the consequences of this seriously. I understand that for whatever reason AI risk seems abstract and doesn't seem real, but this should terrify any person thinking logically about where this could all be heading.
We haven't even solved the most basic AI safety problems yet. RSI right now would almost certainly result in an extremely bad outcome for humanity.
I’m not sure it will be FOOM, maybe it will require AIs to iterate on hardware to get orders of magnitude more compute/storage/energy which would more likely require months/years, but algorithmic progress would likely saturate quickly. I guess it depends on how much you think further AI progress depends on hardware vs. software.
Sam Altman has said "If superintelligence can't discover novel physics, I don't think it's a superintelligence." Is that the test? How far away are we from AI discovering novel physics? It seems within reach.
Just maths isn't really general enough for the G in AGI.
Who is going to wade through this?
It is not peer review if it is all in one company that wants an IPO.