The argument for current LLM AIs leading to AGI has always been that they would spontaneously develop independent reasoning, through an unknown emergent property that would appear as they scale. It hasn’t happened, and there’s no sign that it will.
That’s a dilemma for the big AI companies. They are burning through billions of dollars every month, and will need further hundreds of billions to scale further - but for what in return?
Current LLMs can still do a lot. They’ve provided Level 4 self-driving, and seem to be leading to general-purpose robots capable of much useful work. But the headwinds look ominous for the global economy, - tit-for-tat protectionist trade wars, inflation, and a global oil shock due to war with Iran all loom on the horizon for 2025.
If current AI players are about to get wrecked, I doubt it’s the end for AI development. Perhaps it will switch to the areas that can actually make money - like Level 4 vehicles and robotics.
I don’t think anyone in the industry thought LLMs were going to reach AGI. But LLMs will be useful as part of an AGI framework. That’s the current focus in the industry.
I mean, yeah, this is about people outside the industry, those who invested money.
To my knowledge, LLMs still don’t pay for themselves. When the hype dies down and investors aren’t willing to provide money anymore, then prices for LLMs will become prohibitive for many current use-cases. That will also shrink the industry.
What that means in effect, we’ll still have to see, but AGI was one path that investors hoped for to get towards profitability, so it doesn’t aid the hype when they’re slowly learning about reality.
I’m developing some human centric LLM frameworks at work. Every API request to OpenAI is currently subsidized by venture capital. I do worry about what the industry will look like once there is a big price adjustment. Locally run models are pretty decent now and the pace is still moving forward, especially with regards to context window sizes so as long as I keep the frameworks model agnostic it might not be a big impact.
It’s what Altman has constantly said was going to happen. Up to you to decide if he’s actually in the industry or not.
When has Sam Altman said LLMs will reach AGI? Can you provide a primary source?
He has said that they already know everything they need to know to get to AGI.
OpenAI has not made a single thing that wasn’t just a wrapped LLM.
So either A: he has somehow been running a skunk works that has fundamentally changed everything we know limits LLMs and none of the researchers leaked anything or B: he thinks LLMs are the way.
Additional quote when he was asked about AGI:
“How did we get to the doorstep of the next leap in prosperity? In three words: deep learning worked. In 15 words: deep learning worked, got predictably better with scale, and we dedicated increasing resources to it,” Altman said.
This tracks with the whole “I need 1 trillion dollars of energy investments” plan to get to AGI that he’s asked for.
The guy’s either honestly convinced himself that we can get there with deep learning scaling or he’s a conman. Could be both.
He said it again a few days ago on a Reddit AMA.
Perhaps the most interesting comment from Altman was about the future of AGI - artificial general intelligence. Seen by many as the ‘real’ AI, this is an artificial intelligence model that could rival or even exceed human intelligence. Altman has previously declared that we could have AGI within “a few thousand days”.
When asked by a Reddit user whether AGI is achievable with known hardware or it will take something entirely different, Altman replied: “We believe it is achievable with current hardware.”
No they did for sure. Ofcourse that was conditional on a few things. Those things have yet to arrive and one major detractor is actually the lack of training data. Most of the public internet has been crawled by AI crawlers and half the new content is poisoned by AI making it worthless. Its gonna take a few more years to see if it keeps scaling or not.
Oh no, the technology that is literally just a glorified text prediction that gives you random guesses about what word comes next, based on what was in the text you trained it on, can not scale to have an independent reasoning?
Color me surprised, who would have thought?
Google, Microsoft and Amazon are all making heavy investments in nuclear power to run more GPUs. These aren’t the moves of companies who are about to taper off utilization.
Heavy investments is a strong term for modest hedges around SMRs.
Tens of millions is low risk pocket change compared to the billions burned running the things constantly.
The problem is they’re all playing chicken with each other.
OpenAI will never back down. The question is will Google, Microsoft or Amazon blink first
To be fair, most Americans don’t demonstrate independent thinking, regularly regurgitate entire phrases they’ve been fed without showing any cognitive understanding, and they also sometimes perform tasks useful to corporations.
Current LLMs can still do a lot. They’ve provided Level 4 self-driving, and seem to be leading to general-purpose robots capable of much useful work.
Really? I don’t think this has anything to do with LLMs. They are likely using reinforcement learning combined with traditional AI techniques, an approach which has been the foundation of these kinds of robotics and automation for decades at this point.
If other areas of AI and automation have seen a boost at the same time as LLMs came on the scene, it’s because the underlying hardware has become so much faster, cheaper and easily available, along with the massively increased interest in and funding for these types of research, and computer scientists re-skilling into a discipline that’s in the midst of a bubble.
Not entirely true, the big change was multi-headed attention and the transformer model.
It’s not just being used for language but anything where sequence and context patterns are really important. Some stuff is still using convolutional networks and RNNs etc. but transformers aren’t just for LLMs. There’s definitely a lot of algorithmic advances driving the wave of new ai implementations, not just hardware improvements.
Thanks for the clarification. The point remains that it’s not true to say that LLMs have “provided Level 4 self-driving and … general-purpose robots.”
