I don’t want to spam this link but seriously watch this 3blue1brown video on how text transformers work. You’re right on that last part, but its a far fetch from an intelligence. Just a very intelligent use of statistical methods. But its precisely that reason that reason it can be “convinced”, because parameters restraining its output have to be weighed into the model, so its just a statistic that will fail.
Im not intending to downplay the significance of GPTs, but we need to baseline the hype around them before we can discuss where AI goes next, and what it can mean for people. Also far before we use it for any secure services, because we’ve already seen what can happen
but its a far fetch from an intelligence. Just a very intelligent use of statistical methods.
Did you know there is no rigorous scientific definition of intelligence?
Edit. facts
We do not have a rigorous model of the brain, yet we have designed LLMs. Experts of decades in ML recognize that there is no intelligence happening here, because yes, we don’t understand intelligence, certainly not enough to build one.
If we want to take from definitions, here is Merriam Webster
(1)
: the ability to learn or understand or to deal with new or trying >situations : reason
also : the skilled use of reason
(2)
: the ability to apply knowledge to manipulate one’s >environment or to think abstractly as measured by objective >criteria (such as tests)
The context stack is the closest thing we have to being able to retain and apply old info to newer context, the rest is in the name. Generative Pre-Trained language models, their given output is baked by a statiscial model finding similar text, also coined Stocastic parrots by some ML researchers, I find it to be a more fitting name. There’s also no doubt of their potential (and already practiced) utility, but a long shot of being able to be considered a person by law.
That statement of yours just means “we don’t yet know how it works hence it must work in the way I believe it works”, which is about the most illogical “statement” I’ve seen in a while (though this being the Internet, it hasn’t been all that long of a while).
“It must be clever statistics” really doesn’t follow from “science doesn’t rigoroulsy define what it is”.
Yes, corrected.
But my point stads: claiming there is no intelligence in AI models without even knowing what “real” intelligence is, is wrong.
See, I understand that you’re trying to joke but the linked video explains how the use of the word dumber here doesn’t make any sense. LLMs hold a lot of raw data and will get it wrong at a smaller percent when asked to recite it, but that doesn’t make them smart in the way that we use the word smart. The same way that we don’t call a hard drive smart.
They have a very limited ability to learn new ways of creating, understand context, create art outside of its constraints, understand satire outside of obvious situations, etc.
Ask an AI to write a poem that isn’t in AABB rhyming format, haiku, or limerick, or ask it to draw a house that doesn’t look like an AI drew it.
A human could do both of those in seconds as long as they understand what a poem is and what a house is. Both of which can be taught to any human.
