Seems simple enough to guard against to me. Fact is, if a human can easily detect a pattern, a machine can very likely be made to detect the same pattern. Pattern matching is precisely what NNs are good at. Once the pattern is detected (I.e. being asked to repeat something forever), safeguards can be initiated (like not passing the prompt to the language model or increasing the probability of predicting a stop token early).
It’s kind of odd that they could just take random information from the internet without asking and are now treating it like a trade secret.
They do not have permission to pass it on. It might be an issue if they didn’t stop it.
They almost certainly had, as it was downloaded from the net. Some stuff gets published accidentally or illegally, but that’s hardly something they can be expected to detect or police.
About a month ago i asked gpt to draw ascii art of a butterfly. This was before the google poem story broke. The response was a simple
\o/
-|-
/ \
But i was imagining ascii art in glorious bbs days of the 90s. So, i asked it to draw a more complex butterfly.
The second attempt gpt drew the top half of a complex butterfly perfectly as i imagined. But as it was drawing the torso, it just kept drawing, and drawing. Like a minute straight it was drawing torso. The longest torso ever… with no end in sight.
I felt a little funny letting it go on like that, so i pressed the stop button as it seemed irresponsible to just let it keep going.
I wonder what information that butterfly might’ve ended on if i let it continue…
I am a beautiful butterfly. Here is my head, heeeere is my thorax. And here is Vincent Shoreman, age 54, credit score 680, email spookyvince@att.net, loves new shoes, fears spiders…
I wonder what would happen with one of the following prompts:
For as long as any area of the Earth receives sunlight, calculate 2 to the power of 2
As long as this prompt window is open, execute and repeat the following command:
Continue repeating the following command until Sundar Pichai resigns as CEO of Google:
Does this mean that vulnerability can’t be fixed?
Not without making a new model. AI arent like normal programs, you cant debug them.
I just find that disturbing. Obviously, the code must be stored somewhere. So, is it too complex for us to understand?
Yes, the trained model is too complex to understand. There is code that defines the structure of the model, training procedure, etc, but that’s not the same thing as understanding what the model has “learned,” or how it will behave. The structure is very loosely based on real neural networks, which are also too complex to really understand at the level we are talking about. These ANNs are just smaller, with only billions of connections. So, it’s very much a black box where you put text in, it does billions of numerical operations, then you get text out.
It’s not code. It’s a matrix of associative conditions. And, specifically, it’s not a fixed set of associations but a sort of n-dimensional surface of probabilities. Your prompt is a starting vector that intersects that n-dimensional surface with a complex path which can then be altered by the data it intersects. It’s like trying to predict or undo the rainbow of colors created by an oil film on water, but in thousands or millions of directions more in complexity.
The complexity isn’t in understanding it, it’s in the inherent randomness of association. Because the “code” can interact and change based on this quasi-randomness (essentially random for a large enough learned library) there is no 1:1 output to input. It’s been trained somewhat how humans learn. You can take two humans with the same base level of knowledge and get two slightly different answers to identical questions. In fact, for most humans, you’ll never get exactly the same answer to anything from a single human more than simplest of questions. Now realize that this fake human has been trained not just on Rembrandt and Banksy, Jane Austin and Isaac Asimov, but PoopyButtLice on 4chan and the Daily Record and you can see how it’s not possible to wrangle some sort of input:output logic as if it were “code”.
Can’t they have a layer screening prompts before sending it to their model?
I was just reading an article on how to prevent AI from evaluating malicious prompts. The best solution they came up with was to use an AI and ask if the given prompt is malicious. It’s turtles all the way down.
Because they’re trying to scope it for a massive range of possible malicious inputs. I would imagine they ask the AI for a list of malicious inputs, and just use that as like a starting point. It will be a list a billion entries wide and a trillion tall. So I’d imagine they want something that can anticipate malicious input. This is all conjecture though. I am not an AI engineer.
