corbin
Even better, we can say that it’s the actual hard prompt: this is real text written by real OpenAI employees. GPTs are well-known to easily quote verbatim from their context, and OpenAI trains theirs to do it by teaching them to break down word problems into pieces which are manipulated and regurgitated. This is clownshoes prompt engineering done by manager-first principles like “not knowing what we want” and “being able to quickly change the behavior of our products with millions of customers in unpredictable ways”.
That’s the standard response from last decade. However, we now have a theory of soft prompting: start with a textual prompt, embed it, and then optimize the embedding with a round of fine-tuning. It would be obvious if OpenAI were using this technique, because we would only recover similar texts instead of verbatim texts when leaking the prompt (unless at zero temperature, perhaps.) This is a good example of how OpenAI’s offerings are behind the state of the art.
Not with this framing. By adopting the first- and second-person pronouns immediately, the simulation is collapsed into a simple Turing-test scenario, and the computer’s only personality objective (in terms of what was optimized during RLHF) is to excel at that Turing test. The given personalities are all roles performed by a single underlying actor.
As the saying goes, the best evidence for the shape-rotator/wordcel dichotomy is that techbros are terrible at words.
NSFW
The way to fix this is to embed the entire conversation into the simulation with third-person framing, as if it were a story, log, or transcript. This means that a personality would be simulated not by an actor in a Turing test, but directly by the token-predictor. In terms of narrative, it means strictly defining and enforcing a fourth wall. We can see elements of this in fine-tuning of many GPTs for RAG or conversation, but such fine-tuning only defines formatted acting rather than personality simulation.
Sometimes folks need a reminder that the Sun is an eldritch being, an elder one whose very presence scorches us and whose shrieking gibberish is blessedly quelled by the vast gulf of space, in order to appreciate the apt analogy of cosmic horror. Other times it’s more useful to think about a soggoth as, say, several hundred tons of artfully-arranged FOOF. Peace be with you, Mr. “it’s a computer doing math.”
I had to go digging for it, but previously, on Mastodon, I posted this video from “The Real Adventures of Jonny Quest”. I don’t know if this is where Yud got the idea, but it’s where I picked it up as a kid along with stuff like DNA-based computing and mind uploads. Similar stuff has been on the air ever since Carpenter’s version of The Thing in 1982, and there’s even older deeper sci-fi roots. Yud gets no more credit than Lovecraft.
I didn’t realize we had a #BigYud Fediverse tag. I gotta use that more often. Also ping @Soyweiser@awful.systems @scruiser@awful.systems to enjoy this.
I’ve thought about this angle a lot too. As an apostate Christian and practicing Pastafarian, I keenly feel the difference between high-control and low-control religious groups, and the control bothers me much more than the religiosity. BITE is still my gold standard to this day for understanding whether somebody is being coerced/controlled.
Also, if you think cultists get pissed at their beliefs being called a “cult”, watch how much more they flip out at being called a “high-control group”. It’s a very good disarming technique.
That’s 100% my weird late-night word choices. You can reuse it for whatever.
I agree with your sentiment, but the wording is careful. Scaffolding is inherently temporary. It only is erected in service of some further goal. I think what I wanted to get across is that Yud’s philosophical world was never going to be a permanent addition to any field of science or maths, for lack of any scientific or formal content. It was always a farfetched alternative fueled by science-fiction stories and contingent on a technological path that never came to be.
Maybe an alternative metaphor is that Yud wanted to develop a new kind of solar panel by reinventing electrodynamics and started by putting his ladder against his siding and climbing up to his roof to call the aliens down to reveal their secrets. A decade later, the ladder sits fallen and moss-covered, but Yud is still up there, trapped by his ego, ranting to anybody who will listen and throwing rocks at the contractors installing solar panels on his neighbor’s houses.
He’s talking like it’s 2010. He really must feel like he deserves attention, and it’s not likely fun for him to learn that the actual practitioners have advanced past the need for his philosophical musings. He wanted to be the foundation, but he was scaffolding, and now he’s lining the floors of hamster cages.
Why are techbros such shit at Lojban? It’s a recurring and silly pattern. Two minutes with a dictionary tells me that there is {seldikca} for being charged like a capacitor and {nenzengau} for charging like a rechargeable battery.