Comment by self@awful.systems on Copilot AI calls journalist a child abuser, Microsoft tries to launder responsibility

Copilot then listed a string of crimes Bernklau had supposedly committed — saying that he was an abusive undertaker exploiting widows, a child abuser, an escaped criminal mental patient. [SWR, in German]

These were stories Bernklau had written about. Copilot produced text as if he was the subject. Then Copilot returned Bernklau’s phone number and address!

and there’s fucking nothing in place to prevent this utterly obvious failure case, other than if you complain Microsoft will just lazily regex for your name in the result and refuse to return anything if it appears

it helps they did it to someone with contacts and it was on prime time news telly

god, so this is actually the best the AI researchers can do with the tools they’ve shit out into the world without giving any thought to failure cases or legal liability (beyond their manager on ~~slack~~Teams claiming it’s been taken care of)

so fuck it, let’s make the defamation machine a non-optional component of windows. we’ll just make it a P0 when someone who could actually get us in legal trouble complains! everyone else is a P2 that never gets assigned.

so this is actually the best the AI researchers can do

Highly unlikely. This is what corporation’s public facing products can do.

are there mechanisms known to researchers that Microsoft’s not using that can prevent this type of failure case in an LLM without resorting to whack-a-mole with a regex?

To be blunt, LLMs are one of the stupider ways to try and use AI. There is incredible potential in many other applications which don’t attempt to interface with something as irrational and unpredictable as people.

I agree; LLMs and generative AI are indelibly a product of capitalism, and they can’t exist without widespread theft, exploitation of labor, massive concentrations of capital, and a willingness to destroy the environment. they are the stupidest use of technology I’ve ever seen, and after cryptocurrencies the bar for stupid was pretty fucking high. that the products themselves obscure the theft and exploitation that went into training them is a feature for the corporations developing this horseshit, not a bug.

and that’s why it’s notable that the self-described AI researchers behind these garbage products can’t even do basic shit like have the LLM not call a journalist a pedophile without resorting to an absolute hack that won’t scale. there’s no fixing LLMs; systemically, they are what they are. and now this absolute horseshit is a component of what’s unfortunately still the dominant desktop operating system.

Yeah there’s already a lot of this in play.

You run the same query multiple times through multiple models and do a web search looking for conflicting data.

I’ve had copilot answer a query, then erase the output and tell me it couldn’t answer it after about 5 seconds.

I’ve also seen responses contradict themselves later paragraphs saying there are other points of view.

It would be a simple matter to have it summarize the output it’s about to give you and dump the output of it paints the subject in a negative light.

It would be a simple matter to have it summarize the output it’s about to give you and dump the output of it paints the subject in a negative light.

lol. like that’s a fix

(Hindenburg, hitler, great depression, ronald reagan, stalin, modi, putin, decades of north korea life, …)

It would be a simple matter to have it summarize the output it’s about to give you and dump the output of it paints the subject in a negative light.

“it can’t be that stupid, you must be prompting it wrong”

Exactly, and all of this is a simple matter of having multiple models trained on different instances of the entire public internet and determining whether their outputs contradict each other or a web search.

I wonder how they prevented search engine results from contradicting data found through web search before LLMs became a thing?

llms are (approximately) advanced versions of predictive text, any censorship will make them worse.

worse at what, exactly?

How do you measure good/bad at predicting words? What’s the metric? Cause it doesn’t seem to be “the words make factual sense” if you’re defending this.

lazily regex

I’m having a sneaking suspicion that this is what they do for all the viral ‘here the LLM famously says something wrong’ problems, as I don’t think they can actually reliably train the model it made an error.

That’s the most straightforward fix. You can’t actually fix the output of an LLM, so you have to run something on the output. You can have it scanned by another AI but that costs money and is also fallible. Regex/delete is the most reliable way to censor.

Yes, and then the problem is that this doesn’t really scale well. Esp as it is always hard to regexp all the variants correctly without false positives and negatives. Time to regexp html ;).

Yeah, and you can really see this in image generation. There’s often blocks on using the names of celebrities in the prompts, but if you misspell the names enough it can bypass the censor, and the image generator still understands it.

Big brain tech dude got yet another clueless take over at HackerNews etc? Here’s the place to vent. Orange site, VC foolishness, all welcome.

This is not debate club. Unless it’s amusing debate.

For actually-good tech, you want our NotAwfulTech community

Copilot AI calls journalist a child abuser, Microsoft tries to launder responsibility(pivot-to-ai.com)

TechTakes

!techtakes@awful.systems

Community stats

Community moderators