madsen
I don’t think there’s any AI involved. The article mentions nothing of the sort, it’s at least 8 17 years old (according to the article) and the input is 35 yes/no questions, so it’s probably just some points assigned for the answers and maybe some simple arithmetic.
Edit: Upon a closer read I discovered the algorithm was much older than I first thought.
The article mentions that one woman (Stefany González Escarraman) went for a restraining order the day after the system deemed her at “low risk” and the judge denied it referring to the VioGen score.
One was Stefany González Escarraman, a 26-year-old living near Seville. In 2016, she went to the police after her husband punched her in the face and choked her. He threw objects at her, including a kitchen ladle that hit their 3-year-old child. After police interviewed Ms. Escarraman for about five hours, VioGén determined she had a negligible risk of being abused again.
The next day, Ms. Escarraman, who had a swollen black eye, went to court for a restraining order against her husband. Judges can serve as a check on the VioGén system, with the ability to intervene in cases and provide protective measures. In Ms. Escarraman’s case, the judge denied a restraining order, citing VioGén’s risk score and her husband’s lack of criminal history.
About a month later, Ms. Escarraman was stabbed by her husband multiple times in the heart in front of their children.
It also says:
Spanish police are trained to overrule VioGén’s recommendations depending on the evidence, but accept the risk scores about 95 percent of the time, officials said. Judges can also use the results when considering requests for restraining orders and other protective measures.
You could argue that the problem isn’t so much the algorithm itself as it is the level of reliance upon it. The algorithm isn’t unproblematic though. The fact that it just spits out a simple score: “negligible”, “low”, “medium”, “high”, “extreme” is, IMO, an indicator that someone’s trying to conflate far too many factors into a single dimension. I have a really hard time believing that anyone knowledgeable in criminal psychology and/or domestic abuse would agree that 35 yes or no questions would be anywhere near sufficient to evaluate the risk of repeated abuse. (I know nothing about domestic abuse or criminal psychology, so I could be completely wrong.)
Apart from that, I also find this highly problematic:
[The] victims interviewed by The Times rarely knew about the role the algorithm played in their cases. The government also has not released comprehensive data about the system’s effectiveness and has refused to make the algorithm available for outside audit.
Didn’t something similar happen in Turkey with Erdogan a few years back? Pretty sure he was accused of being behind it himself too; don’t know what the final verdict was though.
I think it’s a pretty common accusation, just like when a politician is attacked, someone will invariably suggest that they staged it in order to get more support.
This is such a fun and insightful piece. Unfortunately, the people who really need to read it never will.
Couldn’t you just program it to start (and stop) at a given time, or make a note of how long it says on the display that it’ll take?
It seems (to me) like a very, very minor improvement for a huge cost, namely that your washing machine is on your network and is internet connected.