Cops are now using AI to generate images of fake kids, which are helping them catch child predators online, a lawsuit filed by the state of New Mexico against Snapchat revealed this week.
According to the complaint, the New Mexico Department of Justice launched an undercover investigation in recent months to prove that Snapchat "is a primary social media platform for sharing child sexual abuse material (CSAM)" and sextortion of minors, because its "algorithm serves up children to adult predators."
As part of their probe, an investigator "set up a decoy account for a 14-year-old girl, Sexy14Heather."
Despite Snapchat setting the fake minor's profile to private and the account not adding any followers, "Heather" was soon recommended widely to "dangerous accounts, including ones named 'child.rape' and 'pedo_lover10,' in addition to others that are even more explicit," the New Mexico DOJ said in a press release.
And after "Heather" accepted a follow request from just one account, the recommendations got even worse. "Snapchat suggested over 91 users, including numerous adult users whose accounts included or sought to exchange sexually explicit content," New Mexico's complaint alleged.
"Snapchat is a breeding ground for predators to collect sexually explicit images of children and to find, groom, and extort them," New Mexico's complaint alleged.
Posing as "Sexy14Heather," the investigator swapped messages with adult accounts, including users who "sent inappropriate messages and explicit photos." In one exchange with a user named "50+ SNGL DAD 4 YNGR," the fake teen "noted her age, sent a photo, and complained about her parents making her go to school," prompting the user to send "his own photo" as well as sexually suggestive chats. Other accounts asked "Heather" to "trade presumably explicit content," and several "attempted to coerce the underage persona into sharing CSAM," the New Mexico DOJ said.
"Heather" also tested out Snapchat's search tool, finding that "even though she used no sexually explicit language, the algorithm must have determined that she was looking for CSAM" when she searched for other teen users. It "began recommending users associated with trading" CSAM, including accounts with usernames such as "naughtypics," "addfortrading," "teentr3de," "gayhorny13yox," and "teentradevirgin," the investigation found, "suggesting that these accounts also were involved in the dissemination of CSAM."