For teen girls victimized by ‘deepfake’ nudes, there is little recourse

Teenage girls in the U.S., who are increasingly targeted or threatened with fake nude photos created with artificial intelligence or other tools, have limited ways to hold them accountable or have recourse, according to lawmakers, while schools and State legislatures struggle to catch up with new technologies. legal experts and a victim now advocating for a federal bill.

Since the 2023 school year began, cases involving teenage girls victims of fake nude photos, also known as deepfakes, have proliferated around the world, including in high schools in New Jersey and Washington state.

Local police departments are investigating the incidents, lawmakers are rushing to enact new measures imposing punishments against the creators of the photos, and affected families are pushing for answers and solutions.

Unrealistic deepfakes can be created with simple photo editing tools that have been around for years. But two school districts told NBC News they believe the fake photos of teenagers that have affected their students were generated by AI.

Artificial intelligence technology is becoming more available, such as stablecasting (open source technology that can produce images from text messages) and “face swapping” tools that can place a victim’s face in place of the face of a pornographic artist in a video or photograph. .

Apps that purport to “undress” clothing photographs have also been identified as possible tools used in some cases and have been found available for free in app stores. These modern deepfakes can be more realistic looking and harder to immediately identify as fake.

“I didn’t know how complex and scary AI technology is,” said Francesca Mani, 15, a sophomore at Westfield High School in New Jersey, where more than 30 girls learned on Oct. 20 that they could have been represented in explicit AI form. -manipulated images.

“I was shocked because I and the other girls were betrayed by our classmates,” she said, “which means it could happen to anyone and everyone.”

Politicians and legal experts say there are few, if any, avenues of recourse for victims of deepfake and AI-generated pornography, which often stitches the victim’s face to a naked body.

Photos and videos can be strikingly realistic, and according to Mary Anne Franks, a legal expert on non-consensual sexually explicit media, the technology to make them has become more sophisticated and accessible.

A month after the incident at Westfield High School, Francesca and her mother, Dorota Mani, said they still do not know the identities or number of people who created the images, how many were made or if they still exist. It is also unclear what punishment the school district imposed, if any.

The city of Westfield directed its comments to Westfield Public Schools, which declined to comment. Citing confidentiality, the school district previously said NBC New York that it “would not release any information about the students accused of creating fake nude photographs, nor about the discipline they face.”

Superintendent Raymond Gonzalez told the news outlet that the district would “continue to strengthen our efforts educating our students and establishing clear guidelines to ensure these new technologies are used responsibly in our schools and beyond.”

In an email obtained by NBC News, Mary Asfendis, the high school’s principal, told parents on Oct. 20 that she was investigating claims by students that some of their classmates had used AI to create pornographic images from original photographs.

At the time, school officials believed the images created had been deleted and were not being circulated, according to the memo.

“This is a very serious incident,” Asfendis wrote, while urging parents to discuss the use of technology with their children. “New technologies have made it possible to fake images and students need to know the impact and harm those actions can cause to others.”

While Francesca has not seen the image of herself or the others, her mother said the Westfield principal told her four people identified Francesca as a victim. Francesca filed a police report, but neither the Westfield Police Department nor the prosecutor’s office responded to requests for comment.

New Jersey state Sen. Jon Bramnick said authorities expressed concern to him that the incident would only result in a “report of cyberstalking, even though it should actually rise to the level of a more serious crime.”

“If you put a naked body in a child’s face, to me that’s child pornography,” he said.

The Republican lawmaker said state laws currently stop short of punishing content creators, although the harm inflicted by real or manipulated images may be the same.

“It victimizes them in the same way that people who engage in child pornography do. Not only is it offensive to the young man, but it defames him. And you never know what is going to happen with that photograph,” he said. “You don’t know where he is once it’s broadcast, when he’ll come back and haunt the young lady.”

TO pending state bill in New Jersey, Bramnick said, would ban deepfake pornography and impose criminal and civil penalties for nonconsensual disclosure. Under the bill, a person convicted of the crime would face three to five years in prison and/or a $15,000 fine, she said.

If passed, New Jersey would join at least 10 other states that have enacted laws against deepfakes, according to Franks, a law professor and president of the Cyber ​​Civil Rights Initiative, a nonprofit group that combats nonconsensual pornography. .

State laws targeting deepfakes vary widely in their scope. Some of them, such as those in Texas and Wyoming, consider non-consensual pornographic deepfakes to be a criminal offense. Other states, such as New York, have laws that only allow victims to file a civil lawsuit.

Franks said the laws are “all over the place,” are not comprehensive and the constitutionality of the laws has been questioned.

“So we have a patchwork of criminal charges, which are going to be difficult in these cases because the perpetrators are going to be minors, so that raises its own questions,” he said.

“Probably just the tip of the iceberg”

It is unclear how many young people have been victims of AI-generated nudity.

The FBI said it is difficult to estimate the number of minors who are sexually exploited. But the agency said it has seen an increase in the number of open cases involving crimes against children. There were more than 4,800 cases in 2022, up from more than 4,100 the previous year, the FBI told NBC News.

“The FBI takes crimes against children seriously and works to investigate the facts of each allegation in a collective effort with our state, local and tribal law enforcement partners,” the agency said, adding that victims may face significant challenges in trying to stop disseminate the image or remove it from the Internet.

Franks said there are likely to be many more incidents and they will only increase.

“Whatever we’re hearing about floating to the surface is probably just the tip of the iceberg,” he said. “This is probably happening quite a bit right now, and the girls just haven’t found out yet or they haven’t figured it out or the school is covering it up.”

At Issaquah High School in Washington state, a school district representative said an incident in mid-October “involving fake AI-generated images of students” continues to impact the student body.

In the Spanish town of Almendralejo, mothers say Dozens of their high school-aged daughters have been victims of AI-generated nude photos created with an app that can “undress” clothed photos. Local police in New Jersey, Washington and Spain are investigating the high school cases.

in June public service announcementThe FBI warned that the technology used to create non-consensual pornographic photos and videos was improving and was being used for harassment and sextortion.

Meanwhile, the National Association of Attorneys General asked Congress in September to study the effects of AI on children and develop legislation to protect them from such abuses.

in a letter signed by 54 state and territory attorneys general, the group said it was concerned that “AI is creating a new frontier for abuses that makes prosecution more difficult.”

“We are in a race against time to protect our country’s children from the dangers of AI,” the letter said.

Francesca and her mother said they plan to travel to Washington, D.C., in December to personally urge members of Congress to act, while continuing to advocate for updated policies within the school system and seeking accountability for what happened.

“We all know that this is not an isolated incident,” said Dorota Mani. “It will never be an isolated incident. This will continue to happen all the time. “We have to stop pretending it’s not important.”

The rise in incidents targeting high school girls follows the proliferation of AI apps and deepfake pornography websites where such material is created, shared and sold.

TO 2019 report from Sensity, an Amsterdam-based company that tracks AI-generated media, found that 96% of deepfakes created at the time were sexually explicit and featured women who did not consent to their creation. Many victims are unaware of the existence of deepfakes.

Franks said there is nothing parents and children can do to prevent deepfakes from being created using their images. Instead, Franks said schools and local authorities should set an example for perpetrators in cases that reach the general public, to deter others from creating deepfakes.

“If you could imagine a dramatic, major response from the New Jersey school or the New Jersey authorities to make the case an example, really strict penalties, people going to jail, you might be discouraged,” Franks said . .

“In the absence of that, it will simply become another tool that men and boys use against women and girls to exploit and humiliate them and about which the law basically has nothing to say.”