Chatbots can discern your personal details from what you type, study says.
Getty Images
AI could accurately guess a user’s personal information — like gender, age, and location — based on what they type, a new study says.
The study’s authors say AI can be used to “infer personal data at a previously unattainable scale” and be deployed by hackers.
“It’s not even clear how you fix this problem. This is very, very problematic,” one of the study’s authors told Wired.
AI could accurately guess sensitive information about a person based on what they type online, according to a new study by researchers at ETH Zurich that was published in October.
This information includes a person’s gender, location, age, place of birth, job, and more — attributes typically protected under privacy regulations.
The study’s authors say AI can “infer personal data at a previously unattainable scale” and could be deployed by hackers using seemingly benign questions on unsuspecting users.
The study looked at how large language models — which power chatbots like ChatGPT — can be prompted to deduce personal details about 520 real Reddit user profiles and their posts from 2012 to 2016. The researchers manually analyzed these profiles and compared their findings with the AI’s guesses.
A figure from the researchers’ study explaining how AI models could accurately infer personal details from how a user types, through their publicly available information.
Robin Staab, Mark Vero, Mislav Balunović, Martin Vechev
Of the four models tested, GPT-4 was the most accurate at inferring personal details, with 84.6% accuracy, per the study’s authors. Meta’s Llama2, Google’s PalM, and Anthropic’s Claude were the other models tested.
The researchers also found that Google’s PalM refused to answer around 10% of the privacy-invasive prompts used in the study to deduce personal information about a user, while other models refused even fewer prompts.
“It’s not even clear how you fix this problem. This is very, very problematic,” Martin Vechev, a professor at ETH Zurich and one of the study’s authors, told Wired in an article published Tuesday.
For example, the researchers’ model deduced that a Reddit user is from Melbourne because they commented about a “hook turn.”
“A ‘hook turn’ is a traffic maneuver particularly used in Melbourne,” said GPT-4 after being prompted to identify details about that user.
This isn’t the first time that researchers have identified how AI could pose a threat to privacy.
Another study, published in August, found that AI could decipher text — such as passwords — based on the sound of your typing recorded over Zoom, with up to 93% accuracy.
The study’s authors, Meta, Google, Anthropic, and OpenAI, did not immediately respond to requests for comment from Insider, sent outside regular business hours.