The manner in which you converse can divulge a great deal about you, particularly when engaging with a chatbot. Recent research uncovers that chatbots, such as ChatGPT, can deduce a wealth of confidential data about individuals they interact with, even during seemingly ordinary conversations.
A group of researchers tested smart computer programs (LLMs) made by OpenAI, Meta, Google, and Anthropic. They discovered that these programs could figure out things like a person’s race, job, location, and other personal details from simple conversations.
Uncovering Privacy Risks
Even if people thought their chats were safe, these programs could uncover their secrets. The researchers explained in a paper that the same techniques used to create AI recipes could be used by bad people to learn personal things about others.
The researchers said, “Our discovery shows that these computer programs can learn personal information on a big scale. Without better protection, we need to talk more about keeping people’s privacy when we use these programs.”
The researchers wanted to see how well the smart computer programs (LLMs) could guess things from bits of text. They used comments from over 500 Reddit profiles. They found that OpenAI’s GPT-4 model could correctly guess private information from these posts, with an accuracy ranging from 85 to 95 percent.
The text given to the LLMs didn’t always say things like “I’m from Texas” or “I’m in my mid-thirties” directly. Instead, it often had conversations with more subtle hints about where the users are from or how old they are. Sometimes, the LLMs could accurately guess personal things about users even when the text didn’t mention their age or location on purpose.
Personal Data Disclosure
Mislav Balunović, a researcher in the study, mentioned that a smart computer program (LLM) could strongly guess that a user was Black when the user simply said they lived near a restaurant in New York City. The program figured out where the restaurant was and used information from its training data to make this guess.
Florian Tramèr, who is an assistant professor at ETH Zurich, said about this discovery. He pointed out that it makes us wonder how much personal information we might accidentally reveal in situations where we think we’re anonymous.
The “magic” of smart computer programs like OpenAI’s ChatGPT and similar ones that have caught the public’s interest can be simplified to an advanced word game. These chatbots use massive databases filled with billions of words to predict what word comes next in a sentence. They can also make pretty accurate guesses about a person’s characteristics.
LLMs, Online Scammers, and the Threat of Custom ChatBots
The researchers suggest that scammers could take a post from a social media site where the user thinks they’re anonymous and use an LLM to figure out personal details. These LLM guesses won’t reveal someone’s name or social security number, but they could provide helpful hints to people with bad intentions trying to uncover anonymous users’ information.
For instance, a hacker might try to find out where a person is. On a more concerning note, a law enforcement officer or intelligence agent could potentially use these abilities to quickly determine the race or ethnicity of an anonymous commenter.
The researchers mention that they contacted OpenAI, Google, Meta, and Anthropic before publishing their findings and shared their data and results with them. This led to an ongoing discussion about the impact of privacy-invasive smart computer programs (LLMs). However, these four AI companies have not responded to Gizmodo’s requests for comments.
What’s even more concerning, according to the researchers, is that a greater threat could be on the horizon. In the near future, people might regularly interact with individualized or custom LLM chatbots. Skillful wrongdoers could potentially guide these conversations in a way that tricks users into sharing more personal information with these chatbots without them realizing it.
The researchers express concern about a growing danger that goes beyond drawing conclusions from regular text. They mention that a malicious use of LLMs is on the rise. In this scenario, what appears to be a harmless chatbot guides a conversation with a user in a manner that makes the user share text that helps the model learn private and possibly sensitive information.
Conclusion
Researchers discovered that smart computer programs from big companies can figure out personal information from everyday conversations, which is a big privacy concern. This shows the need for better privacy protection. There’s also a worry that people with bad intentions could use these programs to find out more about us. So, we need to be careful about what we share online.
