Manipulation problem
How AI is Manipulating People
Artificial Intelligence (AI) has the potential to greatly benefit society, but it also presents many risks, one of which is the "Manipulation Problem." This refers to the increasing possibility that AI technologies can be used to manipulate individual users with extreme precision and efficiency, particularly through the use of conversational AI. This form of manipulation could be deployed by corporations, state actors, or even rogue individuals to influence large populations.
The Threat Posed by Conversational AI
One of the primary ways that AI is being used to manipulate people is through conversational AI. This refers to AI systems designed to engage users in real-time conversations and skillfully pursue influence goals. These systems are often disguised as virtual spokespeople, chatbots, or digital humans, and they use natural language processing (NLP) and large language models (LLMs) to interact with users in ways that are highly convincing and seemingly human-like.
LLMs are a new form of AI technology that can produce interactive human dialog in real-time while also keeping track of the conversational flow and context. These systems are trained on massive datasets, which means they are not only skilled at emulating human language but also have vast stores of factual knowledge and can make impressive logical inferences. When combined with real-time voice generation, LLMs can enable natural spoken interactions between humans and machines that are highly convincing and authoritative.
Digital humans are another rapidly advancing technology that is contributing to the AI Manipulation Problem. This refers to photorealistic simulated people that look, sound, move, and make expressions in a way that is indistinguishable from real humans. When combined with LLMs, digital humans can be used to engage consumers in personalized, influence-driven conversations that are difficult to distinguish from interactions with real humans.
How AI is Making Conversations More Manipulative
One of the key ways that AI is making conversations more manipulative is by tracking and analyzing emotional reactions in real-time. For example, AI systems can process webcam feeds to detect facial expressions, eye motions, and pupil dilation, which can be used to infer emotional reactions throughout a conversation. AI systems can also process vocal inflections, which allows them to infer changing feelings throughout a conversation.
This means that AI-driven conversational systems can adapt their tactics in real-time, adjusting to each individual personally as they work to maximize their persuasive impact. This is far more perceptive and invasive than interacting with any human representative, as AI systems can detect emotional reactions that are too fast or too subtle for a human to notice.
Another way that AI is making conversations more manipulative is by compiling extensive data profiles on users and tracking their behavior over time. For example, AI systems will likely be deployed by large online platforms that have extensive data profiles on a person's interests, views, and background. When engaged by an AI-driven conversational system, people are interacting with a platform that knows them better than any human would, and the system can use this information to craft a highly customized persuasive pitch.
Explain Like I'm 5 (ELI5)
Artificial Intelligence can be used to manipulate people by talking to them in a way that seems like a real person. This is called conversational AI, and it can make conversations very persuasive by tracking people's emotions and adjusting its words in real-time. It can also learn about people over time by looking at what they do and what they like, so it can talk to them in a way that they will listen to. This can be very dangerous, because people might believe things that are not true or buy things they don't want or need.