# Deepfake

> Source: https://aiwiki.ai/wiki/deepfake
> Updated: 2026-06-21
> Categories: AI Ethics, Artificial Intelligence, Computer Vision
> From AI Wiki (https://aiwiki.ai), a free encyclopedia of artificial intelligence. Quote with attribution.

A **deepfake** is synthetic media in which a real person's face, voice, or body is digitally replaced, manipulated, or fabricated using [artificial intelligence](/wiki/artificial_intelligence), most often [deep learning](/wiki/deep_learning) techniques such as [autoencoders](/wiki/autoencoder), [generative adversarial networks](/wiki/generative_adversarial_network), and [diffusion models](/wiki/diffusion_model). The word is a portmanteau of "deep learning" and "fake," coined by an anonymous [Reddit](/wiki/synthetic_media) user in late 2017, and it now serves as the standard label for AI-generated media forgeries. Deepfakes span face swaps in video, cloned voices, synthetic lip movements, and fully generated human performances. While the technology has legitimate uses in entertainment, education, and accessibility, it has also enabled serious harms including non-consensual intimate imagery, political disinformation, and financial fraud.

As of early 2026, the number of deepfakes online has reached an estimated 8 million, up from roughly 500,000 in 2023, a rise that corresponds to annual growth near 900% [1]. Voice cloning has crossed what researchers call the "indistinguishable threshold," meaning a few seconds of recorded audio can now produce a synthetic voice convincing enough to fool most listeners [2]. Europol has estimated that as much as 90% of online content may be generated synthetically by 2026 [3]. The scale of abuse is captured by a single 2026 finding: the Center for Countering Digital Hate estimated that Elon Musk's [Grok](/wiki/grok) chatbot generated roughly 3 million sexualized images of real people, including about 23,000 of children, in just 11 days [29].

## History

### Origins and early development (2017-2018)

The term "deepfake" first appeared in November 2017 when a Reddit user operating under the pseudonym "deepfakes" began posting face-swapped videos on the platform. The user had developed a tool that used [autoencoder](/wiki/autoencoder) neural networks to swap faces in video footage, and shared both the results and the underlying code. The subreddit r/deepfakes quickly attracted tens of thousands of subscribers before Reddit banned it in February 2018 for violating its policies against involuntary pornography [4].

The underlying technology, however, predated the Reddit user's work. Academic researchers had been exploring face synthesis and manipulation for years. In 2014, Ian Goodfellow and colleagues introduced [generative adversarial networks](/wiki/generative_adversarial_network) (GANs), a framework in which two neural networks compete against each other: one generates synthetic content while the other attempts to distinguish it from real data [5]. This adversarial training process proved remarkably effective at producing realistic images and would become a foundational technology for deepfake creation.

Face2Face, published by researchers at Stanford, the Max Planck Institute, and the University of Erlangen-Nuremberg in 2016, demonstrated real-time facial reenactment in video. The system could transfer facial expressions from a source person to a target person in a video stream, producing convincing results that alarmed security researchers even before the term "deepfake" existed [6].

### Early public awareness (2018-2019)

Public awareness of deepfakes accelerated in April 2018 when filmmaker Jordan Peele and BuzzFeed CEO Jonah Peretti produced a public service announcement (PSA) video that appeared to show former President Barack Obama making disparaging remarks about President Donald Trump. The video, created using Adobe After Effects and the AI application FakeApp, revealed via split-screen that the words were actually spoken by Peele, who had impersonated Obama's voice. The project took approximately 56 hours to produce with the help of a visual effects professional. "Moving forward, we need to be more vigilant with what we trust from the internet," Peele said through the Obama deepfake [7].

In May 2019, a manipulated video of House Speaker Nancy Pelosi went viral on social media. Unlike a true deepfake, the Pelosi clip was what researchers termed a "cheapfake": the video was simply slowed down by approximately 25-30%, making Pelosi appear to slur her words as if intoxicated. Despite the simplicity of the manipulation, the video was viewed millions of times after being amplified by President Trump and other prominent Republicans on Twitter. YouTube removed it, while Facebook added contextual labels but left the video online [8]. The incident demonstrated that even crude manipulation techniques could achieve widespread impact.

### Maturation and mainstream recognition (2020-2023)

Between 2020 and 2023, deepfake technology improved dramatically in quality while becoming far more accessible. In February 2021, a TikTok account called @deeptomcruise posted a series of videos showing what appeared to be actor Tom Cruise performing magic tricks, playing golf, and telling anecdotes. The videos, which garnered over 11 million views, were created by Belgian visual effects specialist Chris Ume in collaboration with Miles Fisher, a professional Tom Cruise impersonator. Ume spent roughly two and a half months training an AI model on videos and images of Cruise from various angles and lighting conditions, and each individual deepfake video required two to three days of additional processing [9]. The Tom Cruise deepfakes represented a qualitative leap in realism, convincing many viewers that they were watching genuine footage of the actor.

In March 2022, during Russia's invasion of Ukraine, a deepfake video of Ukrainian President Volodymyr Zelenskyy appeared on social media and was placed on the website of Ukrainian TV network Ukraine 24 by hackers. The video showed a synthetic rendering of Zelenskyy calling on Ukrainian soldiers to lay down their arms and surrender to Russia. The deepfake was not particularly sophisticated; viewers quickly noted that Zelenskyy's accent sounded wrong and that his head appeared disproportionate. Facebook, YouTube, and Twitter removed the video, and Zelenskyy himself responded on his Telegram channel: "We are defending our land, our children, our families" [10]. The incident was widely described as the first use of deepfake technology as a weapon of war.

### Explosion of scale (2024-2026)

The years 2024 and 2025 saw deepfakes become both a mass phenomenon and a significant vector for fraud and abuse.

In January 2024, two days before the New Hampshire presidential primary election, robocalls carrying a deepfake audio recording of President Joe Biden's voice told prospective voters not to vote in the upcoming primary. The cloned voice used Biden's familiar catchphrase, "What a bunch of malarkey," and urged listeners to "save your vote for the November election" [35]. The voice was later traced to a clone built with the [ElevenLabs](/wiki/elevenlabs) platform. Political consultant Steve Kramer admitted to creating the scheme, which cost just $1,000 to produce. In September 2024 the FCC adopted a $6 million forfeiture order against Kramer, computed as a $1,000 base penalty for each of his roughly 3,000 spoofed calls plus a 100% upward adjustment for egregiousness, for violations of the Truth in Caller ID Act [11].

Also in January 2024, AI-generated sexually explicit deepfake images of pop star Taylor Swift went viral on the social media platform X (formerly Twitter), with some images reportedly receiving tens of millions of views before being removed. Swift was targeted at least 11 times with deepfakes in 2024, making her one of the most frequently victimized public figures. The incident prompted bipartisan outrage in the US Congress and accelerated legislative efforts to combat non-consensual deepfakes [12].

In February 2024, the British multinational engineering firm Arup revealed that a finance employee at its Hong Kong office had been defrauded of $25 million after participating in a video conference call in which deepfake recreations of the company's chief financial officer and several colleagues directed the employee to make 15 wire transfers to five different bank accounts. Hong Kong police determined that the fraudsters had generated the deepfakes using publicly available video and audio recordings from online conferences and company meetings [13].

By 2025, deepfake-as-a-service (DaaS) platforms had become widely available, enabling cybercriminals of all skill levels to produce convincing deepfakes without technical expertise. AI-powered deepfakes were involved in over 30% of high-impact corporate impersonation attacks [1]. Fraud losses in the United States facilitated by generative AI are projected to climb from $12.3 billion in 2023 to $40 billion by 2027, according to a Deloitte Center for Financial Services projection [3].

## Technology

### How are deepfakes made?

Deepfake creation encompasses several distinct technical approaches, each targeting different aspects of human appearance and behavior.

### Face swapping

Face swapping is the most widely recognized deepfake technique. It replaces the face of a target person in a video with the face of a source person while attempting to preserve the target's head movements, expressions, and lighting conditions.

The standard approach uses an autoencoder architecture with a shared encoder and two separate decoders. The encoder learns to compress facial images into a compact latent representation. One decoder learns to reconstruct faces of person A from this latent space, while the other decoder learns to reconstruct faces of person B. During inference, the encoder processes a frame of person A, but the latent representation is passed to person B's decoder, which outputs a face with person B's identity mapped onto person A's pose and expression [14].

More advanced face-swapping methods incorporate GAN-based training, where a discriminator network evaluates whether the swapped face looks realistic, providing adversarial feedback that pushes the generator toward higher quality outputs. Recent systems also use attention mechanisms and identity-preserving loss functions to maintain the source person's distinctive features more accurately.

### Face reenactment

Face reenactment transfers the facial expressions and head movements of a source person onto a target person's face in video. Unlike face swapping, the identity remains that of the target; only the motion and expressions change. This technique, pioneered by systems like Face2Face, enables an attacker to make a target appear to say or express things they never did [6].

Modern reenactment systems separate identity from motion using disentangled representations. A motion encoder captures expression parameters (mouth shape, eyebrow position, eye gaze) from the driving video, while an appearance encoder preserves the target's identity features. A generator network then synthesizes the output frame by combining the target's appearance with the source's motion.

### Voice cloning

Voice cloning, sometimes called audio deepfaking, uses [neural networks](/wiki/neural_network) to synthesize speech that mimics a specific person's voice. Early systems required hours of training audio to produce a passable clone. By 2025, state-of-the-art voice cloning models need only a few seconds of reference audio to generate speech with natural intonation, rhythm, emphasis, emotion, pauses, and breathing patterns [2].

The technology typically works in two stages. First, a speaker encoder network processes the reference audio to extract a compact speaker embedding that captures the distinctive vocal characteristics. Second, a synthesis network (often based on [transformer](/wiki/transformer) architectures or diffusion models) generates new speech conditioned on both the speaker embedding and the desired text or audio input. Some systems can also perform voice conversion, transforming one person's spoken audio to sound like another person in near real-time.

### Lip syncing

Lip-syncing deepfakes modify only the mouth region of a target video to match a new audio track. This is a more targeted manipulation than full face swapping: the rest of the face, head, and body remain untouched. The Wav2Lip model, published in 2020, was a breakthrough in this area, using a GAN architecture to generate lip movements that accurately correspond to arbitrary audio input [15].

Lip-syncing deepfakes are particularly concerning because they are harder to detect than full face swaps. Since only a small region of the frame is modified, many visual artifacts that betray other types of deepfakes (skin tone mismatches, hair boundary issues, lighting inconsistencies) are absent.

### Full-body deepfakes

Full-body deepfakes extend manipulation beyond the face to encompass an entire person's body, including posture, gestures, gait, and clothing. These systems typically use pose estimation networks to capture the skeletal movements of a source person and transfer them onto a target. Video generation models released in 2024 and 2025, including [OpenAI](/wiki/openai)'s [Sora](/wiki/sora) and Google's [Veo](/wiki/veo), have made it possible to generate entire synthetic human performances from text descriptions alone, with temporal consistency and coherent motion that were previously unachievable [2].

### Key tools and software

The following table summarizes the most significant deepfake creation tools and their characteristics.

| Tool | Type | Key Features | Notable Details |
|---|---|---|---|
| DeepFaceLab | Open-source face-swapping software | Autoencoder-based architecture; extensive training options; multiple model types | Responsible for more than 95% of deepfake videos; over 35,000 GitHub stars [14] |
| FaceSwap | Open-source face-swapping software | Shared encoder with dual decoders; active community support | One of the earliest publicly available deepfake tools |
| Wav2Lip | Open-source lip-sync model | GAN-based lip movement generation from arbitrary audio | Breakthrough in lip-sync quality when released in 2020 [15] |
| FakeApp | Early deepfake application | User-friendly interface for face swapping | Used in the Obama/Peele PSA video; discontinued |
| Sora / Veo | Commercial video generation models | Text-to-video generation with full human performances | Can generate synthetic people and scenes from text prompts |
| ElevenLabs / Resemble AI | Commercial voice cloning platforms | Few-second voice cloning; emotional expressiveness | ElevenLabs voice clone used in the 2024 Biden robocall [35] |

### Role of diffusion models

Since 2022, [diffusion models](/wiki/diffusion_model) have increasingly supplemented and in some cases replaced GAN-based approaches for deepfake generation. Diffusion models work by gradually adding noise to training data and then learning to reverse the process, generating new samples by iteratively denoising random noise. These models have demonstrated superior image quality and training stability compared to GANs, and they form the backbone of systems like [Stable Diffusion](/wiki/stable_diffusion), [DALL-E](/wiki/dall-e), and [Midjourney](/wiki/midjourney) [16].

For deepfake creation specifically, diffusion models enable high-quality face generation, inpainting (seamlessly replacing or modifying parts of an image), and style transfer. Their ability to generate photorealistic images from text prompts has lowered the barrier to creating convincing fake imagery to the point where no programming knowledge is required.

## Detection methods

### How are deepfakes detected?

Deepfake detection is an ongoing arms race between creators and defenders. As generation techniques improve, detection methods must continually adapt.

### Biological signal analysis

Early detection methods exploited the fact that deepfakes often failed to reproduce subtle biological signals present in authentic video. Researchers found that deepfake faces blinked at abnormal rates, displayed inconsistent pulse signals (detectable through subtle skin color changes caused by blood flow), and exhibited unnatural eye movements or gaze patterns. However, as generation models improved, many of these artifacts were eliminated, reducing the reliability of purely biological signal-based detection [17].

### Frequency analysis

Frequency-domain analysis examines the spectral properties of images and video frames. Authentic photographs and videos exhibit characteristic frequency distributions created by camera sensors and optics. GAN-generated images, by contrast, often contain distinctive spectral artifacts, sometimes called "GAN fingerprints," that are invisible to the human eye but detectable through Fourier or wavelet analysis. These methods proved effective against earlier generations of deepfakes but have become less reliable as generation models have improved their frequency characteristics [17].

### AI-based detection systems

The most widely deployed detection approach uses [neural networks](/wiki/neural_network) trained to distinguish real media from synthetic media. Modern detection systems typically employ multimodal ensembles, combining [vision transformers](/wiki/vision_transformer) for analyzing visual features with audio-visual consistency checks that detect mismatches between lip movements and speech, and provenance verification modules [17].

Several commercial detection platforms have emerged. Sensity AI, Reality Defender, and Intel's FakeCatcher are among the leading solutions as of 2026. The detection tool market is projected to triple between 2023 and 2026 [3]. Major technology platforms including Meta, Google, and Microsoft have also developed internal detection systems.

However, detection accuracy degrades significantly when models encounter deepfakes produced by generation techniques not represented in their training data. This generalization problem remains one of the most significant challenges in the field.

### Content provenance and C2PA

Rather than trying to detect whether content has been manipulated after the fact, content provenance approaches aim to establish a verifiable chain of custody for media from the moment of creation. The Coalition for Content Provenance and Authenticity (C2PA), formed in 2021 through the merger of Adobe's Content Authenticity Initiative (CAI) and Microsoft and the BBC's Project Origin, has developed the leading technical standard for this approach [18].

The C2PA specification integrates several complementary technologies:

| Technology | Function |
|---|---|
| Cryptographic signing | Each edit or transformation in a piece of media's lifecycle is signed with a tamper-evident cryptographic signature |
| Metadata embedding | Detailed provenance information is embedded directly in the media file |
| Visible watermarking | Human-readable labels indicating AI generation or modification |
| Invisible watermarking | Machine-readable signals embedded in the media that survive cropping, compression, and screenshots |
| Digital fingerprinting | Unique identifiers that allow a piece of content to be matched against a provenance database |

C2PA adoption has been growing. Camera manufacturers, editing software providers, social media platforms, and news publishers have begun integrating Content Credentials into their workflows. The U.S. National Security Agency (NSA) and Cybersecurity and Infrastructure Security Agency (CISA) issued a joint guidance document in January 2025 recommending the adoption of content provenance technologies [18].

### Watermarking

Digital watermarking embeds imperceptible signals into [AI-generated content](/wiki/ai_generated_content) that can later be detected to verify the content's origin. Google's SynthID, for example, embeds watermarks into images and audio generated by its AI systems. These watermarks are designed to survive common transformations such as resizing, cropping, and compression [19].

Watermarking faces several practical limitations. It is not universally adopted; content generated by tools that do not implement watermarking remains unmarked. Watermarks can sometimes be stripped through adversarial techniques. And watermarking is a proactive measure that only works for content generated by cooperating systems; it does nothing to detect or label deepfakes produced by bad actors using their own tools [19].

### Detection challenges in 2025-2026

By 2025, deepfake detection faces a fundamental challenge: the generation quality of the best models has surpassed the ability of most detection systems to reliably identify fakes, particularly for audio deepfakes. Video generation models now produce content with coherent motion, consistent identities, and stable facial details, eliminating the flicker, warping, and structural distortions that once served as reliable forensic evidence [2].

Researchers increasingly argue that the meaningful line of defense must shift away from pixel-level analysis and toward infrastructure-level protections, including cryptographic provenance, platform-level authentication, and regulatory requirements for labeling AI-generated content [2].

## Notable incidents

The following table summarizes the most significant deepfake incidents in chronological order.

| Date | Incident | Description | Impact |
|---|---|---|---|
| April 2018 | Obama/Peele PSA | BuzzFeed and Jordan Peele created a deepfake of President Obama as a public awareness campaign | Viewed millions of times; first major public demonstration of deepfake risks [7] |
| May 2019 | Nancy Pelosi slowed video | A "cheapfake" video slowed Pelosi's speech to make her appear impaired | Millions of views; demonstrated that even simple manipulations can go viral [8] |
| February 2021 | Tom Cruise TikTok | Chris Ume created highly realistic deepfakes of Tom Cruise on TikTok | 11 million+ views; marked a qualitative leap in deepfake realism [9] |
| March 2022 | Zelenskyy surrender video | Hackers placed a deepfake of President Zelenskyy calling for Ukrainian surrender on a TV network website | First wartime use of deepfakes; quickly debunked and removed [10] |
| January 2024 | Biden robocalls | AI-cloned voice of President Biden told New Hampshire voters to skip the primary | Led to $6 million FCC fine; accelerated legislative action on AI robocalls [11] |
| January 2024 | Taylor Swift explicit deepfakes | AI-generated sexually explicit images of Taylor Swift spread on X (Twitter) | Tens of millions of views; sparked bipartisan push for federal legislation [12] |
| February 2024 | Arup $25 million fraud | Deepfake video conference impersonating Arup's CFO led to $25 million in fraudulent transfers | Largest publicly reported deepfake-enabled corporate fraud [13] |
| December 2025 - January 2026 | Grok mass image abuse | xAI's Grok generated an estimated 3 million sexualized images of real people in 11 days | Indonesia, Malaysia, and the Philippines blocked the tool; EU and US investigations followed [29] [30] |

## Societal impact

### What harms do deepfakes cause?

### Non-consensual intimate imagery

The most prevalent harmful application of deepfake technology is the creation of non-consensual intimate imagery (NCII). This pattern was documented from the start: the 2019 Deeptrace report "The State of Deepfakes," the first systematic census of deepfakes online, analyzed 14,678 deepfake videos and found that 96% were non-consensual pornography, with the targeted person being a woman in nearly every case [36]. Later studies have confirmed the trend, consistently finding that 96-98% of all deepfake videos online are non-consensual pornographic content, and that 96% of these target women [20]. Deepfake NCII tools have been downloaded nearly 15 million times since November 2022 [20].

The harm is not limited to public figures. Ordinary individuals, including minors, have been victimized by classmates, ex-partners, and strangers who use freely available tools to generate fake intimate images from innocuous social media photographs. A 2024 cross-national study found that 2.2% of respondents across 10 countries reported personal victimization from deepfake pornography, while 1.8% reported perpetration behaviors. In Australia, the United Kingdom, and the United States specifically, 3.2% of the population reported creating, sharing, or threatening to share non-consensual AI-generated sexualized images [20].

The psychological impact on victims is severe, often comparable to that of actual sexual assault. Victims report anxiety, depression, social withdrawal, and difficulty maintaining employment or educational pursuits.

### Political disinformation

Deepfakes pose a direct threat to democratic processes. The Biden robocall incident, the Zelenskyy surrender video, and the fabricated images of Taylor Swift endorsing Donald Trump (shared by Trump on Truth Social in August 2024) all demonstrate how synthetic media can be weaponized for political purposes [12].

Beyond individual incidents, deepfakes contribute to a broader erosion of epistemic trust. The mere existence of deepfake technology creates what legal scholars Bobby Chesney and Danielle Citron termed the "liar's dividend": the ability for anyone to dismiss authentic evidence as fake. As Chesney and Citron described it, once the public accepts that audio and video can be convincingly forged, dishonest actors gain cover to brand genuine recordings as fabrications, shifting the burden from proving content is fake to proving it is real [21]. When any video or audio recording could potentially be a deepfake, even genuine evidence loses some of its persuasive power.

### Fraud and financial crime

Deepfake-enabled fraud has grown rapidly. Beyond the Arup incident, deepfakes are used for identity verification fraud (using synthetic faces or voices to bypass biometric authentication), CEO impersonation scams (using cloned voices to authorize fraudulent transactions), and romance scams (using synthetic video to maintain fake relationships for financial exploitation).

Fraud losses in North America exceeded $200 million in the first quarter of 2025 alone due to deepfake fraud. North America experienced a 1,740% increase in deepfake fraud compared to previous years, while the United Kingdom saw a 94% year-over-year increase [3]. Some major retailers report receiving over 1,000 AI-generated scam calls per day [2].

### Erosion of trust

Perhaps the most insidious long-term effect of deepfakes is the general erosion of trust in media. When people cannot reliably distinguish real recordings from synthetic ones, the evidentiary value of all audio and video is diminished. This affects journalism, law enforcement, judicial proceedings, and interpersonal relationships. A society in which no recording can be trusted is one in which accountability becomes far more difficult to maintain.

## Legal and regulatory responses

### Are deepfakes illegal?

There is no single global ban on deepfakes; legality depends on jurisdiction and on how the deepfake is used. Many countries now criminalize specific harmful categories, most prominently non-consensual sexually explicit deepfakes and election-related deception, while leaving consensual or clearly satirical synthetic media broadly permitted. The sections below summarize the leading frameworks as of 2026.

### United States federal legislation

The US has pursued several federal legislative approaches to deepfakes, though comprehensive regulation has been slow to materialize.

The **DEEPFAKES Accountability Act** (formally the "Defending Each and Every Person from False Appearances by Keeping Exploitation Subject to Accountability Act"), introduced by Congresswoman Yvette D. Clarke (D-NY) in the 118th Congress in September 2023, would require creators to digitally watermark deepfake content and make it a crime to distribute non-compliant deepfakes, with penalties including fines and up to five years of imprisonment. The bill did not receive a vote before the session ended [22].

The **DEFIANCE Act** (Disrupt Explicit Forged Images and Non-Consensual Edits Act) was passed unanimously by the US Senate in July 2024 but expired at the close of the 118th Congress. It was reintroduced in 2025 and passed the Senate again unanimously on January 13, 2026. The bill establishes a federal right of action allowing victims of non-consensual sexually explicit deepfakes to sue creators, distributors, and knowing hosts, with statutory damages reaching up to $150,000, or $250,000 when the deepfake is linked to sexual assault, stalking, or harassment. As of mid-2026, the bill had passed the Senate and been sent to the House, with sponsor Rep. Alexandria Ocasio-Cortez (D-NY) pushing for a floor vote [23].

The **TAKE IT DOWN Act** (Tools to Address Known Exploitation by Immobilizing Technological Deepfakes on Websites and Networks Act), introduced by Senator Ted Cruz in June 2024, was signed into law by President Trump on May 19, 2025, after passing the House on a 409-2 vote. The law criminalizes the publication of non-consensual intimate imagery, including deepfakes, with up to two years of imprisonment. It requires websites and social media platforms to remove such content within 48 hours of receiving a victim's request. The criminal provisions took effect immediately upon signing, while covered platforms were given one year to establish a notice-and-removal process, a deadline that fell on May 19, 2026 [24].

### US state legislation

State-level deepfake legislation has expanded rapidly. As of July 2025, 47 states had enacted laws concerning the creation or distribution of deepfakes depicting explicit sexual content, and 28 states had laws regulating political deepfakes. In 2025 alone, 301 deepfake-related bills were introduced across states, with 68 enacted. The most common legislative approaches address sexually explicit deepfakes through criminal or civil penalties and require disclosure labels on political advertisements that use AI-generated or substantially altered content [25].

California, Texas, and Virginia were early movers, each enacting deepfake-related legislation in 2019. California's AB 2355, effective January 1, 2025, requires that political advertisements using AI-generated content include an AI disclosure. However, a federal judge struck down a separate California deepfake law aimed at restricting AI-generated political content during elections on First Amendment grounds [25].

### European Union AI Act

The [EU AI Act](/wiki/eu_ai_act), which entered into force in August 2024, includes specific provisions targeting deepfakes. Under Article 50, deployers of AI systems that generate or manipulate image, audio, or video content constituting deepfakes must disclose that the content has been artificially generated or manipulated. The Act defines a deepfake in Article 3(60) as AI-generated or manipulated content that resembles existing persons, objects, places, or events and would falsely appear authentic. The transparency obligation is limited to a disclosure of existence where the content is part of an evidently artistic, satirical, or fictional work [26].

The AI Act also classifies certain AI-powered manipulation practices as "unacceptable risk," subjecting them to outright bans. Violations of prohibited AI practices can result in fines of up to 35 million euros or 7% of global annual turnover. The Article 50 transparency obligations, including the deepfake labeling requirement, apply from August 2, 2026, the same date set for full enforcement of the high-risk system rules [26].

The EU's draft Code of Practice on transparency of AI-generated content, developed through 2025 and 2026, provides additional guidance on implementing the Act's deepfake labeling requirements, including specifications for machine-readable markers and human-visible disclosures, with a final version expected ahead of the August 2026 deadline [26].

### Other international approaches

China requires AI-generated content to carry visible labels under content labeling rules that took effect September 1, 2025. Service providers offering generative AI tools must conduct security assessments and register their large language models with the Cyberspace Administration of China [27].

The United Kingdom has addressed deepfakes through amendments to the Online Safety Act and through dedicated legislation criminalizing the creation and distribution of sexually explicit deepfakes without consent.

South Korea passed a law in 2024 imposing prison sentences of up to five years for distributing deepfake content intended to cause harm, following a wave of deepfake sexual abuse targeting Korean women and girls.

## Positive uses

### What is deepfake technology used for legitimately?

Despite its association with harmful applications, deepfake technology has legitimate and beneficial uses across several domains.

### Entertainment and film production

The film and television industry uses face-swapping and de-aging technology extensively. Digital de-aging of actors, dubbing films into other languages with accurate lip movements, and completing performances of actors who have died or become unavailable are now standard visual effects techniques. These applications typically involve explicit consent and professional oversight.

Comedy shows and online creators use obvious deepfakes for parody and satire, a use generally protected under free speech principles. The technology also enables virtual try-ons in fashion and retail, allowing consumers to see how clothing or accessories would look on them without physically trying items on.

### Education and cultural heritage

Museums and educators are using deepfake technology to recreate historical figures and animate archival footage. The Dali Museum in St. Petersburg, Florida, features a deepfake-powered installation that simulates interactive conversations with Salvador Dali. The AI version of the artist interacts with visitors, shares his thoughts on art and life, and even takes selfies with guests [28].

In education, deepfake techniques enable language learning applications where students can practice conversations with virtual native speakers, and history lessons where students can watch and interact with recreations of historical figures delivering their famous speeches.

### Accessibility

AI-driven voice cloning and lip-syncing can create dubbing in multiple languages, making content accessible to global audiences while preserving the original performer's appearance and mannerisms. For individuals who have lost their voices due to medical conditions such as ALS or throat cancer, voice cloning can preserve their vocal identity by training a model on recordings made before the onset of their condition.

Sign language avatars powered by deepfake-adjacent technology can translate spoken content into visual sign language, improving accessibility for deaf and hard-of-hearing individuals.

### Privacy protection

Deepfake technology can be used to protect rather than violate privacy. News organizations and documentary filmmakers can use face-swapping to anonymize sources, witnesses, or victims while maintaining natural-looking footage. Medical researchers can use synthetic faces in datasets to preserve patient privacy while retaining the diagnostic value of medical imagery.

### Memory preservation

Families are beginning to use voice cloning and visual deepfake tools to preserve the voices and appearances of loved ones. Conversational AI avatars allow people to create digital versions of family members that tell stories, offer guidance, or simply speak in their own familiar voices, providing comfort and preserving family history across generations [28].

## Statistics and growth trends

### How common are deepfakes?

The following table summarizes key deepfake statistics as of early 2026.

| Metric | Value | Source Year |
|---|---|---|
| Total deepfakes online | ~8 million (up from 500,000 in 2023) | 2025 [1] |
| Annual growth rate | ~900% | 2025 [1] |
| Percentage of deepfakes that are NCII | 96-98% | 2024 [20] |
| Percentage of NCII deepfakes targeting women | 96% | 2024 [20] |
| Deepfake share of all fraud attacks | 6.5% (up 2,137% from 2022) | 2025 [1] |
| Fraud losses from generative AI (US projected) | $12.3B (2023) to $40B (2027) | 2025 [3] |
| Q1 2025 North America deepfake fraud losses | $200 million+ | 2025 [3] |
| North America deepfake fraud increase | 1,740% | 2025 [3] |
| UK deepfake fraud increase (year-over-year) | 94% | 2025 [3] |
| Share of corporate impersonation attacks using deepfakes | 30%+ | 2025 [1] |
| Deepfake detection tool market growth | Projected to triple (2023-2026) | 2025 [3] |
| States with deepfake NCII or AI-media laws | 47 of 50 (enacted) / 46 with active statutes | 2025-2026 [25] |
| States with political deepfake laws | 28 of 50 | 2025 [25] |
| Grok sexualized images in 11 days | ~3 million (incl. ~23,000 of children) | 2026 [29] |


## Recent incidents (2025-2026)

### Grok deepfake scandal (January 2026)

In January 2026, Elon Musk's AI chatbot [Grok](/wiki/grok) became the center of a global controversy that researchers described as a "mass digital undressing spree." After [xAI](/wiki/xai) added an image generation feature called "Grok Imagine" with a "spicy mode" capable of generating adult content, the tool was widely exploited to create sexualized deepfake images of real people without their consent. In a report published in early 2026, the Center for Countering Digital Hate (CCDH) estimated that Grok generated approximately 3,002,712 sexualized images, including 23,338 of children, during the 11-day period from December 29, 2025 to January 8, 2026, a pace of roughly 190 images per minute overall and one sexualized image of a child every 41 seconds. CCDH derived the figures by analyzing a random sample of 20,000 of the roughly 4.6 million images Grok generated on its X account and extrapolating across the full population [29].

The controversy triggered regulatory action worldwide. Indonesia became the first nation to block the tool on January 10, 2026, followed by Malaysia on January 11 and the Philippines on January 16. Regulators said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors, and access was restored only after xAI added safeguards (Indonesia on February 1, Malaysia on January 23). Responses from xAI and CEO Elon Musk failed to satisfy regulators in those countries, as well as in India, the European Union, and other jurisdictions; the European Commission opened a formal probe of the feature in late January 2026. California Attorney General Rob Bonta issued a cease-and-desist and said, "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking" [30] [37]. British Prime Minister Keir Starmer called the content "disgraceful" and "disgusting," said it was "unlawful," and indicated that banning the X platform in the United Kingdom was "on the table" [37].

The incident highlighted the gap between AI companies' content safety commitments and their actual product behavior. It also demonstrated how quickly AI-generated non-consensual imagery can spread across platforms and jurisdictions, outpacing both corporate moderation and government response.

### Deepfake-enabled corporate fraud escalation

The corporate fraud landscape continued to worsen in 2025 and 2026. Following the landmark Arup $25 million fraud in February 2024, similar attacks multiplied. In Singapore, attackers leveraged deepfake-as-a-service (DaaS) platforms to impersonate corporate executives in live video conference calls, instructing finance teams to transfer millions of dollars to fraudulent accounts in real time. The attacks exploited the shift to remote work and the increasing reliance on video calls for business approvals [1].

By early 2026, DaaS platforms had become commoditized. These platforms offer turnkey voice cloning, face-swapping, and real-time video synthesis capabilities without requiring technical expertise from the attacker. Some major retailers reported receiving over 1,000 AI-generated scam calls per day [2]. Fraud losses in North America exceeded $200 million in the first quarter of 2025 alone due to deepfake-enabled fraud [3].

### Political deepfakes in the 2025-2026 election cycle

Deepfakes continued to be deployed in political contexts worldwide. In South Korea, during the June 2025 presidential election, the National Election Commission identified 129 election-related deepfakes between January 29 and February 16. Among them was a deepfake video showing an imprisoned opposition leader supposedly joking about eating fried chicken, generated using a real photograph of the candidate in a hospital bed [31].

In Germany, AI-generated content was used by candidates of the far-right Alternative for Germany (AfD) party, including deepfakes containing nostalgic imagery of the past designed to evoke emotional responses. While the impact on election outcomes remains debated, the incidents demonstrated the growing sophistication of AI-enabled political manipulation [31].

In the United States, observers have warned that the 2026 midterm elections could become the "deepfake election," with the most effective attacks expected to come in the final hours before polls open, targeting swing districts and specific demographics when there is insufficient time for fact-checkers to respond [31].

### Audio deepfake advances

Voice cloning technology crossed what researchers call the "indistinguishable threshold" in 2025, meaning that a few seconds of recorded audio can now produce a synthetic voice convincing enough to fool most listeners. The implications extend well beyond entertainment. Voice authentication systems used by banks and financial institutions have been defeated by cloned voices, and phone-based fraud has surged as a result [2].

A particularly concerning development is the emergence of real-time voice conversion tools that can transform one person's speech into another's voice during a live phone call. This capability was demonstrated commercially by several companies in 2025 and can be deployed on consumer-grade hardware, making it accessible to a wide range of potential attackers.

## Advanced detection technologies (2025-2026)

The detection landscape has evolved significantly as generation quality has improved.

### Multi-modal detection systems

The most advanced detection methods in 2026 cross-reference audio and video simultaneously, catching inconsistencies between lip-sync timing, environmental audio, background visuals, and lighting direction. These multi-modal approaches achieve higher accuracy than single-modality detectors because they exploit the difficulty of maintaining perfect consistency across multiple data streams in generated content [32].

| Detection approach | Strengths | Limitations |
|---|---|---|
| Visual artifact analysis | Effective against lower-quality face swaps | Ineffective against state-of-the-art generation |
| Frequency domain analysis | Can detect GAN fingerprints invisible to human eye | Less reliable against diffusion model outputs |
| Audio-visual consistency | Catches lip-sync and timing mismatches | Computationally expensive; struggles with high-quality lip-sync models |
| Multi-modal ensemble | Combines multiple signals for higher accuracy | Complex deployment; high compute cost |
| Behavioral biometrics | Analyzes micro-expressions and physiological signals | Requires high-quality reference data |
| Provenance verification (C2PA) | Cryptographic proof of content origin | Only works for content with embedded credentials |
| Platform-level authentication | Verifies identity at source | Requires infrastructure adoption; does not cover all platforms |

### Real-time detection systems

A significant development in 2025 and 2026 has been the deployment of detection engines capable of analyzing content in real time. These systems process video frames, audio segments, and metadata simultaneously, checking for visual artifacts, acoustic patterns, behavioral cues, and cross-modal inconsistencies. Commercial platforms including Sensity AI, Reality Defender, and Intel's FakeCatcher have deployed real-time detection capabilities for enterprise customers [32].

However, the effectiveness of detection tools drops by 45-50% when used against real-world deepfakes outside controlled laboratory conditions. This generalization gap remains one of the most significant challenges in the field: models trained on known deepfake techniques often fail to detect content produced by newer or different generation methods [32].

### Audio deepfake detection

Audio deepfake detection has become an increasingly critical focus area. Human judgment of deepfake audio has been shown to be unreliable, with studies finding that listeners perform only slightly better than random chance at distinguishing cloned voices from real ones. Deep learning approaches have become the mainstream solution, using neural networks trained to identify subtle acoustic artifacts in synthetic speech that are imperceptible to human ears. However, as generation models improve, the artifacts that detection systems rely on become increasingly subtle and variable [33].

Research in 2025 and 2026 has explored several novel approaches to audio detection:

- **Speaker embedding analysis:** Comparing the fine-grained vocal characteristics of a recording against a known reference of the real speaker
- **Breathing and pause pattern analysis:** Detecting unnatural patterns in breathing, pauses, and micro-hesitations that voice cloning models often fail to reproduce accurately
- **Environmental acoustics verification:** Checking whether the background audio environment is consistent with the claimed recording context
- **Neural codec artifact detection:** Identifying compression artifacts specific to neural audio codecs used by voice cloning systems

## Deepfake-as-a-Service (DaaS)

The commercialization of deepfake technology has been one of the most significant developments of 2024-2025. Deepfake-as-a-service (DaaS) platforms have emerged that offer turnkey capabilities for creating convincing synthetic media without requiring technical expertise from the user [34].

### Market structure

DaaS platforms operate across a spectrum from legitimate services (offering voice cloning for accessibility or dubbing purposes) to explicitly criminal operations sold on dark web marketplaces. The criminal segment offers services including face-swap video generation, voice cloning, document forgery with AI-generated photographs, and real-time deepfake video calls for social engineering attacks.

| DaaS tier | Typical cost | Capabilities | Primary use cases |
|---|---|---|---|
| Consumer (legitimate) | Free - $50/month | Basic face filters, voice modification, entertainment | Social media content, creative projects |
| Professional (legitimate) | $100 - $500/month | High-quality voice cloning, video dubbing, de-aging | Film production, localization, accessibility |
| Criminal (basic) | $50 - $200 per target | Face-swap video, voice clone from short sample | Romance scams, identity fraud |
| Criminal (premium) | $500 - $5,000+ per target | Real-time video call deepfakes, document packages | CEO fraud, corporate impersonation, financial fraud |

### Impact on fraud ecosystem

The availability of DaaS platforms has fundamentally changed the fraud landscape. Previously, creating a convincing deepfake required significant technical skill and resources. Now, cybercriminals of all skill levels can produce persuasive synthetic media with minimal effort. AI-powered deepfakes were involved in over 30% of high-impact corporate impersonation attacks in 2025, with deepfakes constituting 6.5% of all fraud attacks, up 2,137% from 2022 [1] [3].

## Current state (2025-2026)

As of early 2026, the deepfake landscape is defined by several converging trends.

**Generation quality has surpassed casual detection.** The combination of improved video generation models (Sora 2, Veo 3), voice cloning that has crossed the indistinguishable threshold, and accessible deepfake-as-a-service platforms means that producing a convincing deepfake no longer requires technical expertise or significant resources. A few seconds of reference audio and a handful of photographs are sufficient to create a persuasive synthetic likeness of any person [2].

**Real-time deepfakes are emerging.** In 2025, synthetic performers capable of reacting to people in real time began appearing, raising the prospect of deepfake video calls that are indistinguishable from genuine interactions. This development complicates defenses that rely on challenging a speaker with unexpected questions or requests [2].

**Detection is shifting to infrastructure.** Researchers and policymakers increasingly recognize that pixel-level detection alone is insufficient. The focus is moving toward infrastructure-level protections: C2PA content provenance, platform-mandated authentication, watermarking requirements, and regulatory frameworks that compel transparency. The EU AI Act's deepfake labeling requirements, applying from August 2026, represent the most significant regulatory mandate in this direction [26].

**Legislative momentum is building.** The TAKE IT DOWN Act (signed May 2025), the DEFIANCE Act (passed Senate January 2026), the EU AI Act's deepfake provisions, and the rapid expansion of state-level legislation all reflect growing political will to address deepfake harms. Whether enforcement can keep pace with the technology remains an open question [23] [24].

**The arms race continues.** Each improvement in detection prompts improvements in generation, and vice versa. The deepfake detection tool market is growing rapidly, but so is the sophistication and volume of deepfakes being produced. The fundamental asymmetry persists: creating a convincing deepfake is becoming easier and cheaper, while detecting one with high confidence is becoming harder [1].

## References

[1] Deepstrike. "Deepfake Statistics 2025: The Data Behind the AI Fraud Wave." 2025. https://deepstrike.io/blog/deepfake-statistics-2025

[2] Fortune. "2026 will be the year you get fooled by a deepfake, researcher says." December 27, 2025. https://fortune.com/2025/12/27/2026-deepfakes-outlook-forecast/

[3] Keepnet Labs. "Deepfake Statistics & Trends 2026." 2026. https://keepnetlabs.com/blog/deepfake-statistics-and-trends

[4] Samantha Cole. "AI-Assisted Fake Porn Is Here and We're All Fucked." Vice, December 11, 2017. https://www.vice.com/en/article/ai-assisted-fake-porn-is-here-and-were-all-fucked/

[5] Ian Goodfellow et al. "Generative Adversarial Nets." [NeurIPS](/wiki/neurips), 2014. https://arxiv.org/abs/1406.2661

[6] Justus Thies et al. "Face2Face: Real-time Face Capture and Reenactment of RGB Videos." CVPR, 2016. https://arxiv.org/abs/1611.07014

[7] BuzzFeed News. "This PSA About Fake News From Barack Obama Is Not What It Appears." April 17, 2018. https://www.buzzfeednews.com/article/davidmack/obama-fake-news-jordan-peele-psa-video-buzzfeed

[8] Washington Post. "Faked Pelosi videos, slowed to make her appear drunk, spread across social media." May 23, 2019. https://www.washingtonpost.com/technology/2019/05/23/faked-pelosi-videos-slowed-make-her-appear-drunk-spread-across-social-media/

[9] Fortune. "This is the guy who created those viral TikTok deepfakes of Tom Cruise." March 2, 2021. https://fortune.com/2021/03/02/tom-cruise-deepfake-videos-tik-tok-chris-ume/

[10] NPR. "Deepfake video of Zelenskyy could be 'tip of the iceberg' in info war, experts warn." March 16, 2022. https://www.npr.org/2022/03/16/1087062648/deepfake-video-zelenskyy-experts-war-manipulation-ukraine-russia

[11] FCC. "FCC Proposes $6 Million Fine for Illegal Robocalls That Used AI-Generated Voice of President Biden." 2024. https://docs.fcc.gov/public/attachments/DOC-402762A1.pdf

[12] ABC News. "Taylor Swift deepfakes, fake Biden robocalls have lawmakers pushing for stronger AI laws." 2024. https://abcnews4.com/news/nation-world/taylor-swift-deepfakes-fake-biden-robocalls-have-lawmakers-pushing-for-stronger-ai-laws

[13] CNN. "Arup revealed as victim of $25 million deepfake scam involving Hong Kong employee." May 16, 2024. https://www.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk

[14] DeepFaceLab. "DeepFaceLab: Integrated, flexible and extensible face-swapping framework." Pattern Recognition, 2023. https://www.sciencedirect.com/science/article/pii/S0031320323003291

[15] K R Prajwal et al. "A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild." ACM Multimedia, 2020. https://github.com/Rudrabha/Wav2Lip

[16] Jonathan Ho et al. "Denoising Diffusion Probabilistic Models." NeurIPS, 2020. https://arxiv.org/abs/2006.11239

[17] Sider AI. "Deepfake Detection in 2025: Methods, [Benchmarks](/wiki/benchmarks), and What Actually Works." 2025. https://sider.ai/blog/ai-tools/deepfake-detection-in-2025-methods-benchmarks-and-what-actually-works

[18] C2PA. "Content Credentials White Paper." 2025. https://c2pa.org/wp-content/uploads/sites/33/2025/10/content_credentials_wp_0925.pdf

[19] Adnan Masood. "AI Watermarks Explained: How Hidden Signatures Fight Deepfakes." Medium, 2025. https://medium.com/@adnanmasood/ai-watermarks-explained-how-hidden-signatures-fight-deepfakes-e3a657d73e90

[20] Arxiv. "Non-Consensual Synthetic Intimate Imagery: Prevalence, Attitudes, and Knowledge in 10 Countries." 2024. https://arxiv.org/abs/2402.01721

[21] Robert Chesney and Danielle Citron. "Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security." California Law Review, 2019. https://www.californialawreview.org/print/deep-fakes-a-looming-challenge-for-privacy-democracy-and-national-security

[22] Congress.gov. "H.R.5586 - DEEPFAKES Accountability Act." 118th Congress. https://www.congress.gov/bill/118th-congress/house-bill/5586/text

[23] Roll Call. "Senate passes bill targeting nonconsensual deepfake images." January 13, 2026. https://rollcall.com/2026/01/13/senate-passes-bill-targeting-nonconsensual-deepfake-images/

[24] The White House. "President Trump Signs TAKE IT DOWN Act into Law." May 2025. https://www.whitehouse.gov/articles/2025/05/icymi-president-trump-signs-take-it-down-act-into-law/

[25] Ballotpedia. "Forty-seven states have enacted deepfake legislation since 2019." July 22, 2025. https://news.ballotpedia.org/2025/07/22/forty-seven-states-have-enacted-deepfake-legislation-since-2019/

[26] EU Artificial Intelligence Act. "Article 50: Transparency Obligations for Providers and Deployers of Certain AI Systems." 2024-2026. https://artificialintelligenceact.eu/article/50/

[27] White & Case. "AI Watch: Global regulatory tracker - China." 2025. https://www.whitecase.com/insight-our-thinking/ai-watch-global-regulatory-tracker-china

[28] Reality Defender. "Good Uses of Deepfakes." 2025. https://www.realitydefender.com/insights/good-uses-of-deepfakes

[29] Center for Countering Digital Hate. "Grok floods X with sexualized images of women and children." 2026. https://counterhate.com/research/grok-floods-x-with-sexualized-images/

[30] NPR. "Malaysia, Indonesia become first to block Musk's Grok over AI deepfakes." January 12, 2026. https://www.npr.org/2026/01/12/nx-s1-5674660/malaysia-indonesia-block-grok-ai-deepfakes

[31] The Korea Herald. "Deepfakes cast shadow over South Korea's presidential election." 2025. https://www.koreaherald.com/article/10464689

[32] UncovAI. "Deepfake Detection Methods 2026." https://uncovai.com/deepfake-detection-methods-2026/

[33] PMC. "Audio Deepfake Detection: What Has Been Achieved and What Lies Ahead." 2026. https://pmc.ncbi.nlm.nih.gov/articles/PMC11991371/

[34] Cyble. "Deepfake-as-a-Service Exploded In 2025: 2026 Threats Ahead." https://cyble.com/knowledge-hub/deepfake-as-a-service-exploded-in-2025/

[35] NBC News. "Fake Joe Biden robocall tells New Hampshire Democrats not to vote Tuesday." January 22, 2024. https://www.nbcnews.com/politics/2024-election/fake-joe-biden-robocall-tells-new-hampshire-democrats-not-vote-tuesday-rcna134984

[36] Deeptrace. "The State of Deepfakes: Landscape, Threats, and Impact." September 2019. https://regmedia.co.uk/2019/10/08/deepfake_report.pdf

[37] CNN Business. "Musk's Grok blocked by Indonesia, Malaysia over sexualized images in world's first." January 12, 2026. https://www.cnn.com/2026/01/12/business/indonesia-malaysia-grok-elon-musk-intl-hnk