How Are Deepfakes Made? Full Pipeline 2026

Every deepfake, whether it is a film studio de-aging an actor or the fraudsters who impersonated a CFO to steal $25.6 million from engineering firm Arup, runs through the same production pipeline. A deepfake is made by collecting source material of a target, encoding their identity into a compact mathematical representation, feeding that representation through a generative neural network, synchronizing the output, blending it into a host frame, and delivering it down an attack channel. Six stages, repeated since 2017. What has changed is not the shape of the pipeline but the engine inside it, and the fact that all six stages now run on a single consumer GPU in minutes.

This guide breaks down each stage of how deepfakes are made: what happens, what it requires, which neural architecture does the work, and, critically for defenders, what telltale artifact each stage leaves behind. If you want a definition-level primer first, start with our companion explainer on what a deepfake is. If you want the architecture-first version of this material, see how deepfakes are made by architecture. This article takes the pipeline as the spine.

‍

What Is the Deepfake Pipeline?

A deepfake is synthetic media that replicates a specific real person's face, voice, or likeness, produced by deep generative models trained on examples of that person. The deepfake pipeline is the repeatable, end-to-end sequence of steps that turns raw source material (photos, audio, video) into a finished synthetic artifact ready to be published, played, or streamed.

It helps to separate two things that are often blurred together. The architecture is the type of neural network doing the generation: an autoencoder, a generative adversarial network (GAN), or a diffusion model. The pipeline is the larger workflow that the architecture sits inside: everything from scraping a target's LinkedIn videos to streaming the finished fake into a Zoom call. You can swap the architecture out and the pipeline barely changes. That modularity is exactly why deepfake production has industrialized so quickly: each stage is a replaceable component, and most components are now free, open-source, and runnable on a gaming PC.

The pipeline framing also matters for defense. Detection is usually discussed as a single yes/no verdict on a finished file, but synthetic media accumulates evidence at every stage. Understanding where each artifact is introduced is what separates a detector that generalizes from one that only works on last year's generator.

‍

The Six-Stage Pipeline at a Glance

Before the deep dives, here is the full pipeline in one view. Every targeted deepfake, whether entertainment or fraud, passes through these stages, though low-effort attacks collapse several into a single hosted tool.

‍

‍

‍

The remainder of this guide walks each stage in order.

‍

Stage 1: Target Reconnaissance and Data Collection

The pipeline begins with source material. For an impersonation deepfake, an operator gathers any media that captures the target's appearance, voice, or movement: public videos, conference talks, podcast appearances, earnings calls, social posts. The Arup attackers, according to Hong Kong police, built their fakes from publicly available video and audio of the firm's executives, the kind of footage scrapable from a corporate YouTube channel or a recorded webinar.

The headline change of the last three years is how little data is now required. The original 2017 method needed hundreds or thousands of images of each face and hours of training. Today, image-conditioned face swaps work from a single reference photo, and few-shot voice cloning produces a recognizable clone from a few seconds of clean audio. Higher fidelity still benefits from more data, but the floor has dropped through the basement. The practical implication for threat modeling is that any executive with a public profile has already shipped the attacker their training set.

The artifact this stage leaves behind is subtle but exploitable: training data is overwhelmingly frontal, so deepfakes degrade on profile views, extreme head turns, and occlusions like a hand crossing the face. Defenders sometimes prompt a live subject to turn their head precisely for this reason.

‍

Stage 2: Reference Encoding

Raw media is not fed directly into the generator. First it is compressed into embeddings: fixed-length numeric vectors that capture the essential identity information and discard everything else.

For faces, an encoder produces a face embedding that encodes structure, and for voice a speaker embedding captures pitch contour, formant structure, and timbre, the acoustic fingerprint that makes a voice recognizable. These embeddings are the currency the rest of the pipeline trades in. The shift to embedding-based, few-shot pipelines is what collapsed the barrier to entry: instead of training a bespoke model on a target, an operator now extracts an embedding and conditions a pre-trained foundation model on it at inference time. No GPU cluster, no week of training.

‍

Stage 3: The Generation Engine

This is the heart of the pipeline and the stage where the architecture matters. Deepfake research has converged on three image-and-video architectures plus a separate voice track. All of them solve the same underlying problem: they learn a probability distribution over images or audio, then sample from it. But they do it on different mathematical principles, and each leaves a different fingerprint.

‍

Autoencoders: the original face-swap method

The 2017 deepfakes ran on a paired autoencoder. The design is elegant: two faces share one encoder but get separate decoders. The shared encoder is forced to learn a general representation of "a face" (pose, expression, lighting) while each decoder learns to reconstruct one specific person. To swap person A onto person B's body, you pass B's video frame through the encoder to capture B's pose and expression, then run that representation through A's decoder to render A's face in B's pose. The synthesized face is then blended back into B's footage, historically using Poisson image editing to hide the seam.

Autoencoder swaps still power hobbyist tools such as DeepFaceLab, FaceSwap, and FaceFusion. They are simple and fast at inference but struggle outside controlled, frontal conditions and leave visible blending residuals at the chin and hairline.

Generative adversarial networks (GANs)

GANs, introduced in 2014, drove the second wave. Two networks train against each other: a generator produces candidate images, and a discriminator tries to tell real from fake. The generator improves until its output is statistically indistinguishable from real samples. GANs produced the first photorealistic synthetic faces, NVIDIA's StyleGAN line, and powered fast face-swap models like SimSwap and FaceShifter.

Their weaknesses are well documented in the research literature: training is unstable and prone to mode collapse, and the upsampling layers leave periodic high-frequency artifacts, the "checkerboard" signature that most pre-2023 detectors were quietly trained to spot.

Diffusion models: the 2026 state of the art

Diffusion models have overtaken both autoencoders and GANs for image and video synthesis since 2023. The logic runs in reverse compared to a GAN. During training, a real image has Gaussian noise added step by step until it is pure static; a neural network learns to reverse one step of that process. To generate, you start from pure noise and apply the trained denoiser repeatedly, typically 20-50 steps, until a clean image emerges. Text prompts, reference images, and identity embeddings are injected at each step through cross-attention, which is what makes "render this specific person performing this specific action" work cleanly.

Diffusion wins on three counts: higher output quality, stable training with no adversarial loop, and clean compositional conditioning. It powers Stable Diffusion, Midjourney, Sora, and Runway Gen-3. Its one drawback, the cost of running many denoising steps, has been largely engineered away by distillation and consistency models that cut generation to a handful of steps, which is what makes near-real-time diffusion video on a single GPU feasible in 2026.

The voice track

Voice cloning is a separate lineage. Audio is tokenized by a neural codec such as EnCodec or SoundStream, and an autoregressive transformer predicts the next token conditioned on a speaker embedding and the target text. This is the same family of model that underlies large language models, which is why voice and text generation advanced in lockstep. Tools like ElevenLabs and Tortoise-TTS work this way, and few-shot speaker adaptation lets them clone from a short reference clip with no per-speaker training.

‍

Stage 4: Synchronization

A face and a voice generated independently will not line up. The synchronization stage aligns mouth movement to the generated audio. The canonical tool is Wav2Lip, a GAN-based model that takes a target face video and an audio clip and rewrites the mouth region to match the speech. Its key innovation is a pre-trained "lip-sync expert" discriminator, built on the SyncNet architecture, that scores audio-visual alignment and forces the generator to focus on fine-grained lip shape rather than the surrounding image. Diffusion-era successors now exist, but the principle is unchanged: a dedicated model enforces temporal coherence between sound and lips.

This stage is where many fakes still fail under scrutiny. Imperfect sync, slight blurring at the lip boundary, and a mouth that does not quite match plosive consonants are common tells.

‍

Stage 5: Post-Processing and Blending

Synthesized output rarely drops cleanly into the host media. Post-processing is the cleanup pass: the generated face is blended into the source frame with boundary-aware techniques so the seam at the jaw and hairline disappears; color and lighting are corrected to match the host scene; and frames are interpolated to hit the target frame rate. For audio, the clone is processed for room acoustics, adding reverb so a voice sounds consistent with the channel it will arrive on, whether a phone line or a Zoom recording.

Two artifact classes are introduced or exposed here. The first is blending and lighting inconsistency: a face lit differently from its surroundings, or a boundary that flickers between frames. The second is metadata: an authentic recording usually carries camera-specific EXIF data and PRNU sensor noise that synthesized content lacks. Metadata is trivially stripped or spoofed, so it is a weak signal on its own, but its absence is one more data point.

‍

Stage 6: Delivery

The finished artifact is only useful once it reaches a victim. Delivery is the stage that turns a generated file into an attack. The three dominant channels are upload (posting to social platforms for disinformation or non-consensual imagery), playback (a cloned voice played down a phone line in a vishing call), and live streaming, piping a real-time face swap into a Zoom or Teams meeting through virtual-camera and virtual-microphone software.

The Arup case was a live, multi-person video-call deepfake: every participant the finance employee saw and heard, except himself, was synthetic, and he authorized 15 transfers totaling $25.6 million before discovering the fraud by calling head office. Live delivery is the most operationally demanding channel and the one that most directly defeats "I saw and spoke to them" as a verification standard.

‍

The takeaway from the hardware table is uncomfortable for threat modelers: the "nation-state-only" framing of 2020 is gone. A single RTX 4090, retailing around $1,800, produces 4K face-swap video at near-real-time frame rates, and cloud GPU time runs roughly $0.50-$2.00 per hour. Assume any motivated individual has access to the full pipeline.

‍

What Each Stage Leaves Behind: The Detection View

Because every stage introduces its own artifact, detection is really a question of which evidence a system is equipped to read. This is the single most important concept for anyone buying or building detection.

‍

The reason this matters: a detector trained exclusively on GAN checkerboard artifacts is blind to diffusion content, whose statistical signatures live elsewhere in the frequency spectrum. This architecture-specific brittleness is the technical root of the lab-to-production accuracy gap, and it is why detection accuracy fell sharply in 2023-2024 as generation shifted to diffusion before retraining caught up. Robust systems therefore run an ensemble that covers autoencoder, GAN, and diffusion artifacts simultaneously, plus a separate voice track. DuckDuckGoose's DeepDetector takes this ensemble approach and adds per-prediction explainability, so an analyst can see which model fired and which region of the image drove the verdict. See our breakdown of explainable AI in deepfake detection for how that works in practice.

‍

Common Misconceptions About How Deepfakes Are Made

"You need a huge dataset and a GPU cluster." True in 2017, false now. Few-shot, embedding-based pipelines generate from one photo or a few seconds of audio on a single consumer GPU.

"A deepfake is one model." It is an assembly of three to five components (generator, voice clone, lip-sync, blending, delivery), most of them swappable and open-source. The pipeline, not any single tool, is the product.

"A good detector catches everything." Detectors are architecture-specific unless deliberately built otherwise. One trained on pre-2023 GAN content will miss modern diffusion fakes. Always ask whether a detector tests against diffusion-generated content from 2024 or later.

"Metadata will tell me." Sometimes, but metadata is trivially removed or forged. High-confidence detection requires pixel-level and frequency-domain analysis, not file inspection.

"It still looks obviously fake." Not to the human eye under realistic conditions. iProov found only 0.1% of people could reliably distinguish real from synthetic media, and participants were 36% worse at spotting fake video than fake images. For the signals that are worth checking, see how to spot a deepfake.

‍

Practical Recommendations for Security and IDV Teams

Map your controls to the delivery channel, not the generation method. For high-value financial transactions, the Arup pattern argues for out-of-band callback verification and pre-agreed authorization steps that a live video deepfake cannot satisfy on its own: no transfer authorized solely inside a video call, regardless of who appears to be on it.

Do not rely on human review as a control. The data is unambiguous: people cannot see the difference, and confidence does not correlate with accuracy. Where verification is automated (onboarding, KYC, account recovery, video-conferencing), layer in detection that covers all three image-and-video architectures and a dedicated voice path, because attackers will use whichever leaves the artifact your system cannot read. For voice-led attacks specifically, which sit at the front of most impersonation fraud, our voice cloning threat analysis covers the audio pipeline in depth, and DuckDuckGoose's Waver is purpose-built for synthetic-audio detection.

‍

‍

Finally, treat the threat as economically inevitable rather than exotic. With Deloitte projecting $40 billion in U.S. generative-AI fraud losses by 2027, the question for most organizations is not whether they will face a pipeline-built deepfake but which stage of their own process it will target first.

‍

Frequently Asked Questions

How are deepfakes made, in simple terms? A deepfake is made by collecting examples of a target's face or voice, compressing their identity into a numeric representation, and feeding that into a generative neural network that produces new synthetic media in their likeness. The output is then synchronized, blended into a host frame, and delivered through an attack channel.

What are the stages of making a deepfake? Six: target reconnaissance and data collection, reference encoding, generation, synchronization, post-processing, and delivery. The sequence has been stable since 2017; what changed is the architecture inside the generation stage and the dramatic drop in how much data and hardware each stage needs.

How much data do you need to make a deepfake of someone? Far less than most people assume. A modern image-conditioned face swap can work from a single reference photo, and few-shot voice cloning needs only a few seconds of clean audio. More data improves fidelity but is no longer required for a convincing result.

What software is used to make deepfakes? The capability surface matters more than the specific names, which rotate constantly. Representative tools include DeepFaceLab and FaceFusion for face swaps, DeepFaceLive for real-time streaming into video calls, Stable Diffusion and text-to-video models for synthesis, ElevenLabs and Tortoise-TTS for voice cloning, and Wav2Lip for lip-sync. Most are free and open-source.

How long does it take to make a deepfake? A voice-only clone takes seconds. A 30-second targeted face-swap video with cloned voice takes roughly 10-30 minutes on a single consumer GPU. Real-time live deepfakes for video calls run at sub-100-millisecond latency.

Do you need to train a model to make a deepfake? Increasingly, no. Modern pipelines condition a pre-trained foundation model on a reference embedding at inference time, so "training" is effectively zero. Training a custom model from scratch still takes hours to days but is rarely necessary now.

Which stage of the pipeline is hardest to get right? Post-processing and synchronization. Boundary blending, consistent lighting across frames, and precise lip-sync to plosive consonants are where many fakes still break under close inspection, which is also why those stages are rich sources of detection artifacts.

Can a deepfake be detected at a single stage? No single stage is a reliable detector on its own. Each stage leaves a different, architecture-specific artifact, so robust detection reads evidence across the whole pipeline using an ensemble that covers autoencoder, GAN, diffusion, and voice signatures together.

Generation has become a commodity; detection has to keep pace. DuckDuckGoose's DeepDetector covers GAN, autoencoder, and diffusion artifacts in a single explainable ensemble. Talk to our team about adding deepfake detection to your IDV pipeline or video-conferencing stack.

This article is updated quarterly. Last update: Q2 2026.