What is DeepFaceLab?

DeepFaceLab is an open source project that can be found on GitHub.  For a couple of years now, it has been possible to use this software to create deepfakes. The quality of the deepfake that the software makes depends partly on the quality of the data available, but mainly on the skill of the user. The famous Tom Cruise deepfakes on Tik-Tok are a result of the models and features inside DeepFaceLab.

‍

‍

What is DeepFaceLive?

In August 2021 a new github repository was released dubbed DeepFaceLive. As stated by the name, it's a real-time implantation of DeepFaceLab. This means that if your computer is strong enough, you will be able to compute DeepFaceLab models (like Tom Cruise) as fast as a video stream is moving in, i.e. about 30 frames a second. You can effectively join Zoom calls using this model meaning that you will be able to present yourself with another face: with another identity.

You may have heard this all before. For years now, people have been talking about the threat of deepfakes, how the Infocalypse is going to happen, and that no one will know what's real and what’s fake. So what's different this time? Surely, you might think: “It's scary to think something as realistic as Tom Cruise can ‘technically’ pop up in an online meeting, but that's never going to happen in my meeting”, right?

‍

Are things that bad?

Not yet, but this new implementation of DeepFaceLab has a extra advantage not yet discussed:

“It greatly improves the speed and quality of the creative process”

Creating a high quality deepfake takes time, and will still take time. To make a deepfake look realistic is an art not a science. You have to try certain videos with certain models with certain parameters and see what works best. The longer you're in the game, the better you are at it. Yet, every iteration still takes time to make. Learning what works best takes even longer. I have played around with DeepFaceLab for about a year now, and I have been doing so on my MacBook Air from 2012 - yes, you can actually use DeepFaceLab on a laptop that old. However, the video I wanted to try would have to compile for about 10 hours. So, I would leave my laptop running overnight and the next morning I had one more iteration. Then, I would already have a new idea as to what would work even better and try again. This process took me months. And what was I able to achieve in those 3 months?

‍

Using 3 minutes of the only available video Of Leon Trotsky before his assassination I was able to bring back life to a man that’s been dead for 80 years. (Who was Leon Trotksy?) Imagine now that you could see your results real-time instead of having to wait a day to see it. This will speed up your trial and error process immensely and as a consequence you can create better deepfakes faster.

‍

How creative can you get?

DeepFaceLab, and hence DeepFaceLive, is a program with many functions. Functions to create your own dataset, sort the dataset and even enhance the dataset. You will train models on these datasets and these models can again have different parameters which will have different results. One of the things you can train for is which part of the video is part of the face and which part is not.

You can train this yourself, so you choose to recognize only the actual face, or take into account the entire head and hairstyle. This can also be used to train for the case where objects are obscuring the face.

Using the new Real time implementation you can quickly check the limits of your deepfake model. “Can you pass your hand over your face? What if half my face is obscured? Or something partly see through?”

This training process allows for many creative options. Since you yourself can train what part of the image is to be deepfaked and what not. You can for example train to exclude the mouth. Meaning that the mouth, lips and teeth are still real and are not limited to the software’s capabilities. Now the deepfake can eat, drink or make tongue movements. Even though the software itself is not capable of such movements.

“Now that must be a real person right?” However the true results of these trainings are somewhat random so you need many tries to get it right. And many tries is what's now possible.

‍

What's going to happen?

Right now, we are waiting for the models to be trained. As more models are trained, and more models are shared in online communities, more and more deepfakes will start to appear online. Deepfakes of High Definition quality.
I predict that in the next 6 months:

• The amount of deepfakes online will greatly increase
• The amount of fraud cases around deepfakes will increase
• The question if a zoom call was with the actual person will be asked more often

Before DeepFaceLive, the best possible Deepfake model you could run real time was the First Order Motion Model. Which is cute, but not scary.

What will the next generation deepfakes model look like? Or will we be living in the world of Blade Runner by then?

‍

‍