Deepfakes: The What, How, and Why Behind This Advancing Tech

Today, most people know you can’t always trust a digital photo. Since the dawn of Photoshop, it’s become exponentially easier to manipulate images with often jilting realism. (Take, for example, the popular photo of a shark swimming down a flooded highway that resurfaces after every major hurricane.) By now, most people have figured it out: When a picture on the Internet seems a little too outlandish to be true, it usually is.

Videos, though, have historically been a little harder to alter — at least, until now.

Over the last couple of years, we’ve witnessed a proliferation of something called deepfakes — videos that employ artificial intelligence (AI)-powered tech to superimpose existing imagery onto other videos or images. And while some people and organizations leverage this technology for nefarious purposes, there are plenty of positive applications, too — specifically for brands seeking new ways to connect with buyers.

Here’s what you need to know about how deepfakes work, important ethical implications, and how to use this emerging tech to captivate your audience.

What are deepfakes?

The term deepfake was coined in 2017 as a portmanteau of “deep learning” and “fake.” It refers to the act of splicing together multiple people in an often imperceptible amalgamation. Take, for example, the infamous Jennifer Lawrence and Steve Buscemi mash-up, or the wave of clips with Nicholas Cage’s likeness superimposed onto various other movie characters.

Like Photoshop hoaxes, deepfakes are used for everything from light and fun entertainment purposes to potentially damaging “fake news.” Unlike Photoshopped images, though, deepfakes create a much more realistic, immersive, and impactful experience.

“We still struggle to agree on a definition of deepfakes,” says Venture Capitalist Sunny Dhillon in an article for TechCrunch. “I think of it as any mimicry, manipulation, or synthesis of video or audio that is enabled by machine learning. Face-swapping, body puppetry, copying someone’s voice, and creating entirely new voices or images all fall into this category. Your Photoshop efforts, valiant though they are, don’t.”

How to create a deepfake

To create a deepfake, you’ll need four things:

• A plethora of source images and videos, called "training data"
• A “destination video,” or video onto which the deepfake is placed
• Access to sophisticated AI programs called General Adversarial Networks (GANs)
• Lots (and lots) of time

While a crucial part of the deepfake creation process lies in the software, AI isn’t magic. To ensure the deepfake is as realistic as possible, you have to start with a significant amount of source material to help the software create a model of the subject’s face. Once the program creates the model, it can reconstruct the subject’s face onto a different face in a video. The more iterations of the subject’s face you can provide, and the more time you can dedicate to training the algorithm, the more accurate the results.

The video below shows the difference between 20 minutes and 17 hours of training time.

‍

‍

The complicated ethics of deepfakes

Early deepfakes were fairly easy to spot. For example, you might notice unnatural facial expressions, differences in lighting between the face and body, an unfamiliar voice, or odd-looking, morphed features.

But as deepfake technology rapidly advances, videos are becoming more challenging to detect. As you can imagine, a highly accurate deepfake could be easily weaponized. In the wrong hands, GANs could be used to spread mass misinformation, blackmail people, and commit other sinister deeds.

“While synthetically generated videos are still easily detectable by most humans, that window is closing rapidly. I’d predict we see visually undetectable deepfakes in less than 12 months,” says Jeffrey McGregor, CEO of Truepic, a startup that develops image-verification technology, in an article for the Wall Street Journal. “Society is going to start distrusting every piece of content they see.”

There’s no doubt deepfakes can breed public distrust. That’s why, when brands use this tech for marketing, entertainment, or any other application, it’s essential they’re transparent about their practices.

‍

How to responsibly use deepfakes to benefit your brand

It’s easy to dwell on deepfake technology’s darker implications, but it can be leveraged for exciting, creative projects, too. Here are a few ways you can use this immersive tech to connect with audiences and streamline the production of marketing and advertising efforts:

Easier globalization of advertising

Global brands know creating ads for multiple regions can be a time-consuming and expensive endeavor. To maintain cohesion across a campaign, organizations often shoot the same ad multiple times in different languages — which also usually means hiring several actors. (Simply adding subtitles rarely elicits the same emotional connection as giving audiences an ad in their own language.) Using deepfakes, though, you could shoot once, and then manipulate footage to match dubbed foreign tracks.

For example, nonprofit Malaria No More UK recently launched a campaign featuring David Beckham. In the video, which uses automated facial re-animation and previously recorded audio tracks from other speakers, Beckham appears to speak nine languages. This concept could be easily borrowed by other nonprofits hoping to deliver a global message, or any brand seeking to reach a worldwide audience.

Highly advanced personalization

It’s become increasingly more important for brands to scale personalization efforts to remain competitive. Deepfakes offer what some might consider the pinnacle of marketing personalization: putting consumers directly into your ads.

For example, imagine serving a consumer an ad in which the model has their face, their skin tone, and their approximate body type. That’s precisely what AI-powered face-swapping app Superpersonal does — in only three minutes, according to Forbes. Instead of seeing a stranger modeling clothing or engaging with a product, people will see themselves — which can create a much stronger and more personal connection.

Bring historical figures to the present day

When an artist or musician passes away, we lose much of the context surrounding their work. And while museums may keep their creations available for future generations to enjoy, it can be challenging for audiences to fully connect. Deepfakes allow people to experience and interact with historical figures, fostering a greater understanding and appreciation for their work.

For example, the Dalí Museum in St. Petersburg, Florida developed a life-size recreation of the famous surrealist painter in an exhibit called Dalí Lives. Creators used more than 6,000 frames and 1,000 hours of machine learning to train the algorithm on the painter’s face, according to The Verge. Then, they superimposed his facial expressions onto video footage of an actor with similar body proportions. The team also created audio by having an actor read actual quotes from his letters and interviews, mimicking Dalí’s unique accent. The deepfake version of Dalí also comments on current weather conditions and even takes selfies with museum-goers, which they can save and share.

The opportunities are endless

Like most technology, deepfakes can be leveraged for both positive and negative means. While altered videos can be used to trick or fool audiences into believing misinformation, they can also be used to delight, entertain, educate, and inspire. For brands, the opportunities to connect with audiences through deepfake technology are practically limitless.