In this era of politicians addicted to post-truth and the ad hominem fallacy, a new project from the University of Washington has created algorithms that generate videos with highly advanced lip-syncing from a simple audio clip. A trained eye will detect errors in the process without difficulty, but that's not the point. After all, all that's needed is a simulation good enough—and to repeat it until exhaustion…
For those who have not yet read 1984 by George Orwell, Winston Smith is an editor at the Ministry of Truth tasked with historical revisionism—that is, altering previous records to fit the vision and will of the State. Today, the belief that "nobody survives the archive" remains firm, but if we add the possibility of editing the archive or creating it from scratch that digitization enables, separating truth from lies becomes much harder. That said, a new project from the University of Washington demonstrates the astonishing power of neural networks applied to visual processing and lip-syncing. It also leaves us concerned.
How the Technology Works
Basically, the software generates precise lip and mouth movements from an audio clip, then places them onto a person's face in an existing video. The project leaders say this "realistic audio-to-video conversion" has practical applications such as optimizing video conferencing (instead of transmitting an entire video signal, you receive only the audio and a local model "talks" to you), or in the not-too-distant future, holding a conversation with historical figures and actors via virtual reality. Why did they choose Barack Obama? Simply a matter of available material. The neural network needs to be trained, and there is an enormous amount of public-domain video of the former president.
Limitations and the Future
No, the synchronization is not perfect, and its creators know it, but it's only a matter of time before the effects of the Uncanny Valley are left behind. Currently, the neural network can only be trained with data from one person at a time. According to co-author and professor Steve Seitz, "it's not possible" to take anyone's voice and turn it into a video of President Obama. However, if we go by the comments on the videos (one already has them disabled), people think differently.
