The web is no longer the same. We have deepfakes, deepnudes, fake people, fake landscapes… and of course, voices. With enough training, sooner or later an algorithm will be able to clone voices without sacrificing too much precision or speed. In mid-2018 we talked about Lyrebird (now Descript) and its ability to work with small samples. Today it's the turn of Real Time Voice Cloning, an open project by Corentin Jemine that barely needs five seconds of voice for real-time reproduction.
Imagine the consequences. A "leaked" audio of a politician or a celebrity saying horrible things. Contradictory and untenable statements. Lies. Mistakes. And all the product of an algorithm, an artificial intelligence trained to reproduce those messages, following a specific tone. Cloning voices has ceased to be a Terminator privilege. Today, those with adequate knowledge can take free and open source code, available at this moment, and start reproducing voices in real time. If you need proof, well...
... I think this demonstration from machine learning engineer Corentin Jemine is more than enough. His project Real Time Voice Cloning can generate arbitrary sentences in real time using minimal samples as a basis. And when I say 'minimal' I mean five seconds.
His work is based on a study published in January of this year by a team from Google Research, called "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis". On the GitHub page we also find references to other documents oriented to the WaveRNN vocoder, the Tacotron 2 synthesizer, and the GE2E encoder.
The fine print
In essence, there are no binaries to download. Any user who decides to get their feet wet with Real Time Voice Cloning will need some Python knowledge, download pre-trained models, or take care of the training.
Unfortunately, the developer is working full time, and his time for the repository is nil. Anyway, it's a matter of patience. The same happened with the Spleeter voice separator, and within a few weeks there was already an online version...
Real Time Voice Cloning (GitHub): Click here
