In recent months, we have seen neural networks create paintings, modify videos, and even answer very interesting questions, but all of that is part of the training they receive. In each case, a human chooses the material and "feeds" it to the network to see what it does with it. Researcher Terence Broad decided to do something very special: train a neural network with the goal of decoding and reconstructing video. And the "source" of his work was none other than Blade Runner.
It all started with a handful of images and the freedom for a neural network to do whatever it wanted with them. Then the results of a chatbot were published, and its answers left us thinking for days. At that point, Google had no choice but to release the "artistic code" of its DeepDream neural network, allowing users to experiment with it, and a few days later someone repeated the original process with a short film, generating one of the most impressive "digital acid trips" we have ever seen. In the end, we all know that neural networks are the product of their training. There is still no "consciousness" or "sense of self," so to speak. However, that does not change the possibility that a neural network can learn new things. In this particular case, reconstructing video.
Researcher Terence Broad chose for his dissertation the title "Autoencoding Video Frames", which on its own doesn't tell us much, but in broader terms, it's about implementing deep learning and neural networks in video, with the goal of reconstructing it. When encoding and compressing a piece of video, well-defined parameters are used (by humans, of course), which affect final quality, sound, format, and other elements. Broad's work aims to eliminate that human element and let the neural network do the encoding on its own, but first he had to do something even more complicated: train the neural network to watch a movie, recognize its frames and their similarities, reject "false" data that wasn't part of the film, transform each frame into 200-bit representations, reconstruct them from those representations, and of course, put them in the correct order.
As the title suggests, the main movie chosen by Broad was Blade Runner, and he also processed another adaptation of Philip K. Dick's work, A Scanner Darkly. Logically, the final result doesn't fool any human, and the quality of the "reconstruction" made by the neural network is below the original film... but it didn't get the order of frames wrong, nor details like speed and rhythm. With each session (he processed both movies a total of six times) the neural network delivered slightly "cleaner" frames, and although that's not enough for our eyes and brains... it was for Warner's bot, which tried to block the videos on YouTube.
