They say the best effects are those that go unnoticed. That is usually one of the main differences between a quality project and one that cut too much of its budget, but the market is very interested in launching more robust tools at a lower cost. The Adobe Cloak experiment is an excellent example of that. In simple terms, Adobe Cloak is the “content-aware fill” function applied to moving video. Select the object to erase, and let the software handle it. How about that?
One of the most powerful resources that Adobe Photoshop (and other similar editors) has in its arsenal is content-aware fill. We choose the object to be removed, and the algorithms do their magic to solve the problem. Needless to say, the fill does not always work perfectly, which makes user intervention inevitable, but it serves as a solid starting point in most projects. Now... what about video? If you want to erase something from a short, ideally it should be covered in green or blue, as we can see thanks to a very popular film. When the effects department doesn't do its job, very funny things happen that we can forgive in older movies (say, Braveheart's car, and Gladiator's gas tank), but the rest will soon have the help of Adobe Cloak.
The company says it is a simple experiment and that there are no plans to commercialize the technology in the short term, however, something tells me they will change their minds soon. The demos that Adobe shared with the press were very limited, and the only available video lacks embed (you can see it in the source below), but its operation is quite straightforward: Cloak tracks the object to be removed frame by frame, expanding the scope of processing to the blue perimeter. Another interesting detail is that Cloak works with “objects inside objects”, as is the case of the strap that crosses the man's chest.
Removing objects in video has a fundamental advantage, which is knowing what is behind them. If we add the extraordinary advances in element tracking, the dream of removing your ex from vacation videos is closer than ever (?). Of course, Adobe Cloak has many things to improve, starting with handling shadows, but it is on a very good path.
Source: Engadget
