The first time we talked about this possibility was last September: the folks at Runway presented a preview of their new artificial intelligence-backed text-to-video tool, and that gave rise to an initial video-to-video model called Gen-1. Now Runway anticipates the next generational leap, Gen-2. This version promises several processing modes and great ease of use.
You prepare a prompt. You load it into the model. The model makes some mistakes. You adjust the prompt. You try again. Everything you've done up to here is changing and rearranging words. Without Photoshop or advanced tools. Whether it's Midjourney, DALL-E, or Stable Diffusion, AI-generated imagery is becoming simpler and more precise.
But that's just the tip of the iceberg. AI video generation from text is approaching at full speed, and we can already try resources like ModelScope, available on the HuggingFace portal. But one of the major players in this field is Runway, and with the announcement of its new Gen-2 model, users will have the ability to generate three-second clips with eight different processing modes.
Runway Gen-2: From Text to Video, Nonstop
Text to Video is the first of the modes, and as its name suggests, the only thing it asks to start is a text prompt. Next is Text + Image to Video, which also uses a reference image in addition to the prompt. Image to Video removes the prompt's influence to focus all its attention on the image, and Stylization allows transferring a specific style to each frame of an existing video.
Storyboard is ideal for processing mockups (for example, imagining a series of books as buildings in a city), Mask applies modifications to a specific object within a video, Render is like Image to Video but uses 3D animations, and Custom speaks to more advanced training for the model.
If you're interested in trying Gen-2, you'll need to create a Runway account, follow its Twitter profile, and wait. The "coming soon" can be a bit frustrating, but Runway took its time with Gen-1 testing, and there's no reason for it to change its strategy.
Official website: Click here
