Adobe plans to upgrade its Firefly platform by adding AI-powered video generation features, slated for release by the end of 2024. The tools will be available through the Premiere Pro beta app as well as a standalone website, bringing advanced video creation options to users.
New AI Capabilities
The upcoming features include Generative Extend, Text to Video, and Image to Video, currently in private beta testing. Generative Extend will enable users to add two seconds to any input video and will be integrated into the Premiere Pro beta app. Meanwhile, Text to Video and Image to Video will produce five-second videos from text prompts or images, available on Firefly's website. Adobe in its announcement blog says that the duration of these videos may be extended in the future.
The Firefly Video model is designed to simplify video editing tasks by using AI to fill in gaps, remove unwanted objects, and generate additional elements to enhance existing footage. Adobe's model is built to be commercially safe, trained solely on content for which Adobe has secured permission.
Meeting Industry Needs
Ely Greenfield, Adobe's Chief Technology Officer of digital media, emphasized the strong demand for AI tools that can improve and speed up current workflows. He noted that Firefly's generative fill feature in Adobe Photoshop, rolled out last year, has become one of the most widely used features in recent years. The new AI video tools are anticipated to have a comparable impact, making video production faster and more efficient.
Although Adobe has not announced the pricing for these new AI video capabilities, it typically allocates “generative credits” to Creative Cloud subscribers, with more expensive plans providing more credits. In a demonstration, Greenfield illustrated how Generative Extend can add two seconds of footage to a video by predicting the upcoming frames and recreating background sounds, excluding human voices and music due to licensing issues.
The Text to Video and Image to Video features will be accessible via firefly.adobe.com, likely with rate limitations. Adobe has put safeguards in place to prevent the generation of inappropriate content and has ensured that its models are not trained on images of public figures.
Adobe's Firefly aims to distinguish itself from competitors like OpenAI's Sora and Runway's Gen-3 Alpha by offering more controllable and reliable AI tools suitable for professional use. The incorporation of these new features is expected to greatly improve video editing workflows, providing users with sophisticated tools to create high-quality video content efficiently.
