From Text-Based Editing to Generative AI: Adobe's Latest Video Editing Tool

Adobe is set to introduce AI-powered video synthesis tools in Premiere Pro and After Effects, as reported by Reuters. Premiere Pro will offer AI-based text-based video editing from May onwards. Furthermore, Adobe Firefly, an image synthesis mode, will also be expanded to include video and audio generation tools in the future, offering generative video effects. Adobe Premiere Pro is widely used in the TV and Hollywood film industry as a tool for assembling video content.

With Firefly for Video, Adobe aims to streamline the often laborious process of special effects or editing tasks by incorporating generative artificial intelligence. In a demo reel posted on its blog, Adobe showcased Firefly for Video generating a background soundtrack, audio effects, changing the visual season in a video, producing an enhanced visualization on “liquid”, identifying and illuminating a person’s face, and creating sound records through text commands.

In addition to the aforementioned features, Adobe’s promotional blog also references other AI tools that we can expect to see in the future, such as text-based variety upgrades, text and logo animations, automatic storyboard creation, and “co-pilots” to assist users with advanced editing skills. These text-based video editing tools were first teased by Runway in September, but Adobe’s marketing film should be viewed with caution until the product is officially released.

Adobe has also made a separate announcement this week about the integration of text-based video editing into Adobe Premiere Pro. This feature will enable editors to cut and rearrange video based on automatically identified spoken transcripts extracted from video clips. Adobe’s Sensei platform will be used to analyze and transcribe clips, allowing editors to copy and paste sentences into any order they choose and instantly see them appear on the timeline. Similar text-based video editing functionality was first shown off by Adobe as part of “Project Blink” in October, but the company now claims that these functions are part of its Adobe Sensei platform. However, a comparable service for editing videos and podcasts is also provided by Descript.

Firefly is currently undergoing limited beta testing, and Adobe plans to roll out its Firefly for Video capabilities later this year. Meanwhile, text-based video editing will be available in Adobe Premiere Pro from May.