Stability AI Unveils Generative AI Video LLM
Synthetic media startup Stability AI has released its first generative AI foundation model, named Stable Video Diffusion. The open-source foundation model is capable of creating original videos from text prompts and is available as a research preview to those interested.
Stable Video Diffusion
Stable Video Diffusion comes in two models, which can each generate between 14 to 25 video frames at an adjustable frame rate of 3 to 30 frames per second. That’s a short video of under four seconds in most cases. The model can also be fine-tuned for specialized applications like multi-view 3D model spinning. Stability AI plans to build an ecosystem of extended functionalities akin to its hit image generator, Stable Diffusion. The startup claims its external assessments found its models are better than some of the best proprietary models. Stability did make a point of saying that it was trained on videos publicly available for research, which is important considering the multiple lawsuits Stability faces for allegedly using copyrighted images to train its other products.
“This state-of-the-art generative AI video model represents a significant step in our journey toward creating models for everyone of every type,” Stability AI wrote in a blog post. “At the time of release in their foundational form, through external evaluation, we have found these models surpass the leading closed models in user preference studies,” the company said, comparing it to text-to-video platforms Runway and Pika Labs.
Wait-listed users will soon access a web interface showcasing text-to-video use cases spanning advertising, education, entertainment, and other industries. That said, it lacks text input, photorealism, and most camera motion options besides panning. And it’s not a commercial with further safety and quality refinements coming before full release. “We emphasize that this model is not intended for real-world or commercial applications at this stage,” the company explained.
The company has been rapidly expanding its portfolio and improving its tech since raising $101 million in October of last year. The recent list includes SDXL on ClipDrop, the app Stability AI bought last year, with an API in the works, the cartoon-making Stable Animation SDK, and the DeepFloyd IF image generator, which doesn’t use the Stable Diffusion model. Stability has also pushed into non-visual generative AI, releasing its StableLM large language mode capable of composing text and computer code.
Follow @voicebotaiFollow @erichschwartz
Stability AI Releases SDXL 0.9 Photorealistic Text-to-Image Generative AI Model
Stability AI’s New Stable Animation SDK Turns Generative AI into a Cartoon Studio
