Sora: OpenAI's new model creates impressive videos from text

OpenAI has launched a new video-generation model called Sora. The AI company says Sora “can create realistic and imaginative scenes from text instructions.” The text-to-video model allows users to create photorealistic videos up to a minute long — all based on prompts they’ve written.

With this model launch, Open AI joins companies like Runway, Pika and Google (Lumiere) who have deployed impressive text-to-video models.

Sora can “generate complex scenes with multiple characters, specific types of motion, and accurate details of the subject and background,” according to a statement on the Open AI blog. The model also understands what the user has asked for in the prompt and how those things exist in the physical world. 

In an announcement video shared on X, Sora created a video of people moving around in Tokyo during a snowy season. 

While this is a leap in technological advancement from  text-to-image generators like Midjourney, Sora isn’t without its limitations as the company shared.

“The current model has weaknesses. It may struggle with accurately simulating the physics of a complex scene, and may not understand specific instances of cause and effect. For example, a person might take a bite out of a cookie, but afterward, the cookie may not have a bite mark.

The model may also confuse spatial details of a prompt, for example, mixing up left and right, and may struggle with precise descriptions of events that take place over time, like following a specific camera trajectory.”

Currently, Sora is only available a select group of people who are testing the model for potential harms and risks. Open AI is also offering access to some visual artists, designers, and filmmakers to get feedback.