Earlier this year, OpenAI—makers of ChatGPT—announced Sora, an AI video generator. Some of the demos the company showed off were hyper-realistic, from puppies playing in snow to the view from a subway car traveling through a cityscape. In short, it was both impressive, and terrifying, as I explained in my initial thoughts here.
Since then, Sora news has been largely quiet: The company hasn’t made the platform available for public testing, so we haven’t had too many first-hand experiences to work from. That’s changing this week. Not only is OpenAI making Sora more broadly available, they’re opening up their AI video generator to anyone with a ChatGPT Plus account. Things are about to get wild.
What’s new with Sora
OpenAI made the announcement on Monday, following an early reveal by Marques Brownlee. If you followed the original Sora announcement closely, none of the examples here are going to shock you: Long story short, Sora can be prompted to generate photorealistic, short videos in a number of different situations: OpenAI and Marques Brownlee both have demoed drone shots of cliff sides, animals in nature, and people performing tasks “on camera.” But what’s really new today are a number of features OpenAI has added to the Sora program in general.
There’s “Storyboard,” a sort of video editor that lets you stitch together different video prompts to create longer videos of a single subject. For example, you can have one prompt that asks for a crane standing in water, and another asking to have that crane dip its head into the water. Then, Sora will combine those two prompts into one continuous video. “Recut” also acts as a video editor, only here, you can isolate a specific part of your video, and ask Sora to extend it. “Remix” opens a new prompt field, which lets you request changes to an existing video. (You can choose the “strength” of the remix, too, which affects how much of the video is actually changed from your prompt.) Finally, “Blend” lets you choose to turn the subject of one video into another. Sora’s example is to have a butterfly from one video turn into an orchid from a second clip.
Credit: Jake Peterson
Of all these new features, Storyboard seems to be the most interesting. It appears to be a clever workaround to get the AI to generate a complex scene with multiple actions, as trying to cram all that into a single prompt will likely fail. Remix, too, could be useful in theory for fine-tuning elements of a video, without having to throw out the initial generation. But at the end of the day, our collective interest in the model comes from its basic function: You prompt Sora with a video idea, and it generates it for you. Or, you upload a photo from your library, and Sora animates the inanimate subjects into a moving scene.
Sora in action
When you submit a prompt, your video is added to your “queue” for processing. The amount of time a video takes to generate depends on your settings, including resolution, duration, and the number of variations you generate: I have a standard ChatGPT Plus plan, so I’m limited to a maximum resolution of 720p and a maximum duration of five seconds. ChatGPT Pro users can bump that resolution up to 1080p, create videos as long as 20 seconds, and generate as much as four variations of their video.
Unfortunately for me, it seems everyone and their mother is currently trying to use Sora at this time. My first and only prompt attempt (“tracking shot of a taxi driving through a city center”) hung in processing limbo for the entire time I was writing this piece. In fact, OpenAI has halted account creation for now, as too many people are trying to access the video generator.
My video did finally generate, and it was rough. The video quality of the taxi and the city were quite good (again, very photorealistic), but the taxi’s movements were all over the place. First, it drove in reverse, then transformed into a car that was facing a different direction, before speeding away as another taxi drove into the foreground. (My original taxi also disappeared into thin air, while the new taxi did not have a trunk; rather, two fronts.)
Since it’s taking so long for Sora to generate videos, for now, it’s helpful to look at someone, like Marques Brownee, who has spent some time testing out this tool. In his review of Sora, he finds that the tool still struggles to avoid the typical pitfalls of AI-generated videos: Videos might look photorealistic, but they lose realism in movement. Sora will often mix up which leg should be in front and which should be in back during walk cycles, or “forget” about objects altogether. When Brownlee asked for a video of a tech reviewer covering a smartphone, the reviewer holds two smartphones in their hands, and one simply disappears without reason. Some aspects of a video may run in slow-motion, while others run at typical speed, which looks weird to the eye. These glitches are prevalent in most of the Sora outputs I’ve seen: If you’re looking for them, you’ll see them, and they draw attention to the artificiality of the video.
This is true with “low-quality” videos, too, such as generations of CCTV or security cam footage. Cars drive into one another and disappear, or people move in unrealistic ways. But I will say, the low quality of these videos makes it easier to fake: If Sora can figure out the physics, people are going to have a field day inventing CCTV footage that doesn’t exist.
This CCTV video is 100% AI.
Credit: Marques Brownlee/YouTube
In Brownlee’s experience, the things Sora currently does best are not realistic at all: Motion graphics, for example, generally look good, as do some clips of animations and animated characters. An animation of a sketching of the Empire State Building looks like something out of a Netflix series intro, for example. And when Brownlee uploaded an image of animated leaf characters that DALL-E generated, Sora animated the image in a somewhat believable way. It’s a bit easier to ignore the imperfections when the video isn’t purporting to be real at all.
Sora also appears to be decent at generating drone and tracking shots: A drone shot of Mount Fuji, or the Golden Gate Bridge, appears smooth and photorealistic. If you look close, you might notice glitches and imperfections, like waves that aren’t behaving quite like they should, but you could probably slip these shots into shows and movies without many (or most) people noticing.
Where do we go from here?
Sora scared me back in February when it was announced. In the ten months since then, I’m still scared, but not because the videos are that much better. In fact, just based on what I see today, the quality seems about the same—albeit with some new AI features you can use to tweak those videos. The realism is still there when it’s there, as are the flaws, of which there are many.
What scares me is accessibility: Once OpenAI works through the demand, Sora will be available to anyone with a ChatGPT Plus subscription. For $20, you have access to a tool that can generate up to 50 five-second videos per month. Five seconds isn’t very long, of course, so without some cleverness, these videos likely aren’t going to be the ones doing the most damage.
That’s where ChatGPT Pro comes in. This plan is much more expensive ($200 per month), but for that $200, you can create up to 500 videos, each of which can be up to 1080p and up to 20 seconds in length. OpenAI says you can also download these videos without a watermark, which will make detection that much more difficult.
Sure, most of us won’t subscribe to Pro for this, but $200 isn’t much of a deterrent for bad actors who want to spread misinformation. Imagine the next major polarizing crisis, fueled by a flood of videos that “prove” what happened one way or another, when in fact those videos aren’t real at all. OpenAI does have some safety features baked in here, like blocking copyrighted materials or notable figures from being incorporated in a video, but we’ll see how well these roadblocks work in practice.
How to try Sora
At this time, account creation is not available for Sora, but that may change imminently. If you’re interested in trying Sora out for yourself, head to sora.com. From here, click log in, then authenticate yourself with your ChatGPT account. Remember, you need either a ChatGPT Plus account ($20 per month) or a ChatGPT Pro account ($200 per month) to use Sora.
