Meta brings us one step closer to AI-generated movies

Like “Avengers” director Joe Russo, I’m more convinced that fully AI-generated movies and TV shows will be possible within our lifetime.

A host of AI unveilings in the past few months, notably OpenAI’s ultra-realistic-sounding text-to-speech engine, have provided glimpses of this brave new frontier. But the Meta advertisement now puts our future AI-generated content in sharp relief – at least for me.

Meta this morning debuted Emu Video, an evolution of the tech giant’s image creation tool, Emu. Given a caption (eg “A dog running across a grassy knoll”), an image or photo paired with a description, Emu Video can create a four-second animated clip.

Today, video production technology is nothing new. Meta has experimented with this before, as has Google. Meanwhile, startups like Runway are already building businesses on it.

But Emu Video’s clips are easily among the best I’ve seen in terms of their fidelity – to the point where my untrained eye has a hard time distinguishing them from the real thing.

Emu Video

Image Credits: Meta

Well — at least some of them. Emu Video seems to be the most successful at animating simple, mostly static scenes (eg waterfalls and timelapses of city skylines) that stray from photorealism — meaning styles like cubism, anime, “paper cut craft” and steampunk. A clip of the Eiffel Tower at dawn “as a painting,” with the tower visible on the River Seine below it, reminds me of an e-card you might see on American Greetings.

Emu Video

Image Credits: Meta

Despite the best work of Emu Video, however, the diversity created by AI managed to enter – such as strange physics (for example, skateboards moving parallel to the ground) and strange joints (feet toes that curve behind the feet and legs that blend into each other). Objects often appear and disappear from view without much logic to it, too, like the birds above in the aforementioned Eiffel Tower clip.

After spending hours browsing through Emu Video’s creations (or at least the Meta cherry-picked examples), I began to notice something else that was clearly stated: the subjects of the clips are not… ACT many. As far as I know, Emu Video does not appear to have a strong understanding of action verbs, perhaps a limitation of the underlying model architecture.

Emu Video

Image Credits: Meta

For example, a cute anthropomorphized racoon in an Emu Video clip holds a guitar, but not strum the guitar – even though the clip’s caption includes the word “strum.” Or two unicorns “play” chess, but in the sense that they sit curiously in front of a chessboard without moving the pieces.

Emu Video

Image Credits: Meta

So clearly there is work to be done. However, Emu Video is more basic b-roll It wouldn’t be out of place in a movie or TV show today, I’d say – and its ethical implications frankly scare me.

Emu Video

Image Credits: Meta

The deepfakes that are dangerous outside, I fear for the animators and artists whose livelihoods depend on creating these kinds of AI scenes like Emu Video can now be estimated. Meta and its generative AI opponents will likely argue that Emu Video, which Meta CEO Mark Zuckerberg States integrated with Facebook and Instagram, increasing rather than change human artists. But I would say take the optimistic, if not unrealistic, view – especially where money is involved.

Earlier this year, Netflix used AI-generated background images in a three-minute animated short. The company MANILA that tech could help with anime’s supposed labor shortage – but quickly pointed out how low pay and often harsh working conditions are pushing artists out of work.

In the same controversy, the studio behind the credit sequence for Marvel’s “Secret Invasion” admitted to using AI, specifically the text-to-image tool Midjourney, to create much of the work in the sequence. -next. Series director Ali Selim makes the case that the use of AI fits the show’s paranoid themes, but the majority of the artist community and fans strongly disagreed.

Emu Video

Image Credits: Meta

Actors may also be on the chopping block. One of the main sticking points in the recent SAG-AFTRA strike is the use of AI to create digital likenesses. The studios eventually agreed to pay the actors for their AI-created likenesses. But can they rethink as technology advances? I think it is likely.

Adding insult to injury, AI like Emu Video is often trained on images and videos created by artists, photographers and filmmakers — and without informing or paying the creators. In a white paper Along with the release of Emu Video, Meta said only that the model was trained on a data set of 34 million “video text pairs” from 5 to 60 seconds – not where the videos came from, their copyright statuses or whether Meta licensed them.

Emu Video

Image Credits: Meta

There are fits and starts across the industry to allow artists to “opt out” of training or receive compensation for the AI-generated works they contribute. But if Emu Video is any indication, technology – as is often the case – will soon overtake behavior. Maybe it already has.

Leave a comment