Can artificial intelligence replace video editors?

Can artificial intelligence one day completely replace a human video editor? Jim Tierney, president of Digital Anarchy, thinks its use may be more useful than an outright replacement.

NB This is NOT a sponsored post. We published it because it contains genuinely useful information on the increasingly important field of AI in post-production.

At Digital Anarchy, we dove headfirst into the world of AI. So I know the current state of the industry quite well. We were up to our necks last year.

AI is definitely changing the way editors get transcripts and find content in videos. Transcriptive, a product we developed, demonstrates this quite clearly with text. Searching through object recognition is also something that is already happening. But what about the actual video editing?

One of the problems with AI is finishing. Jump in the remaining 10%, if you want. For example, text-to-speech engines have, at best, an accuracy rate of about 95%. That’s about on par with the average human transcriber. For general purpose recordings, human transcriptionists should to worry.

Cumulative errors

But for video editing, there are some differences, and that’s good news. First, and most importantly, errors tend to be cumulative. So if a computer is going to edit a video, at the very least it has to do the transcription and it has to recognize the images. (we’ll ignore other considerations like style, emotion, story, for now). Voice recognition is 95% at best, object recognition is worse. With more layers of AI, these errors will usually multiply (in some cases there might be an improvement, though). While automation may be able to produce a decent rough cut, these errors make it hard to see automation replacing most types of video that professional editors are typically employed for.

Second, if the videos are made for humans, humans often don’t know what they want. Or at least they won’t be able to communicate it in a way that a computer understands and can make changes. If you’ve used Alexa or Echo, you can see how well the AI ​​understands humans. Many situations – especially literal (“Find me the best restaurant”) – work well, many other situations not so much.

Often, as a publisher, the direction you receive from clients is subtle or you have to read between the lines and understand what they want. It’s going to be difficult to get AIs to grasp the way humans typically describe what they want, understand what they actually want, and make those changes.

Third, you address the whole issue of emotion and storytelling, which I don’t think the AI ​​will do well any time soon. The Economist recently published a funny article in which they let an AI write the article. The result is there. Very good at mimicking The Economist’s style, but when it comes to setting up a cohesive narrative…ouch.


Human transcriptionists have reason to worry, but only up to a point

It’s not all good news

There are already phone apps that do basic automatic editing. These are more for consumers who want something quick and dirty. For most of the kinds of things professional editors get paid for, it’s unlikely — from what I’ve seen in apps — to replace humans anytime soon. Although I can see how the technology could be used to create rough cuts and such.

Also, for certain types of videos, weddings or music videos perhaps, you can make a pretty strong case that the AI ​​will soon be able to put together something that looks reasonably professional.

You need neural network training materials to learn how to edit videos. Thanks to YouTube, Vimeo and others, there is an abundance of training material. Do a search for “wedding video” on YouTube. You get 52,000,000 results. 2.3 million people get married in the United States each year. Most of the videos of these weddings are online. I don’t think finding a few hundred thousand professionally made ones will be difficult. It’s probably trivial, actually.

Ditto for music videos. There is enough training material for AIs to learn to do generic editing for many types of videos.

For people who want to pay $49.95 to have their wedding video edited, that option will be there. Probably in a few years. Have your guests shoot the video, upload it, and you’re good to go. You will get what you pay for, but for some people it will be acceptable. Remember that the AI ​​is very good at imitation. So, the end result will be a very simplistic wedding video. However, since many wedding videos are pretty straightforward anyway… for the low end of the market, an AI-edited video may be all “Bridezilla on a Budget” needs. And besides, who looks at these things anyway?

Let the AI ​​do the heavy lifting, not the editing

The short-term losers could be the assistant editors. Many tasks that AI is good for…transcription, sequence finding, etc. are now generally entrusted to assistants. However, this may simply change the types of tasks given to assistant editors. There is a LOT of metadata that needs to be entered and manipulated.

While AI is already showing up in many aspects of video production, it feels like having it do the editing is a long way off. I can see the creation of AI tools that help with editing: creating rough cuts, recommending color corrections or B-roll selection, suggesting timing changes, etc. But it will always require someone to do the editing. It’s still an art form, and the editor plays a crucial role in the actual storytelling of a film. It will be a long time before an AI or machine learning engine can replace that. And this is a good thing.

Back To Top