When we first started to see AI characters appear on our screens they were usually one of two things.
The main characters’ best friend, or their mortal enemy.
Whilst we’ve not reached sentient AI yet, we do have far more applications for the technology in everyday life than we used to.
Every industry is starting to harness the potential benefits of artificial intelligence.
Growing out of CGI and working alongside it, AI is showing its capabilities as an assistant in the TV and Film industries.
From the genesis of the idea, AI can help the filmmaking process every step of the way.
Writers, actors, coders, and employees at every stage of the creation process are wondering how these new advancements will affect their work.
Some embrace the help that automation can provide, some worry both for their jobs and for the quality of the content this software might produce.
Let’s break down some of the many ways AI is changing the entertainment industry.
This is not an article about On Screen AI Representation in TV and Film, but we have that too.
CGI vs AI
Whilst CGI has been employed in film, tv, music videos, and much more for many decades, it is fundamentally not the same thing as AI.
CGI technology is computer-made but has to rely on a human graphics designer or programmer.
They create rules and parameters for the graphics, use 3D modeling and compositing, and have large control over the final output.
The computer responds to these rules and creates within the parameters.
Motion capture characters like Gollum or Davy Jones are created using CGI, where animators spend countless hours bringing them to life.
CGI technology cannot learn independently, only respond to the prompts it has been given by the VFX artists.
AI technology responds to prompts given but uses its database of knowledge to create content.
Machine learning algorithms mean the software is constantly learning and adapting and can create many different responses from the same prompts
Whilst this sometimes makes the output harder to control it also means you can fine-tune content in different ways.
AI technologies can be used to enhance and edit CGI footage, such as changing the lighting or specific aspects of a model.
The Birth of AI Technology In Cinema
AI technology in entertainment production really started with Lord of The Rings.
The demands for battles like Helm’s Deep were far too great to be achieved with the practical effects of any reasonable budget.
A sub-division of Weta Digital, Massive was created specifically to create a crowd system to man these battles.
They created a series of ‘agents’ to represent the different species and fighting classes within the battle.
Each agent had its own move set and options for fighting and other actions.
This meant when two agents interacted with each other in battle, there was no predicting the outcome of the fight.
It was a truly autonomous battle sequence, with no predictability or repeated sequences.
Massive continues to be a large part of movie-making, creating crowd simulations for films such as Avatar and Avengers: Infinity War.
De-aging Actors
Whilst CGI has been turning actors into younger versions of themselves for a while, AI is about to smoothen out that process.
And not just smooth out their faces like Will Smith’s ‘clone’ in Gemini Man.
Metaphysic AI has signed on as the official Generative AI provider for the new film Here, from Miramax.
The film stars Tom Hanks and Robin Wright and, based on the graphic novel of the same name, revolves heavily around different points in time.
De-aging the actors themselves, then, is important to the narrative.
Using the same technology that created the viral Tom Cruise deep fake, Metaphysic will create a digital mesh of the actor’s faces, using their back catalog of work.
This will then be stitched over their live performances, keeping the nuance of their performance.
This technology is cheaper and has the potential to be more realistic than VFX or CGI.
It should remove that uncanny valley sensation that often occurs when films de-age actors.
However, the existence of this deep fake technology in such a way that it is virtually indistinguishable poses its own problems.
It proves that, with enough of a resemblance, you could stitch anyone’s face onto a body double.
This is a cause of concern for actors who may have already given consent for their likenesses to be used for other projects.
And it is becoming a very serious issue, as seen with the ongoing writers and actors strikes.
Reviving Actors
One step further than just de-aging actors on screen, WorldwideXR is reviving stars entirely.
Everyone remembers the incredible Tupac hologram performance from Coachella 2017.
CGI technology allowed the deceased musician to appear on stage and seemingly interact with Snoop Dogg.
However, whilst this was all cleverly coordinated computer graphics, WorldwideXR won’t settle for that.
James Dean is going to star in the upcoming film Back to Eden.
Yes, the James Dean that has been dead for nearly 70 years.
By feeding an AI all of Dean’s body of work, they can create a digital replica of him that can walk, talk, and interact with his costars.
And learning his lines takes seconds!
Whilst this is very exciting, and of course consent has to have been granted from his estate somewhere, it again poses moral questions about the rights we have to our own image in the wake of this new technology.
CGI and AI combined Characters
As previously mentioned, AI doesn’t have to completely overtake CGI technology.
In fact, they work best in tandem with each other.
The Avengers series created Thanos using a blend of CGI, AI, and VFX techniques.
During filming, Josh Brolin, who brought Thanos to life, was equipped with a motion capture suit with around 150 capture dots on his face.
From this data, a mesh of Thanos’ face was created with CGI, that could accurately track the facial movements.
Then the AI program cleaned up this footage, being fed the updated data repeatedly.
Thus with every iteration, the nuances of Brolin’s performance became clearer, and we were presented on screen with a hyper-realistic and emotional, yet entirely computer-generated, face.
Once the AI tools had performed to the best of their ability, VFX artists took over.
They cleaned up the areas of the work that needed a little more refining, particularly around the eyes where precise dots are harder to place.
This was all done painstakingly frame by frame, but it is undeniable that the quality of the final product would not be the same without the human touch.
It is clear that combining all the technology at our disposal can lead to enhanced and state-of-the-art graphics.
By automating at least some of the time-consuming tasks, productions can ensure all the technical aspects are given the time they deserve by the specialists.
Transforming Landscapes
2022’s Everything Everywhere All at Once is a celebration of visual effects in cinema.
And whilst most sequences were all products of human creativity, the film did benefit from AI help.
Runway’s AI systems were used, particularly in the scene with the rock universe, to clean up green screening and background information.
Evan Halleck, a visual effects artist on the film, praised the efficiency that Runway brought to the movie.
He remarked that the software was “cutting things out better than [his] human eye was.”
Removing the background and additional material in the scenes in minutes, versus days, left Halleck with just the finer details to work on.
Assisting with pre-production
Before filming even begins, AI can be used to streamline the pre-production process.
Technology can analyze character descriptions and call out, and scan actor databases to find faces that match.
Similarly, they can recognize desirable emotions or facial expressions from video footage, speeding up the long audition process.
Location scouting can work in a very similar way.
AI can analyze scene descriptions in the text and look for existing places that match those descriptions.
Tools like these that can enhance the creative capabilities of the crew on production are already being implemented.
Scheduling is another manual task that can be automated very easily.
If all the actors’ availabilities are input into an AI system, it can effortlessly create schedules that work around these remits, saving time.
Video Editing And Post Production
Adobe Firefly is transforming the post-production process with its AI editing tools.
The software can quickly assist in color grading and changing lighting within scenes, effortlessly automating the boring parts of the editing process.
AI algorithms can also assess video footage and pull out key highlights which could be used for trailers.
This is very useful when creating multiple trailers for different demographics, as you can easily ask the technology to search for different types of footage.
It can analyze and find key characters and storylines, finding important points for full-shoot editing.
We are also starting to see AI being used to digitally alter actors’ mouths.
This enables the acting to fit dubbed audio in a less jarring way, which is good for distributing work in foreign languages.
Scriptwriting and Storytelling
Artificial intelligence can also be helpful during the writing process.
Creatives can feed the software plenty of existing scripts in genres they want to emulate or with themes they want to focus on.
With a well-written prompt, the AI can create full scripts in minutes.
Most AI writers use a combination of machine learning and natural language processing.
This enables the output to read more naturally and requires less editing.
Of course, the scripts will still need editing by hand to create a fully cohesive and natural script, but AI can create dialogue, scene directions, and text in many forms.
AI software is also equipped to analyze the data presented and write scripts that will perform well ratings-wise, determined by the algorithm.
This way, writers can ensure their creative vision is intact but also have a better chance of writing the next blockbuster.
In fact, many Hollywood studios are already using different software to determine the success of their films before distributing them.
Warner Brothers use Cinelytic to determine the success of their films within different demographics.
And Sony Pictures have already used Scriptbook to analyze the success of 62 of its films.
Script analysis and editing are also helped by AI as it can easily find sections that could need refining, or posit questions on elements of the script.
Editing text with AI technology is an incredibly accessible tool and there are many free-to-use platforms available.
Read our article Pictory and ChatGPT The Best AI Combination For Content Creation to learn more.
Negative Impact On The TV and Film Industry
Whilst there are undeniable benefits to using AI software to save time and energy on almost every aspect of production, there are still a lot of concerns within the industry.
These largely come in the form of concerns with job security, something we are seeing across a lot of sectors.
When a lot of tasks can be automated, the crew requirement diminishes.
There are also many concerns about creativity decreasing in the entertainment industry, as well as ethical concerns surrounding the data being used to train the AI software.
Many creatives are publicly revoking their consent for their works to be used in the training of machine learning programs.
But this can be hard to control, and the transformative nature of the work is protected by most copyright laws.
The Future of TV and Film Production
As previously mentioned, at this stage all good TV and Film needs the human touch at some point to retain that creativity and spark that makes content exciting.
But as technologies develop and it becomes harder to tell the difference between manmade and machine-made content, who knows if this will always be true?
It is undeniable the positive impact AI can have and is already having, on especially the visual effects side of film production.
The editing tools available on many of these programs open up a whole new level of creativity for many artists.
And it makes editing quality content more accessible to a wider range of people.
As AI is democratizing film creation, Pictory is using its own AI software to democratize the video creation process.
Quick, fun, and easy to use with no video editing experience necessary, we’ll give you all the tools you need to make professional video content in minutes.
Sign up for a FREE trial today and see how Pictory can help you.
