×
You will be redirected back to your article in seconds

A Team of MIT Scientists Taught an AI to Get Emotional Over Movies

Artificial intelligence (AI) may not be ready to write the next blockbuster movie, but a team of AI researchers from the Massachusetts Institute of Technology’s (MIT) Media Lab successfully used machine learning to teach computers about emotional arcs in movies.

The researchers, which collaborated for this project with McKinsey, used machine learning to analyze thousands of videos, including movies, TV shows and short films found on Vimeo. “We developed machine-learning models that rely on deep neural networks to ‘watch’ small slices of video—movies, TV, and short online features—and estimate their positive or negative emotional content by the second,” the team wrote in a blog post Monday morning.

The approach didn’t just pay attention to the general plot line of a movie, but also to more subtle aspects, including the score, and close-ups of a person’s face. Using these clues, the project’s machine learning algorithms were able to identify positive and negative emotions, and map out the extend to which each scene would provoke emotional responses — something the researchers called “visual valence.”

Taken together, this would result in a movie’s or video’s emotional arc — as shown below for the opening scene of Pixar’s “Up:”

ai-mit-mckinsey

In other words: Just by analyzing the visual elements of a movie, the algorithms could detect where the tear jerkers would be, and how the overall emotional arc of a story would develop. Researchers also used a control group of human volunteers to map out their response to some of the movies analyzed by computers, and then used that data to refine their models.

By comparing all of the emotional arcs analyzed by their AI, researchers found that most movies and short films follow simple formulas for their emotional arc. They also compared how audiences responded to each of these formulas by looking at comments left on 1500 short films on Vimeo — only to find that even Vimeo viewers like happy endings.

mit ai vimeo responses

That in itself ma not be that surprising — but the fact that artificial intelligence can be used to analyze the emotional arc of a movie, and with that predict an audience response, could be profound for Hollywood, the researchers argued.

“As storytellers increasingly realize the value of AI, and as these tools become more readily available, we could see a major change in the way video stories are created,” they wrote. “In the same way directors can now integrate motion capture in their work, writers and storyboarders might work alongside machines, using their capabilities to sharpen their stories and amplify their emotional pull.”

Popular on Variety

More Digital

  • Gabrielle Carteris

    SAG-AFTRA Urges Gavin Newsom to Sign Law Punishing 'Deep Fake' Videos

    SAG-AFTRA National Executive Director David White is urging Gov. Gavin Newsom to sign legislation that institutes penalties for nonconsensual digital sexually explicit videos, also known as “deep fakes.” The legislation, Assembly Bill 602, targets companies and individuals who create and distribute the videos in California without the consent of the individual being depicted. “We urge Gov. [...]

  • Facebook Portal logo

    Facebook Is Launching a Portal TV Device: Here’s What We Know

    Facebook is getting ready to launch a TV streaming device in time for the holiday season, Variety has been able to confirm with multiple sources familiar with the company’s plans. The new streaming hardware will be part of the company’s Portal family of devices, featuring a camera, and combining video chatting with TV viewing and [...]

  • MoviePass card

    MoviePass Shuts Down, With Parent Company Citing Failure to Raise Funds

    MoviePass, the long-struggling theater subscription service, finally appears to be down for the count. On Friday, MoviePass notified remaining subscribers that it would be shutting down the service effective Sept. 14, 2019, because “its efforts to recapitalize MoviePass have not been successful to date,” parent company Helios and Matheson Analytics announced. “The company is continuing [...]

  • Disney Plus - Full Lineup

    How to Browse the Disney Plus Catalog the Company Is Testing in the Netherlands

    Disney began testing its Disney Plus streaming service in the Netherlands this week. Consumers in the U.S. will have to wait until November 12 to get access to the service, but European streaming aggregator JustWatch already makes it possible to browse the catalog available during this beta test. JustWatch is one of a number of [...]

  • JoJo Siwa'The Angry Birds Movie 2'

    Nickelodeon Star JoJo Siwa Gets Her Own Dance-Tour Mobile Game (EXCLUSIVE)

    Fans of JoJo Siwa will be able to join the teen influencer’s dance squad on stage — virtually — in a new mobile game. In “JoJo Siwa — Live to Dance,” players will become part of the Nickelodon and YouTube star’s worldwide tour on her professional dance team and experience what it’s like to be [...]

More From Our Brands

Access exclusive content