×
You will be redirected back to your article in seconds

A Team of MIT Scientists Taught an AI to Get Emotional Over Movies

Artificial intelligence (AI) may not be ready to write the next blockbuster movie, but a team of AI researchers from the Massachusetts Institute of Technology’s (MIT) Media Lab successfully used machine learning to teach computers about emotional arcs in movies.

The researchers, which collaborated for this project with McKinsey, used machine learning to analyze thousands of videos, including movies, TV shows and short films found on Vimeo. “We developed machine-learning models that rely on deep neural networks to ‘watch’ small slices of video—movies, TV, and short online features—and estimate their positive or negative emotional content by the second,” the team wrote in a blog post Monday morning.

The approach didn’t just pay attention to the general plot line of a movie, but also to more subtle aspects, including the score, and close-ups of a person’s face. Using these clues, the project’s machine learning algorithms were able to identify positive and negative emotions, and map out the extend to which each scene would provoke emotional responses — something the researchers called “visual valence.”

Taken together, this would result in a movie’s or video’s emotional arc — as shown below for the opening scene of Pixar’s “Up:”

ai-mit-mckinsey

In other words: Just by analyzing the visual elements of a movie, the algorithms could detect where the tear jerkers would be, and how the overall emotional arc of a story would develop. Researchers also used a control group of human volunteers to map out their response to some of the movies analyzed by computers, and then used that data to refine their models.

By comparing all of the emotional arcs analyzed by their AI, researchers found that most movies and short films follow simple formulas for their emotional arc. They also compared how audiences responded to each of these formulas by looking at comments left on 1500 short films on Vimeo — only to find that even Vimeo viewers like happy endings.

mit ai vimeo responses

That in itself ma not be that surprising — but the fact that artificial intelligence can be used to analyze the emotional arc of a movie, and with that predict an audience response, could be profound for Hollywood, the researchers argued.

“As storytellers increasingly realize the value of AI, and as these tools become more readily available, we could see a major change in the way video stories are created,” they wrote. “In the same way directors can now integrate motion capture in their work, writers and storyboarders might work alongside machines, using their capabilities to sharpen their stories and amplify their emotional pull.”

More Digital

  • Nancy Pelosi

    Facebook on Defensive Over Fake Pelosi Video

    Facebook faced growing criticism this week over its decision not to remove a video that was doctored to suggest that House Speaker Nancy Pelosi was intoxicated during a recent public event. The video, which has been viewed more than 2.5 million times, had been slowed down notably, giving the impression that Pelosi was slurring her [...]

  • Little-Black-Mirror-Maia-Mitchell

    Netflix Launching 'Little Black Mirror' Video Series Starring Maia Mitchell, Lele Pons, Rudy Mancuso, Juanpa Zurita and More

    To promote next month’s premiere of “Black Mirror” season 5, Netflix is launching a short video series — “Little Black Mirror,” with a cast that includes an ensemble of Latinx social-media stars. The three “mini-stories,” aimed at Spanish-speaking audiences, are inspired by the tech-dystopian universe of Charlie Brooker and Annabel Jones’ anthology series. “Little Black [...]

  • Twitter

    Twitter Permanently Bans Anti-Trump Krassenstein Brothers, Who Deny They Broke Platform's Rules

    Twitter permanently suspended the accounts of Ed and Brian Krassenstein — progressive political activists famous for trolling Donald Trump and his supporters — with the company alleging the brothers used bogus accounts to amplify their reach on the platform. “The Twitter Rules apply to everyone,” a Twitter rep said in a statement. “Operating multiple fake [...]

  • Snapchat

    Snap in Talks to License Music to Let Snapchat Users Embed Songs in Posts

    Snap wants to up Snapchat’s music game. The company has been in negotiations with music companies including the big three — Sony Music Entertainment, Universal Music Group and Warner Music Group — to license song catalogs for the Snapchat app, according to two industry sources familiar with the talks, confirming a Wall Street Journal report. [...]

  • T-mobile - Netflix - John Legere

    T-Mobile Passes Netflix Price Hike Through to Subscribers

    T-Mobile is getting ready to raise prices for subscribers who have taken advantage of its “Netflix On Us” promotion: The mobile carrier will begin charging existing customers who have participated in the promotion an additional $2 per month to account for Netflix’s recent price increase. Consumers will see their bill go up starting on 6/2. [...]

  • Oona King

    Snap Hires Google Exec Oona King as First VP of Diversity and Inclusion

    Snap continues to fill out the ranks of its revamped leadership team: The Snapchat parent tapped Oona King, most recently Google’s director of diversity strategy and a former member of British Parliament with the Labour Party, as its first VP of diversity and inclusion. King, who starts at Snap on June 11, is also the [...]

More From Our Brands

Access exclusive content