×

A Team of MIT Scientists Taught an AI to Get Emotional Over Movies

Artificial intelligence (AI) may not be ready to write the next blockbuster movie, but a team of AI researchers from the Massachusetts Institute of Technology’s (MIT) Media Lab successfully used machine learning to teach computers about emotional arcs in movies.

The researchers, which collaborated for this project with McKinsey, used machine learning to analyze thousands of videos, including movies, TV shows and short films found on Vimeo. “We developed machine-learning models that rely on deep neural networks to ‘watch’ small slices of video—movies, TV, and short online features—and estimate their positive or negative emotional content by the second,” the team wrote in a blog post Monday morning.

The approach didn’t just pay attention to the general plot line of a movie, but also to more subtle aspects, including the score, and close-ups of a person’s face. Using these clues, the project’s machine learning algorithms were able to identify positive and negative emotions, and map out the extend to which each scene would provoke emotional responses — something the researchers called “visual valence.”

Taken together, this would result in a movie’s or video’s emotional arc — as shown below for the opening scene of Pixar’s “Up:”

Popular on Variety

ai-mit-mckinsey

In other words: Just by analyzing the visual elements of a movie, the algorithms could detect where the tear jerkers would be, and how the overall emotional arc of a story would develop. Researchers also used a control group of human volunteers to map out their response to some of the movies analyzed by computers, and then used that data to refine their models.

By comparing all of the emotional arcs analyzed by their AI, researchers found that most movies and short films follow simple formulas for their emotional arc. They also compared how audiences responded to each of these formulas by looking at comments left on 1500 short films on Vimeo — only to find that even Vimeo viewers like happy endings.

mit ai vimeo responses

That in itself ma not be that surprising — but the fact that artificial intelligence can be used to analyze the emotional arc of a movie, and with that predict an audience response, could be profound for Hollywood, the researchers argued.

“As storytellers increasingly realize the value of AI, and as these tools become more readily available, we could see a major change in the way video stories are created,” they wrote. “In the same way directors can now integrate motion capture in their work, writers and storyboarders might work alongside machines, using their capabilities to sharpen their stories and amplify their emotional pull.”

More Digital

  • YouTube TV

    YouTube TV Keeps Fox Regional Sports Networks on Air as Sinclair Talks Continue

    YouTube TV customers for now still have access to Fox Sports-branded regional sports networks and YES Network, after Google agreed to a temporary contract extension with Sinclair Broadcast Group while the two sides try to hammer out a new distribution deal. YouTube TV’s carriage deal for the Fox RSNs and YES had been set to [...]

  • GDC

    Next Month's GDC Cancelled After Companies Pull Out Over Coronavirus Concerns

    The annual Game Developers Conference (GDC) scheduled for next month in San Francisco has been cancelled after multiple companies said they were sitting it out over fears of the coronavirus outbreak. However, GDC organizers said they planned to hold an event later this summer. GDC was set to run March 16-20 at the Moscone Center [...]

  • Netflix in Nigeria

    Netflix's Head of African Originals Lays Out Streamer's Plans for the Continent (EXCLUSIVE)

    Netflix chief content officer Ted Sarandos and a team of high-level executives have spent the week crisscrossing the African continent, wooing local creators and touting plans to thrust the continent center-stage as the streaming giant’s first African original series, “Queen Sono,” drops worldwide on Friday. Sarandos took the mic at a splashy event in Lagos [...]

  • Editorial use only. No book cover

    Everything Coming to Netflix in March

    Netflix is adding a robust line-up of popular programs to its slate in March. In addition to rom-coms like Matthew Mcconaughey’s “Ghosts of Girlfriend’s Past,” “He’s Just Not That Into You” and “Always a Bridesmaid,” the streamer is also adding a few classics like “Goodfellas” and “The Shawshank Redemption” starring Morgan Freeman and Tim Robbins. [...]

  • Illustration of the video streaming company

    Coronavirus Spread Benefits Netflix, Other 'Stay-at-Home' Companies, Analysts Say

    Amid fears over a global economic slowdown from the widening coronavirus outbreak, companies like Netflix that provide in-home services are best positioned to withstand the storm or even see upside from the crisis, according to industry analysts. Netflix “is an obvious beneficiary if consumers stay home due to coronavirus (COVID-19 virus) concerns, and this has [...]

  • YouTube TV

    YouTube TV Is Dropping Fox Regional Sports Nets, YES Network After Sinclair Standoff

    YouTube TV, Google’s internet pay-TV service, said it will drop the Fox regional sports networks and the YES Network from customer lineups this Saturday, Feb. 29 — citing an impasse in negotiations with Sinclair Broadcast Group, which owns the RSNs. “We purchase rights from Sinclair to distribute content to you,” the YouTube TV account said [...]

More From Our Brands

Access exclusive content