×
You will be redirected back to your article in seconds

A Team of MIT Scientists Taught an AI to Get Emotional Over Movies

Artificial intelligence (AI) may not be ready to write the next blockbuster movie, but a team of AI researchers from the Massachusetts Institute of Technology’s (MIT) Media Lab successfully used machine learning to teach computers about emotional arcs in movies.

The researchers, which collaborated for this project with McKinsey, used machine learning to analyze thousands of videos, including movies, TV shows and short films found on Vimeo. “We developed machine-learning models that rely on deep neural networks to ‘watch’ small slices of video—movies, TV, and short online features—and estimate their positive or negative emotional content by the second,” the team wrote in a blog post Monday morning.

The approach didn’t just pay attention to the general plot line of a movie, but also to more subtle aspects, including the score, and close-ups of a person’s face. Using these clues, the project’s machine learning algorithms were able to identify positive and negative emotions, and map out the extend to which each scene would provoke emotional responses — something the researchers called “visual valence.”

Taken together, this would result in a movie’s or video’s emotional arc — as shown below for the opening scene of Pixar’s “Up:”

ai-mit-mckinsey

In other words: Just by analyzing the visual elements of a movie, the algorithms could detect where the tear jerkers would be, and how the overall emotional arc of a story would develop. Researchers also used a control group of human volunteers to map out their response to some of the movies analyzed by computers, and then used that data to refine their models.

By comparing all of the emotional arcs analyzed by their AI, researchers found that most movies and short films follow simple formulas for their emotional arc. They also compared how audiences responded to each of these formulas by looking at comments left on 1500 short films on Vimeo — only to find that even Vimeo viewers like happy endings.

mit ai vimeo responses

That in itself ma not be that surprising — but the fact that artificial intelligence can be used to analyze the emotional arc of a movie, and with that predict an audience response, could be profound for Hollywood, the researchers argued.

“As storytellers increasingly realize the value of AI, and as these tools become more readily available, we could see a major change in the way video stories are created,” they wrote. “In the same way directors can now integrate motion capture in their work, writers and storyboarders might work alongside machines, using their capabilities to sharpen their stories and amplify their emotional pull.”

More Digital

  • Tencent Music presentation at Shanghai

    Shanghai: Tencent Proposes Alliance of Music and Film

    All tech companies strive to be ubiquitous. Within China, behemoth Tencent is a clear leader of the pack. From the company’s “pan entertainment strategy” in 2012 to its 2018 equivalent “neo-creativity strategy,” Tencent keeps unveiling new master plans, each time with more in-house platforms and business approaches involved. On the margins of the Shanghai International [...]

  • Genius Spars With Google Over Lyric

    Genius Accuses Google of Scraping Song Lyrics, Music Data Company Hits Back

    Lyrics annotation service Genius.com has accused Google of scraping its site and stealing its content, the Wall Street Journal reported this weekend. However, a lyrics data provider at the center of the controversy claimed on Monday that those allegations were without merit. The Journal reported that Genius had been complaining to Google about the alleged [...]

  • littlstar ps4 app

    Littlstar Expands Beyond VR With New PS4 App, Signs Up 30,000 Paying Subscribers (EXCLUSIVE)

    A&E-backed immersive content aggregator Littlstar is expanding beyond virtual reality (VR): The New York-based startup launched a dedicated PlayStation4 app this week, giving users of Sony’s game console access to their personal media libraries, as well as both traditional and 360-degree video content from a variety of publishers. Littlstar also announced that it signed up [...]

  • Game of Thrones Season 8 Production

    'Game of Thrones,' Netflix VFX Among Those to Be Featured in SIGGRAPH Production Talks

    VFX pros behind the final season of “Game of Thrones,” the blockbuster film “Avengers: Endgame,” Pixar’s upcoming “Toy Story 4,” last year’s Oscar-winning “Spider-Man: Into the Spider-Verse,” Netflix series, including “Stranger Things,” and more will give SIGGRAPH 2019 attendees a behind-the-scenes look at their work during the conference’s Production Sessions. There will even be a [...]

  • Patrick Drahi

    Altice's Patrick Drahi to Acquire Sotheby's Auction House for $3.7 Billion

    Patrick Drahi, the owner of the telecom group Altice, is on track to purchase Sotheby’s, the high-profile auction house, for $3.7 billion. The acquisition would turn Sotheby’s into a privately owned concern after 31 years as a publicly listed company on the New York Stock Exchange. Drahi said the acquisition had “no capital link with Altice [...]

  • Soundgarden’s Immersive 'Artists Den' Experience Premieres

    Soundgarden’s Immersive 'Artists Den' Experience Premieres This Week (Exclusive Preview)

    There’s no question that the posthumous release is the most difficult to get right: Human beings are unpredictable, and trying to guess what a person’s intentions might have been is an uphill (if not a losing) battle. Yet posthumous releases ranging from Michael Jackson’s “This Is It” to recent Prince and Jimi Hendrix collections prove [...]

More From Our Brands

Access exclusive content