Can the language used in movie reviews hold the tea leaves revealing the winners of the Academy Awards?
That’s the hypothesis of Luminoso Technologies, an artificial-intelligence startup that specializes in natural-language processing, which has already declared the likely best-picture winner of the 2017 Academy Awards before the nominations are even out: Pablo Larraín’s biopic “Jackie,” starring Natalie Portman as Jacqueline Bouvier Kennedy.
Here’s the methodology: The company analyzed user movie reviews for 2013-15 in IMDb, focusing on the 50 most popular movies of each year, to see if there was a correlation behind the concepts that appeared in their language and the eventual Oscar nominees that year. Luminoso’s software found certain specific concepts — such as “cinematography,” “masterpiece,” “stunning,” “visuals” and “experience” — were highly correlated with films that received nominations. Concepts like “narrative” had less correlation with Oscar nods, and a few (like “CGI” and “horror”) had negative correlation.
Luminoso then crunched IMDb reviews for 2016 films to come up with a mathematical score reflecting the likelihood of each one being nominated for best picture in this year’s Academy Awards. All told, it analyzed 84,058 movie reviews on IMDb. Here is its ranking of 2017 Oscars contenders:
- “La La Land”
- “Hell or High Water”
- “Hacksaw Ridge”
- “The Jungle Book”
- “Nocturnal Animals”
- “Manchester by the Sea”
Of course, the business of prognostication is fraught with peril. Look no further than the 2016 U.S. presidential election — when virtually every poll and forecast incorrectly gave Hillary Clinton a lead over Donald Trump leading into Election Day.
It’s also worth noting that the No. 2 and 3 movies on Luminoso’s list are “Moonlight,” which won best motion picture – drama at Sunday’s Golden Globes, and “La La Land,” which took home the trophy for best motion picture – comedy or musical. Both films have scared up more Oscars buzz at this point than “Jackie.”
Eric Pendleton, Luminoso product training manager, says he has 80% confidence in the company’s Oscars predictions. The company applied its algorithm to movies from the 2013-15 period, and only three films that the Luminoso algorithm predicted should have garnered best-picture noms (based on users’ reviews) failed to do so: “Begin Again,” “The Great Gatsby” and “The Danish Girl.”
In its analysis, Luminoso included only reviews written prior to a movie’s Oscar nomination. “Once a movie has been nominated, the language skews,” Pendleton said. For movies released last year, it included only movies with at least 30 reviews as of Dec. 10, 2016, to ensure a reasonable basis for comparison.
The startup’s AI software is based on concepts, not strictly keywords, so it was looking for language that conveyed the meaning of “masterpiece” and other topics. (For what it’s worth, Variety‘s Guy Lodge, in his review of “Jackie,” called the film “extraordinary” and “daring.”) Meanwhile, the concept of “Oscar-worthy” in a review had zero bearing on whether a film was nominated.
“We didn’t know we would find these patterns when we started,” said Pendleton, who conducted the study with solutions engineer Dan Mitus.
Luminoso, founded in 2010 as a spinoff from the MIT Media Lab, produced the Oscars predictions report to show the capabilities of its natural-language processing system for analyzing large volumes of unstructured text. The privately held company’s clients include Hulu, Sprint, Staples, the CDC and Autodesk.
In the case of the Academy Awards, Pendleton admitted, the data doesn’t account for various wildcards — including how much promotion a studio puts behind “for your consideration” campaigns leading up to Oscars voting.
Nominations for the 89th Academy Awards will be announced Tuesday, Jan. 24, with the Oscars to be held Feb. 26.