Machine Learning Saves ‘Avengers’ VFX Artists Time

For visual-effects artists, time is always a struggle. When the call comes in to create something spectacular, artists and supervisors have to calculate how much run-
way they have to get from the point of the idea for the vfx to the deadline. On “Avengers: Infinity War,” the vfx crew found that a new innovation — machine learning — made it possible to create the character Thanos in a way that would have simply been impossible without it.

The filmmakers envisioned a version of Thanos — played by Josh Brolin — that would be CG, but also incorporate all the subtle facial expressions and delicate hallmarks of a physical performance that could only been done by an actor. They knew that the facial tracking tech was there but asking vfx artists to manually adjust every inch of the CG version of the face of Thanos once they had all the tracking and scanning information would have been a disaster. There would not be enough time to do it and the artists would likely burn out.

If they wanted to get to the promised land, machine learning — a type of artificial intelligence that allows a computer to learn how to move the facial features of a CG character — would be their vehicle.

“There’s just no time for a vfx artist to move every inch of a character’s face when you’re on a project this size and you need to move forward because it’s not just creating something,” says Dan Deleeuw, vfx supervisor for Marvel Studios. “With machine learning, we can teach the computer to know Josh Brolin’s face by correcting it each time we do something. We can tell it that a smile or a frown doesn’t quite look right and the corner of the actor’s mouth needs to move a little or the lines in his face need to move with his smile and then the computer learns what looks right for this actor’s face and then we can get into more of the overall look of what we’re doing.”

Though most of the big VFX houses are using their own version of machine learning, human judgment, as with human performance isn’t likely to be replaced anytime soon. The algorithm relies on the feedback from the vfx artists to tell it what’s right and even after that all the refinements are judgment calls for the VFX team to make in the same way the crew looks to an actor’s performance to give them the information to create a character that doesn’t look artificial or wrong.

“The performance clearly belongs to Josh Brolin,” says Kelly Port, VFX supervisor at Digital Domain. “We’re relying on him to create something that only he could create and then we’re working with what he gives us to help bring Thanos to life. Machine learning makes it possible for a vfx crew to have the time to create something amazing when a director asks us for that next step up in visual effects.”

More Film

  • Amazon Developing Original Series Based on

    Amazon Studios Buys 'Selah and the Spades,' Will Develop Original Series (EXCLUSIVE)

    Amazon Studios has acquired worldwide rights to “Selah and the Spades,” a gripping look at a prep school drug dealer, Variety has learned. The film marks the feature debut of writer and director Tayarisha Poe and had its world premiere at this year’s Sundance Film Festival where it was a favorite with critics. Amazon has [...]

  • The Ultimate Guide to 2019 Comic-Con

    The Ultimate Guide to 2019 Comic-Con Parties and Activations

    Hollywood is heading down the California coast to San Diego because It’s time for 2019 Comic-Con International. The annual cosplay celebration officially kicks off tomorrow, July 18, with a preview happening tonight. Here, Variety gives you a guide to this year’s parties and activations. Make sure to check back for updates. Wednesday, July 17Amazon Prime [...]

  • The Wound African Cinema Berlin Film

    Finance Forum Brings African WIP Into Focus at Durban FilmMart

    The 10th edition of the Durban FilmMart, which unspools parallel to the 40th Durban Intl. Film Festival, will feature 10 fiction and 10 documentary works-in-progress taking part in its annual Finance Forum. The leading co-production market on the continent, the Forum brings together producers, distributors, sales agents, broadcasters, funding bodies, and other industry players from across the [...]

  • The Lion King

    'The Lion King' Looks to Roar Life Into Domestic Box Office

    Hollywood just can’t wait for “The Lion King” to hit theaters. That’s because Disney’s highly anticipated remake is expected to draw herds of moviegoers at a time when ticket sales are seriously struggling. Box office watchers predict that the studio’s grand return to the Pride Lands could become one of this year’s biggest hits. “The [...]

  • Sarah Wright Takes Charge of Sky

    Sarah Wright Takes Charge of Sky Cinema After Management Shuffle

    Sarah Wright, Sky’s top acquisitions exec, will take on responsibility for Sky Cinema after a management shuffle at the pay-TV giant. Wright fills a role vacated by Ian Lewis, who recently left Sky after 20 years. He was one of several Sky staffers to leave after its takeover by Comcast. As director of Sky Cinema [...]

  • Jodi Long Running for SAG-AFTRA Secretary-Treasurer

    Jodi Long Running for SAG-AFTRA Secretary-Treasurer on Matthew Modine Slate

    Veteran actress and union activist Jodi Long is running for secretary-treasurer of SAG-AFTRA as a member of Matthew Modine’s progressive Membership First slate. She is facing Camryn Manheim, who announced her candidacy on July 9 as part of the Unite for Strength slate for the re-election of union president Gabrielle Carteris. Unite for Strength and United [...]

  • Toni Monty

    Durban FilmMart Head Toni Monty: ‘Africa Has Its Own Story to Tell’

    The Durban Film Festival’s industry program, the Durban FilmMart, celebrates its 10th anniversary as the leading confab for filmmakers from across the continent, with a lively program of panel discussions, seminars and workshops unspooling from July 19-22 in this sunny seaside city. In addition, 10 fiction and 10 documentary works-in-progress have been selected for the DFM’s [...]

More From Our Brands

Access exclusive content