Ever since the debut of “Rise of the Planet of the Apes” in 2011, Hollywood’s visual effects community and mainstream moviegoers alike have been enthralled by the breakthrough digital motion-capture work that endowed the movie’s simians with uncanny human movements and facial expressions.

That film — the first of a trilogy that continued with 2014’s “Dawn of the Planet of the Apes” and culminates with “War for the Planet of the Apes,” out on July 14 — also catapulted actor Andy Serkis into the controversy over whether mo-cap performances should be considered awards contenders in the acting categories.

In the latest film, Serkis again reprises his role as Caesar, the genius chimp leading the ape rebellion against homo sapiens as the bitter conflict between humans and apes reaches its peak, pushing to the limit the series’ dark themes of racial oppression, animal rights and the inevitable struggle between species.

“War” also marks a new high-water mark in the digital wizardry that captures and expresses the characters’ complex emotions, and much of the credit for that movie magic goes to the team that’s headed by Joe Letteri, senior visual effects supervisor at Weta Digital in Wellington, New Zealand, who left the U.S. in 2001 to work on “The Lord of the Rings: The Two Towers.” He ended up staying and creating effects for more than 20 additional films, including the third “Rings” movie, “King Kong,” “Avatar” and the two “Hobbit” films.

From the start with the “Apes” films, the question was about making the simians look believable. “When I read the first script, I said, ‘Look, this is great — it has to be done,” Letteri recalls. “We just have to figure out how to make them look realistic.’”

“Rise,” which started production in 2010, built on technical breakthroughs that had been developed for earlier Weta projects. Among them: the technique used for Serkis’ portrayal of Gollum in the “Rings” films (2001 through 2003), although that captured the body only, explains Letteri. “We couldn’t capture facial movement at the time; we figured out how to do that in ‘King Kong’ [2005],” he adds.

Karin Konoval plays Maurice, who has significant dialogue in “War.”
Courtesy of 20th Century Fox

With body movement and facial expressions nailed, the next advances came with 2009’s “Avatar.” Director James Cameron wanted the actors to wear head rigs that would capture facial expressions as they were performed, says Letteri. “We learned how to make that work … and that set the stage for what we did with ‘Rise.’ We came up with a way to bring the mo-cap stage to the live action stage. It was pretty exciting.

Weta Digital visual effects supervisor Dan Lemmon, who, like Letteri, worked on all three “Apes” pics, says the problems were different from the first movie to the most recent. “On the first film it was ‘How can we do it?’” he recalls. “We were riding the bleeding edge of the technology, and things were kind of fragile. We had an idea that hadn’t been battle-tested, so there was a lot of inventing and problem-solving.”

On “War,” much of the wintry principal photography was shot in British Columbia by DP Michael Seresin, while the performance-capture work took place largely in New Zealand. The key to
making it work, says Letteri, was to graft the performance capture onto the live action to create an entirely integrated look.

Another huge undertaking was animating the apes’ faces as they learn to speak so that their expressions resemble those of a human. If that couldn’t be accomplished, the characters wouldn’t be credible. Caesar, a chimpanzee, was relatively easy compared with Maurice (Karin Konoval), an orangutan who only speaks two words in the two previous films but has a significant number of lines in “War.”

“On the first film it was ‘How can we do it?’ We were riding the bleeding edge of the technology.”
Weta Digital visual effects supervisor Dan Lemmon

Says Lemmon: “The apes speak more as the movies progress. We had to figure out how to get Maurice to speak with that big muzzle and small beady eyes, how to push those phonemes and get him to credibly articulate what he’s saying — that was its own challenge.”

Indeed, Weta Digital animation supervisor Daniel Barrett — who also worked on all three films — was taken aback by the script for “War.” “It caught me off guard when I found out that Maurice speaks,” he says. “I realized we’d have to take dialogue into account.”

Working with face puppets, Barrett performed tests for the typical hoop look of an orang’s lips, which stretch a long way forward and flare out at the ends. “In order to make a sound believable when you see Maurice making it, you have to move it back to a more human shape,” Barrett explains.

This human-ape compromise is applied throughout the film — an invisible form of cheating that makes the ape characters seem real to human eyes.

For Letteri, the accomplishments of Weta Digital, culminating with “War,” constitute nothing less than the addition of an entirely new unit to film production, equal in importance to such areas as cinematography, costumes and production design.

“Film has been made the same way for 100 years,” he says. “Essentially we had to bring a new department to the process.”