F/x can replace an appearance, but not a performance. Yet.

In Andrew Niccol’s summer New Line satire “Simone,” movie director Viktor Taransky (Al Pacino) creates the perfect virtual actor — a “vactor” — one who doesn’t complain, make demands or disagree, all under control of a computer’s keyboard. Simone is so real — and such a good actress — she attracts millions of fans worldwide, all unaware that her head is filled with pixels. “Our ability to manufacture fraud exceeds our ability to detect it,” Taransky announces proudly. “I am the death of real.”

Not quite — at least not yet. Visual f/x technology has advanced to the point of being able to replace an actor’s appearance, but not an actor’s performance. It is the marriage of the two that is now coming of age, as evidenced by New Line’s “The Lord of the Rings: The Two Towers” wowing audiences with its robust visuals and fascinating fabricated characters, in particular the wiry Gollum.

The character has the industry talking Oscar attention — for the visual f/x team and for Andy Serkis, the British actor who portrayed him and inspired the animation team who created him.

“Andy’s extraordinary performance drives the entire performance of Gollum,” says Dean Wright, visual effects producer for 3 Foot 6, the film’s production company in New Zealand, where effects for the third “Rings” film are about to get under way. “But the key is, if he delivers a great performance, and the animation isn’t able to reproduce that, then it’s lost.”

To create the tortured, scheming slave to the ring, Serkis was directed in, typically, three performances of each scene. First, donning a special suit to simplify his appearance, Serkis was filmed by director Peter Jackson on set, acting alongside cast members Elijah Wood and Sean Astin, though strictly for rehearsal purposes. “It was basically for the actors to get an idea of where Gollum will be in the scene, how they will react to him, and for timing,” Wright tells Daily Variety.

Once a scene had been rehearsed with Wood and Astin interacting with Gollum, Serkis would step out of the frame, and the scene was filmed again, the actors re-creating their performances of a moment before, but without a physical presence to act against.

Not long after, Serkis was filmed again, this time alone on a motion-capture stage, re-creating his on-set performance for the computer. A third opportunity to present the same lines came on the ADR stage.

The Weta Digital animators then took the motion-capture data of Serkis and began animating Gollum with the actor’s computer-captured shape, to eventually place it into the background plate footage of Wood and Astin interacting with where Serkis used to be.

There were some exceptions, however.

“Sometimes, a rehearsal take with Andy still in the frame represented the best take from Elijah or Sean, so we ended up using that take,” explains Wright. “We would have to then rotoscope Andy out of the plate shot, interpolate the background, one frame at a time, and put the digital Gollum right back where Andy had been in the frame.”

Such work was difficult enough when Serkis was in the same frame as the other actors but separated from them, much less physically interacting with them. “The most difficult one was in their initial interaction, with Andy and Sean and Elijah wrestling on the ground. Weta had a rotoscope department of about 10 people just working on that scene alone,” Wright says.

The animators would typically build the Gollum character, as mentioned, in the usual manner, beginning with a low-resolution digital “puppet,” which allowed for numerous iterations to be produced efficiently. For those scenes where motion capture wasn’t possible, animators relied entirely on key frame animation, the digital counterpart to traditional cel animation (i.e., creating the characters essentially from scratch as opposed to basing them on digital skeletons generated from motion-capture data).

Regardless, the animation of the character was wholeheartedly based on Serkis’ performance. “We did a split screen, with Andy’s performance on one side, and the animated puppet in our background on the other,” says Wright. “So the animators were looking at Andy and watching Andy, his eyes, his face and his expression.

“But the fact that Gollum isn’t exactly Andy meant that they would then have to take another step and translate his performance to Gollum.”

Serkis met regularly with the animators to discuss his vision of the character, describing his intention and motivation in certain scenes. “Ultimately, Peter’s direction would rule the day, but it was extremely helpful for them to talk to him and get his insights,” Wright says.

Interesting, in fact, was Jackson’s direction to the animation team. The helmer typically viewed f/x dailies while in London during the film’s music scoring phase, the images having been uploaded onto the Internet for his review. “Most of Peter’s notes weren’t about, ‘Oh, he has to hit that mark,’” says Wright. “They weren’t technical notes, they were performance notes. ‘Gollum needs to feel the struggle here, etc.; the panic, the desperation.’”

Recalls New Line visual f/x senior VP Lauren Ritchie, “Peter would talk to the animators as though he were talking to an actor. In all of the movies I have worked on I have never seen such detailed performance-driven direction given for a computer-generated character. Although, then again, I do not believe that there has been a movie to date with a CG character demonstrating such depth and complexity.”

That combination — skillful direction and animation, rooted in the performance of a good actor — are critical to the success of a CG-based film. “It’s the difference between a cartoony or unbelievable performance and one that is grounded in reality,” says Wright.

While, he notes, work is being done to try to develop technology that will automate facial capture from an actor’s face, the CG animators still need the actor. “The stronger the performance you get from both of those realms (acting and animation), then the final product will be of the highest caliber.

“If you just have great animators, but no one to lead them on a road, you can end up with something that’s not very interesting to look at at the end of the day. The sophistication level of the audience today is so high that to leave the actor out of the equation and try to deliver a polished performance will fall far short of what people’s expectations are.”

Follow @Variety on Twitter for breaking news, reviews and more