With “motion capture” technology, real actors help define the movement of their onscreen characters — but traditional animators need not file for unemployment just yet. Jay Redd and Troy Saliba, vfx and animation supervisors on “Monster House,” respectively, explain the process on that pic step-by-step:
1. The crew builds a performance-capture “volume” in which the characters will play out scenes. The volume is then outfitted with chicken-wire “sets” invisible to the capture cameras, but physical enough for the cast to interact with.
2. Actors dressed in special unitards act out their scenes while two sets of cameras record their movement: 200 infrared cameras pick up reflective points on the costumes, while six video cameras record the action for reference.
3. Technicians feed the simple optical-point data from all 200 IR cameras into a computer system in Imageworks’ tracking department. The computer identifies each point from at least three cameras’ recordings and begins tracking them through the scene.
4. In the rough integration step, those points are then applied to matching spots on the animators’ rough character skeletons. Since the actors’ cartoon counterparts feature exaggerated proportions (oversized heads and hands), animators make any necessary adjustments so the characters don’t bump into one another by accident. If the director likes part of an actor’s performance from one take, but wants to inject an element from another take, the integration artists can mix and match data.
5. The director and d.p. work with the layout department to select lenses and place virtual cameras within the spaces to view the action.
6. In final integration, integration artists add face data to the characters. “The Polar Express” used a “direct drive” process, which creates a literal reproduction of the actor’s face. “Monster House” pioneered Sony’s proprietary Facial Action Coding System (FACS), in which the mo-cap data drives a library of approved expressions created by the animators.
7. Once final layout is completed, the animators add hands and finesse other details (such as eye movements) that mo-cap didn’t cover, using traditional keyframe techniques. Some scenes must be created entirely from scratch, while others require only minor fine-tuning.