By the time I met Emily O’Brien, a twenty-eight-year-old actress, I had looked at the computer-generated version of her so many times that seeing her in person was at once anticlimactic and strange. Non-Digital Emily was instantly recognizable to me, from her gently crooked smile to her habit of raising her eyebrows for emphasis. She was, I thought, prettier and softer-looking in person, but it was only after observing her in the flesh that I could see any weaknesses in her duplicate. Many people in the visual-effects world consider Digital Emily, which was created in 2008, to be the first computer-generated face that, even in closeup, is persuasive rather than creepy. A demonstration video of Digital Emily shows her chatting in a leather office chair, and many viewers on YouTube have mistaken her for a real person.
In 1970, a Japanese robotics engineer named Masahiro Mori published a paper called “The Uncanny Valley.” We will accept a synthetic human that looks and moves realistically, Mori wrote, but only up to a point; our satisfaction drops precipitately once the resemblance becomes close enough to nearly—but not quite—fool us. We are charmed by R2-D2, fine with the more human-shaped C-3PO, and comfortable for some way along the spectrum from there. But we are repulsed by a humanoid machine that seems slightly off. Mori’s theory gradually made its way beyond the robotics world and into digital-animation circles. Pixellated humans could fall into the Uncanny Valley just as easily as robots could. Stylized cartoons—the “Incredibles” family, the blocky old man in “Up”—often fill us with empathy, but a pseudo-human like the train conductor voiced by Tom Hanks in “The Polar Express” makes us want to hide under the bed. In recent years, one lavish Hollywood film after another has tried to cross the Uncanny Valley—“Beowulf,” “The Adventures of Tintin”—but their digital characters have struck most viewers as dead-eyed and stiff.
O’Brien and I met at the University of Southern California’s Institute for Creative Technologies, in a conference room with Paul Debevec, the computer scientist who invented the technology that created an elaborate 3-D scan of her. Similar scans have been made of many Hollywood actors, and for more than a decade doubles created with Debevec’s technology have been used in movies, in video games, and in military virtual-reality simulations. His method is continually being refined. On a screen in front of us was Digital Ira, the lab’s latest virtual human. Ira’s olive skin, dappled with pores and light freckles, was rendered in even tighter closeup than Digital Emily’s had been. Digital Ira appeared to have shaved a few hours earlier—stubble was just beginning to emerge—and his forehead had a slight sheen. I noticed a little immobility in the eyeballs, but had I not known of Ira’s provenance I certainly would have been taken in. Moreover, the Ira avatar could be manipulated in real time with a computer mouse, like a video-game character; such an interface might one day allow directors to “conduct” virtual performances.
I watched Debevec make the avatar emote: Ira’s pupils dilated and his eyes pooled with tears; Ira grinned madly and made what somebody had called his “White Guy Dancing” face—a squinty-eyed, tight-lipped approximation of coolness. Debevec, who is forty-two, worked on Ira for two years, and he has scrutinized his creature endlessly, in the service of improving future avatars. Ira’s lashes weren’t quite thick enough, he said, and the caruncle—the bit of pink flesh in the inner corner of an eye—didn’t glisten convincingly. But Debevec’s main goal had been achieved. “You do get to feel the skin as this dynamic, scrunchy surface,” he said. Human skin was devilishly hard to simulate: “It’s this crazy landscape of fuzz and squish.”
O’Brien was being scanned again that day for an updated version of Digital Emily, so we walked down to the room that houses one of the “light stages” that Debevec and his team have designed: a latticework geodesic dome, eight feet in diameter, equipped with more than twelve thousand L.E.D.s and seven Canon high-definition sports-photography cameras. A much larger stage is used for capturing a full body. U.S.C. owns the patents on the technology and charges studios sixteen thousand dollars to capture twenty expressions on an actor’s face.
Emily sat on a chair inside the dome, and for about an hour she made an array of facial expressions: snarling, happy, disgusted, sleepy. She recited some Shakespeare and sang “I’ll Be Seeing You” in a warm alto. The data obtained from these scans will be used to make a fresh geometric map of her facial structure and a texture map of its surfaces. The I.C.T. will then give those maps to a video-game or computer-graphics company—Activision and NVIDIA each did versions of Ira—which will use its own facial-animation software to render a digital puppet of Emily. This is painstaking, computation-intensive work: NVIDIA’s animated version of Digital Ira requires two teraflops of processing power.
At one point, the inspiration for Digital Ira, a forty-three-year-old man named Ari Shapiro, poked his head in. He and O’Brien shook hands, a bit awkwardly. Shapiro is a research scientist at the I.C.T., where he works on synchronizing speech and body movements in virtual humans. Although Shapiro has short brown hair, his C.G. counterpart is bald. It takes tremendous computing power to render hair in real time, and because, as Debevec put it, “there do exist valid humans with no hair,” he had decided to make it easier on the prototype team.
Shapiro told me that he was glad that the lab hadn’t called his avatar Digital Ari. He needed to maintain a little distance: “Paul’s team would walk by my office and say, ‘Oh, you should see what they’re doing to Digital Ari.’ And at some point it became, like, ‘O.K., they’re kind of stealing my soul.’ ” He laughed, a little nervously. “I’m telling you, there’s going to be all kinds of psychological literature once this kind of thing becomes common.”
About thirty years ago, the first computer-generated elements started turning up in Hollywood films; in “Young Sherlock Holmes,” for example, fragments of stained glass inside a church magically unite to form an armored knight. By the nineteen-nineties, Debevec told me, special-effects technicians could make “realistic morphing metal men, dinosaurs, spaceships, and steamboats.” But “there was something about the human face that seemed unattainable.” Not only does the face have complex geometries; evolution has primed us to be attuned to the tiniest divergences from reality.
Even if you get the surface appearance of a digital face right, the movements can do you in. As efforts to build android robots have demonstrated, manufactured blinks often seem owlish, and artificial mouths can appear to snap open too wide or too precisely. (Human lips stick together for a fraction of a second at the corners after we open them to speak.)
Mori’s original paper on the Uncanny Valley was more of a thought experiment than a formal study: he didn’t test his hypothesis about virtual humans on any actual humans. But the argument has been borne out in several formal studies since then. It’s not yet clear why not-quite-humans provoke so much anxiety. Maybe characters that fall into the Uncanny Valley remind us of corpses. Perhaps they flood us with uneasy intimations of soullessness. Maybe they confound the brain’s ability to distinguish what is alive and human from what is neither. Thalia Wheatley, a neuroscientist at Dartmouth who has studied how we recognize animacy in others, told me, “Think of horror movies—zombies, vampires, or even clowns, because they have faces painted on that don’t move. It looks like a person’s face but it doesn’t move like one. A conflict arises in the brain, which is unsettling.”
Angela Tinwell, a technology researcher at the University of Bolton, in England, studies the Uncanny Valley, and she thinks that our discomfort stems from our reliance on nonverbal communication to read the intentions of others. In an imperfect humanoid, she told me, the problem often lies in the upper face: “the eyes, the eyelids, the forehead, which we rely on, subconsciously and consciously, for social communication.” She added, “If we don’t see what we expect to see—if a frightened character in a video game doesn’t have the wide eyes, deep wrinkles in the forehead, arched eyebrows, and vertical wrinkle between the eyes which signal fear to most of us—we’re disoriented.” In certain contexts—horror-genre video games, for example—generating that queasy feeling can be useful. But usually the effect undermines efforts to achieve realism.
Tinwell recently completed a study in which she asked subjects to react to the 2009 version of Digital Emily, and she found that even it “fell short of true, authentic human believability.” People weren’t upset by Digital Emily, exactly, but they sensed a faint strangeness. (She hasn’t tested Ira yet, but she would like to do so.) Indeed, Tinwell and her colleagues Mark Grimshaw and Andrew Williams have proposed the term “Uncanny Wall” to describe an emerging phenomenon: ever greater digital realism is stuck in a race against ever greater viewer expectations. “Our discernment of subtle deviations from the norm will keep pace with technology,” Tinwell predicts. Masahiro Mori, meanwhile, said in a recent interview, “I have no motivation to build a robot that resides on the other side of the Valley.” Robots can be useful, he said, but they “should be different from human beings.”
In 1988, animators at Pixar made a short movie called “Tin Toy,” in which a baby menaces, and then charms, an animate toy. It was a successful test-run for “Toy Story,” except for one thing: the infant was inadvertently unnerving. Since then, Pixar has steered away from highly realistic human characters. (The video is on YouTube, and many viewers have underscored their horror in all caps: “HOLY FUCKING SHIT THAT THING IS FREAKY.”) As Karl MacDorman, a roboticist who studies the Uncanny Valley, told me, “All of the toys in that movie are wonderfully animated. The baby was horrible. Its drool was like Silly Putty. It inspired no empathy.” Pixar’s feature films—with their cute robots, cars, animals, fluffy monsters, and humans rendered with goggle-eyed stylization—have nearly all been artistic and box-office successes. Audiences have similarly embraced the blue Na’vi of “Avatar” and the Gollum character in the Tolkien blockbusters, which felt emotionally human yet didn’t pretend to be human. But movies that have attempted to vault over the Uncanny Valley with photorealistic humans based on motion-capture technology rarely charm audiences; the recent “Mars Needs Moms” was a disastrous flop, losing more than a hundred million dollars. Debevec told me, “Sometimes when you try something too early, it becomes sort of a warning sign to everybody else—you know, ‘Here be dragons. Don’t go off the edge of the earth just yet.’ ”
But, thanks in no small part to Debevec, we are now hovering close to that edge. His scanning technology made it possible for Brad Pitt to age persuasively in “The Curious Case of Benjamin Button,” and has allowed spookily realistic stunt doubles to appear in scenes that are too dangerous for human beings. Debevec thinks that it won’t be long before digital actors begin replacing human actors for entire performances. He said, “Ten years from now, if you want to make a commercial for Progressive insurance with a spunky young woman, you could brew her up to look exactly like you want her to look—voice synthesized, with a facial expression that you could modify and tweak.”
When Debevec was a kid, he was so obsessed with programming that he was startled to learn that some people used computers just to do word processing or to play games. He grew up in Illinois, the only child of a physics professor and a psychiatric social worker, and was a fan of special-effects-filled movies like “Star Wars” and “Back to the Future”; he often stayed up late making stop-motion-animation films with an 8-mm. camera. As an undergraduate at the University of Michigan, Debevec amused himself by making a 3-D rendering of his car, a crummy silver Chevette, and simulating it in flight. He went to graduate school at Berkeley, where he made a short computer-generated film, “The Campanile Movie,” that looked as if it had been shot from a plane flying around the campus’s iconic tower. Debevec’s method was used in “The Matrix” (1999) to fill in backgrounds during the famed “bullet time” sequences, in which the action slows down for one character. Seeing his geekery in a big-budget movie was thrilling, if not that profitable—the lab is an academic institution, after all. He told me, “A lot of what drives me still is just the immense cool factor of being able to make something synthetic that looks real.”
Another Debevec breakthrough involved digital lighting. He realized that if you wanted to insert digital objects into an otherwise naturalistic scene, you had to light them as though they had been there in the first place. As he put it, “A digital car will look a lot more persuasive if the richness of the reflection and the shadow on it are telling us that it’s really there.” In 1998, he developed a technique for extracting light measurements from real-world settings and applying them to computer-generated objects and architecture. His short film “Fiat Lux” showed a digital model of St. Peter’s Basilica illuminated by digital light that mimicked photographs taken in Rome. In 1999, Debevec débuted the film at SIGGRAPH, the annual conference for the special-effects industry. A Times reporter took note of the riotous applause that greeted a sequence in which giant ball bearings appeared to fall from the ceiling of the basilica, reflecting “the whole scene with spherical distortion as they bounced down the length of the cathedral.”
One person in the audience was Mark Sagar, a visual-effects designer who was showing his own short film, “The Jester,” at the conference. The film offered an early example of a realistic computer-generated face, but Sagar thought that the lighting made the skin look too waxy. Over the next several months, he met with Debevec, and, as Sagar recalled, “I talked to him about all the problems with creating realistic skin. Then he basically went back and created the light stage.” Capturing hard data about how skin reflected light, Debevec realized, would lead to more believable virtual humans. To re-create a face digitally, you needed to know all about that face’s reflectance properties: not just how light bounced off the surface of the skin, which is called specular shine, but also what it did after penetrating skin, which is called subsurface scattering. The translucency of skin is part of what make us look alive, not plastic. Debevec’s light stages also helped him capture the darting glint of eyeballs and the bone structure revealed by different angles of light.
The first light stage, which was finished in 1999, was humble, employing a two-hundred-and-fifty-watt spotlight pulled around in a spiral by a rope. Nevertheless, in the course of a minute it provided a data set of a face being lit from every possible direction that light can come from. Debevec has been upgrading the design ever since—so far, there have been ten models of the light stage.
In 2002, Mark Sagar took a job at Sony Pictures Imageworks, where he began working on “Spider-Man 2.” He and a colleague, Scott Stokdyk, decided to use Debevec’s light stage to create a digital stunt double for Alfred Molina, who was playing the villain Doc Ock. Stokdyk told me, “With actual stunt doubles, you have to hide the fact that the resemblance may not be that great—the doubles must turn their heads away from the camera, or whatever.” After Debevec scanned Molina, Sony animators created a digital replica of Molina’s face so lifelike that it could be rendered in closeup as Doc Ock sinks to the bottom of the East River. The one complication was that, before shooting began, Molina was told to shave his impressive sideburns. Stokdyk, who loved that Debevec’s avatars were based entirely on “hard data,” had to fill in the skin beneath the missing sideburns, which “defeated the purity of the process.” He explained, “I had to make all these little artistic decisions: how many days’ growth would he have, what would the stubble look like, the blood flow under the skin there. That might have been the most expensive haircut in history.”
Hollywood judged the “Spider-Man 2” experiment a success, and Debevec and the I.C.T. began scanning many actors. For “Avatar,” Zoe Saldana, Sam Worthington, and others were scanned in the light stage, and the data helped animators create Na’vi aliens whose facial characteristics echoed those of the actors. But the filmmakers also created a digital version of Jake Sully—Worthington’s character—in his human form, allowing them to integrate Jake into scenes set in the fully digital realm of Pandora. Toward the end of the movie, Neytiri, the Na’vi played by Saldana, leans over to kiss the human Jake. Last fall, in a talk at the Smithsonian, Debevec said, “There’s a lot of lighting interaction between the faces. She casts a shadow on him. He’s bouncing some pink light onto her; she’s bouncing some blue light onto him. He has various bioluminescent materials creeping up his neck and his ears that are lighting him from other directions. And if he’s a digital asset, then you can just simulate this all on the computer, tweak it until it looks just right, and get a shot that really looks like that digital person is part of this real-world environment.”
Now that Worthington’s face as it looked in his early thirties has been banked in a computer, this “digital asset” could be used in a film decades from now, when Worthington looks very different. Currently, the production companies own the scans that they commission, but Debevec thinks that actors will one day pay to retain the rights to their digital selves. He said, “Within ten years, it will seem anachronistic to have a different actor playing a younger version of a character.”
Debevec’s light-stage technology has often been used in science-fiction movies, but it’s also been crucial to movies that aren’t special-effects driven. For David Fincher’s “The Social Network,” the Winklevoss twins were both played by Armie Hammer; in scenes featuring the twins, Hammer performed alongside an actor whose face was later replaced with a digital version of his own. Fincher, who began his career working in visual effects, at Industrial Light & Magic, also made “Benjamin Button,” and he seems especially comfortable with the concept of digital actors. In 2007, he directed a popcorn commercial featuring a resurrected Orville Redenbacher. More recently, when Fincher was filming “The Girl with the Dragon Tattoo,” in Sweden, it was deemed too dangerous for Rooney Mara’s stunt double to ride a motorcycle on an icy, dark road without a helmet. The stunt double wore a helmet, but Fincher later had the footage tweaked, superimposing a digital version of Mara’s face. In the shots, the fake Mara, who is chasing the movie’s villain, looks ferociously determined. Steve Preeg, an animation supervisor at Digital Domain, a visual-effects company, worked on both “Dragon Tattoo” and “Benjamin Button.” He told me that digital effects were going to make it harder to judge a performance: “Actors are going to have trouble freezing a frame and saying, ‘That’s me.’ They’re going to be saying, more and more, ‘I don’t remember doing that.’ Well, that’s because they didn’t.”
Debevec is a tall man with dark wavy hair; he periodically shaves off and regrows a goatee, as if to make it impossible for a digital double to supplant him. He works all the time—he likes to call himself a “naked-mole-rat scientist”—and does a lot of his socializing with colleagues. But now he is engaged to a U.S.C. graduate student in animation who is also an accomplished tango dancer. (He’s taking lessons.) Debevec has an unflappable, almost lulling manner, and it can escape a listener that he is crafting a vision of the future to which reasonable people might object. His observations on Hollywood can have a nerdy astringency. Whereas others might talk about Tom Cruise’s ineffable star quality, Debevec puts it this way: “He’s got that iconic nose and really deep-set eyes, so that the distance from nose to eyeball is farther than anything I’ve ever seen. His beak is significant. He just has this really anatomically notable face.”
We’d been standing around with his colleague Oleg Alexander, a computer artist, and Javier von der Pahlen, the head of research and development at Activision, which wants to make its video games more realistic. They both nodded at the mention of Cruise’s beak. Debevec continued, “In a movie star, you want one standard deviation away from regularity. You want that notability to create a space in your brain. It’s like the hook of a song.”
“Humphrey Bogart had the most asymmetrical face,” Alexander said.
“And Bradley Cooper!” Debevec said. “Dude, look at him! One eye is half an inch lower than the other one.”
“Well, you did look pretty closely at his data,” Alexander said. (Cooper was scanned for a film that was later shelved.)
Debevec shook his head. “I tried to watch ‘Silver Linings Playbook,’ but I couldn’t not see it.”
They talked for a bit about the importance of including peach fuzz on digital faces. “ ‘Tintin’ had it,” Alexander noted. “That was the first time I ever saw it on a virtual character.” Peach fuzz would be particularly helpful, they agreed, with children’s faces, since there’s so little texture to work with otherwise.
“Kids are a challenge,” von der Pahlen said. “No wrinkles or enlarged pores.”
Debevec was giving me a tour of the I.C.T., which is housed in a sleek new office park in Playa del Rey, on the grounds of the former Howard Hughes airplane factory. The lab’s current scholarly projects have names like Virtual Human Embodiment and the Computational Emotion Group. Much of this research is accomplished with funding from the Army. The military relies on the I.C.T. to create virtual-reality training simulations, including one that teaches soldiers how to dismantle explosives.
Debevec’s office is a place of peculiar juxtapositions. Not long ago, weedy postdocs, most of them guys, sat at their computers, immersed in labyrinthine computations, while Angelina Jolie—who was being scanned for the upcoming “Maleficent”—had her makeup done in one of the conference rooms. The hallway décor alternates between movie posters and whiteboards covered in impenetrable thickets of math.
In one corner of the office, Debevec had set up a radical demonstration for von der Pahlen and me. The light stage had first helped Debevec insert digital humans into films. Now he wanted to insert them into the real world, in the manner of holograms.
Arranged in a semicircle around a shimmery screen was a constellation of tiny projectors. Collectively, they could project two hundred and sixteen different views of a person, based on footage taken on a light stage equipped with thirty video cameras. At the moment, Debevec and his team were using the technique to create a projected replica of Morgan Spurlock, the host of the CNN series “Inside Man,” who was planning to interview his double for an episode on efforts to achieve immortality.
They were also working on a much richer and stranger holographic project: for the U.S.C. Shoah Foundation, they were creating holographic versions of Holocaust survivors. They had made a prototype based on twenty hours of interviews with an eighty-one-year-old concentration-camp survivor named Pinchas Gutter. In March, they filmed him on a light stage with fifty high-definition cameras. Soon, Debevec said, the digital version of Gutter could be projected into a classroom and questioned by students. The holographic replica could draw on the archive of interviews to answer almost any question relevant to Gutter’s personal history; it could also deflect off-the-wall queries. As Debevec and his colleagues envisaged it, Gutter would be answering questions about the Holocaust long after he was dead.
The technique improves on one that Digital Domain had used to make a virtual version of Tupac Shakur perform at the 2012 Coachella festival. (He was killed in 1996.) But the digital Shakur had appeared on a darkened stage, at a distance. The Shoah Foundation and the I.C.T. want the holographic doubles of Gutter and the other Holocaust survivors to be approachable, conversable, and viewable up close. Debevec’s light-stage technology will help the doubles look natural: wherever they are projected—classroom, synagogue, park—they will appear to be illuminated by the light of that environment.
The project seemed to me to be technically wondrous and philosophically weird. Debevec and his colleagues talked about “future-proofing” Holocaust testimony and other historical evidence. They wanted to make documentary artifacts in forms that subsequent generations might come to expect, and design them so that they could be upgraded to work with new technology. Oil paintings used to offer windows into the past; then it was photography, then film—and now we’d have a 3-D person talking to us. David Traum, the head of the natural-language research group at the I.C.T., has worked closely on the survivors project, and he told me that holographic preservations of the dead are “going to be a pervasive part of the future.” He went on, “It’s a way for relatives and others to do more than just watch a video or listen to audio after people are gone. You could actually ask questions, hear their stories again. You could have a conversation, or something like it, with your departed relatives. It won’t be exactly the person, with new inputs and things to say, but you’ll have that pattern of questions and responses. Or even that little thing of ‘I love you,’ ‘I love you, too.’ ”
Perhaps interactive holograms will make more sense to people of the future than books, or even film, will. Maybe analog artifacts and testimony will seem insufficiently responsive—and therefore less real—to them. Yet creating a virtual Holocaust survivor also risks diluting a fundamental truth, which is that death is permanent, and that those deaths did irreparable damage to humanity. There is also an uncomfortable echo of Disney—of treating history as entertainment. Stephen Smith, the Holocaust scholar who directs the foundation, told me that he’d proceeded “very carefully.” Museum professionals had sometimes expressed doubts to him, he said, but the survivors he consulted had “no barriers to using this technology.” He went on, “These are people who can remember having a single telephone in their houses, and who now use smartphones to send pictures to their grandchildren. They understand that technology isn’t stopping because they’re at the end of their natural lives. They want to move forward with what’s going to be right for the next generation.”
The ethics of digital resurrection becomes particularly fraught in a project involving Holocaust survivors. But I wasn’t all that comfortable with some of the other scenarios that Debevec has imagined becoming popular. Would I want a hologram of my beloved, long-dead parents? They would be eternally available but fundamentally out of reach. Would I want to leave one of myself behind for my children? I don’t think so. It could conceivably say “I love you” to my hypothetical grandchildren, but I’d know nothing about them. Even Debevec wasn’t sure that he’d want something like that. When I asked him if he wished he had holograms of dead relatives, he recalled how taciturn his grandfathers had been, and thought maybe not. “A correct simulation of them would be that they’d lose interest and kind of walk off,” he said.
There’s been talk of creating a holographic Marilyn Monroe or Michael Jackson or Elvis Presley, each of whose estates takes in millions of dollars a year. Steve Preeg, at Digital Domain, told me that he’d been approached to work on a singing Marilyn; he declined, because that project, unlike the Tupac one, did not have the blessing of Monroe’s family. But resurrection is already an established special effect in Hollywood. When Philip Seymour Hoffman died, earlier this year, he had not completed shooting the final “Hunger Games” films. Hoffman was never scanned on a light stage, but there are other ways to digitally fill in the gaps. Debevec explained, “The 3-D shapes of many actors—in every expression they could possibly make—are contained in the frames of the movies themselves. There’s a scene at the end of Bruce Lee’s ‘Enter the Dragon’ where Lee walks through a hall of mirrors, and there are frames where you see his face from several viewpoints simultaneously. By comparing the points in the different views, you can triangulate his facial geometry. There’s a nice wardrobe test reel out there of a young Warren Beatty, showing him from all angles, and keeping pretty consistent facial expressions and constant lighting. In ‘Return of the Jedi,’ there’s a shot of a young Harrison Ford on Endor where he slowly turns his head in a closeup, under consistent illumination. These are the types of shots which will be digitized, frame by frame, and analyzed by computer algorithms, to determine the shape of actors who can no longer be scanned.”
Re-creating an actor’s voice is a problem, for now at least, but impersonators could probably handle that well enough to seem authentic to most audiences. Debevec recalled a story that Robert Legato, an Oscar-winning visual-effects supervisor, told him: “He said that his work on ‘Apollo 13’ wasn’t precisely to re-create the footage of an Apollo rocket launch; it was to re-create people’s memory of the Apollo launch. Creating something consistent with our memories of a deceased actor—his or her voice, facial expressions, and mannerisms—is certainly going to be possible. We don’t have to bring the actor back to life. We just have to bring our memories of them back to life.”
Debevec is determined to usher in an age when it will be cheap and easy for filmmakers, including amateurs, to make movies with a cast of their own choosing—or their own manufacture. In a decade or so, after a few more big Hollywood projects have made light-stage techniques routine, duplication technology should become affordable. Debevec spun out all kinds of possibilities, some of them eerie: “Imagine you’re a filmmaker using virtual tools in the future, and you’re like a Dr. Frankenstein—using Sophia Loren’s cheekbones and Greta Garbo’s lips. You could start with somebody specific and modify. What if George Clooney and Naomi Watts had a kid? You do the analysis of the facial shapes and brew up new faces—plausible new people.”
Debevec told me that his vision was a democratic one. “The people who are the best actors are not always the most beautiful,” he pointed out. “If anybody could have any look and play any age, within reason, it could flatten out what looks like a Third World wealth distribution when it comes to actors. And really indie filmmakers, using virtual sets, would be able to make what only huge Hollywood producers, flying actors to the Riviera or wherever, can do now.”
Debevec’s enthusiasm is generally winning, but when he talked this way I grew skeptical. Why engage in such painstaking digital labor when the movies already abound in beautiful, and infinitely various, human faces? When it came to animation, I’d had more emotional reactions to Hayao Miyazaki’s cartoons than I’d had to many effects-laden blockbusters. And it struck me as literal-minded to be bothered by a lack of resemblance between actors playing younger and older versions of a character in a movie. Still, I could see how Debevec’s vision might feed a desire whose strength I sometimes underestimated: the desire to take the means of media production into one’s own hands. Self-published books are an example of this yearning; so is Instagram. We probably aren’t that far off from a time when people will make their own movies with actors they have created digitally.
One evening, I went with Debevec to a party at the Academy of Motion Picture Arts and Sciences. In 2010, he and several colleagues had shared a technical Academy Award for the lab’s facial-capture systems; since then, he had been to the Academy many times, and at the party he seemed to know half the people in attendance. When we ran into Bill Taylor, a prominent visual-effects supervisor, Debevec said to him, “Tell her the Marlon Brando story.” It went like this: In 2001, the director Frank Oz was working with Marlon Brando on what became Brando’s last movie, “The Score.” During the shoot, Brando frequently mocked Oz’s long association with the Muppets, which Oz had created with Jim Henson. While shooting Brando’s final scene, Oz could not get him to offer the little smile that he wanted. “You wish I was a puppet, so you could stick your hand up my ass and make me do what you want,” Brando reportedly said. In post-production, Oz tweaked Brando’s expression and imposed the smile that he’d wanted all along. Taylor told me that this was an origin story: quite possibly, it marked the first digital manipulation of a real actor. Evidently, a director could turn an actor into a puppet.
This summer, “The Congress,” a partly animated science-fiction movie starring Robin Wright and directed by the Israeli filmmaker Ari Folman, is scheduled to be released. A key scene was filmed on an I.C.T. light stage, but its vision of Debevec’s technology is a dark one. Wright plays an aging actress who makes a Faustian bargain. She sells scans of herself that can be used in perpetuity, insuring her immortality, but in contexts that she would never have agreed to in real life—tacky action movies, giant billboards, sexually graphic scenes. After Wright was scanned, Folman told me, she confessed to feeling as if her self “had been sucked out of her.” Folman added, “But when I look at my kids, growing up with an iPhone in one hand and a joystick in the other, I’m not sure if people will mind in fifteen years if they’re watching movies without flesh-and-blood actors.”
A few months after I met Emily O’Brien at the I.C.T., I met her again for tea in Westwood, and we talked about her career. Before being scanned, her biggest role had been on “The Young and the Restless.” Since the Digital Emily project, she’d been getting more and more work as a video-game actress. In many ways, she liked it: working in a motion-capture suit was comfortable, and her avatars often took her far outside herself, in a way that mere costuming and makeup could never achieve. She thought that many Hollywood actors would no longer tolerate prosthetics or stifling makeup if the same effect could be achieved digitally. Still, she said, “I get a little bit afraid for actors. What does it mean for us? It means they can manipulate you to say or do anything they want.”
At the I.C.T., von der Pahlen and Debevec talked about how virtual humans would increasingly be matched up with artificial intelligence—imagine a much better version of the iPhone’s Siri feature. I asked how such technology might be applied. Debevec could see educational uses: “You could have a tutor who could look at you, develop emotional rapport, sense how you’re doing, but who also happened to know every fact on the Internet.”
“Scary,” von der Pahlen said.
Debevec paused. “I mean, I can speculate about what this technology could be useful for, but that’s not what’s driving us. What’s driving us is what’s cool and new. I want to see it.” ♦
