Virtual Production, Motion Capture and the Return of the Film Auteur

 

Everything that was once considered to be integral to film production – sets, make-up, costumes, cameras, etc. – is called “physical production” in the digital effects industry. Many blockbuster films, including Avatar (2009), the highest grossing film in history, are now produced without physical production. These films are the result of what is called “virtual production,” and this development is transforming notions of creativity and creative labour in the film industry.

Virtual production techniques are supposed to reduce costs and make film production more efficient. But at the moment, virtual production means adding more people and technology (and a wider diversity of technologies and specialists who know how to use them). Virtual production expands the scale of film production, and increases the complexity of the industrial apparatus that is commercial filmmaking.

Virtual production is thus an unlikely place for the return of the film auteur — or the notion of the film as a work of individual authorship — but it is here, in the convergence of cinema and 3D imaging, where the auteur is making its 21st century comeback.

Earlier this week, at the Interacting With Immersive Worlds Conference at Brock University, I had the opportunity to hear about state-of-the-art virtual production from digital effects artist, Dejan Momcilovic, who spoke about his work at the New Zealand-based effects firm WETA Digital. Momcilovic spoke about motion capture, a technique used to create the digital effects in Avatar, Lord of the Rings, King Kong, among many other well-known films.

Motion capture is the process of measuring and recording movement in a form that can be processed by computers. Whereas cinematic techniques record the appearance of movement, motion capture extracts information about movement. It’s a method of analyzing movement, which is why, for a long time, motion capture was associated more with rehabilitation medicine than with entertainment.

The video game industry changed all that. After motion capture was used extensively and successfully in action and sports games (Rockstar’s L.A. Noire is probably the most well-known recent example of motion capture in video game production), the film industry began experimenting with it too.

Even if you’ve never heard of motion capture, you’ve probably seen “making of” videos or stills in news articles of actors with dots (or “markers” as they’re called in the industry) placed on their faces. Several cameras (or several dozen, in WETA’s top-of-the-line setup) record the movement of the markers. More detailed and subtle movement require more markers, so typically, the majority of the markers are placed on the actor’s face.

Once motion capture has been used to record a performance in a scene, effects artists use the motion data to make computer-generated bodies move. Bodies that are entirely computer-generated thus have the appearance of live-action filming.

Integrated with other digital effects techniques, motion capture allows directors to film other-worldly characters and environments “as if” they were actually in front of the camera. In virtual production, the characters and the environment of the film can be rendered first and then, using a virtual camera, the director can “film” the scene.

Criticism of motion capture has come mainly from traditional animators who were highly suspicious of this new technique of bridging live-action and animation. As Maureen Furniss writes in her excellent overview these debates, traditional animators saw motion capture as a shortcut around animation production work.

Judging from Momcilovic’s calm and collected manner of speaking about WETA’s digital capture work, today’s motion capture experts no longer see themselves as struggling to meet the artistic standards set by traditional animation.

So it seems like a pretty happy story. Motion capture gives digital effects artists another creative tool. And digital effects artists open up new ways of making film for actors, writers, directors and so on, unconstrained by the technical and aesthetic limitations of traditional cinema. Everyone wins. Or so it seems.

As I watched actor after actor being scanned, transformed into code and then inserted into virtual worlds, I couldn’t help but think of the 1981 film, Looker.

Looker was one of first Hollywood films to seize upon the worries about the replacement of actors by computer simulations. As is common with science-fiction thrillers, Looker revolves around a devious corporation, Digital Matrix, Inc., which, in this case, manipulates television audiences with computer-generated television advertisements. After scanning the actor’s body and producing a 3D model, Digital Matrix programs the virtual actor’s movements so as to hypnotize the viewer (the film doesn’t explain why programmed motion would hypnotize people). Since the real actors might cause trouble for the corporation, Digital Matrix kills the actors/models after digitizing them.

Although Looker is an all-too-familiar tale about the power of media to manipulate the weak and vulnerable audience, it does provide an early commentary on the way 3D capture and digital rendering techniques encourage a view of actors’ bodies and actions as “raw material.”

As with most media impact narratives, which focus on the consequences of technology on society and culture, Looker oversimplifies the relationships between digital media, cinema, and the broader history of techniques for recording movement. Histories of cinema also tend to oversimplify the technological and cultural history of cinema. While the techniques of documenting motion and producing the illusion of motion are at least as old as chronophotography (Leland Stanford and Etienne-Jules Marey’s horse locomotion experiment is probably the most well-known example), cinema was not the inevitable outcome of photography. Animation techniques, for instance, could have led to a very different mode of producing film along with a very different cultural forms of cinema.

But chronophotography comes out of the same obsession with breaking motion down to its basic, mathematic description as motion capture does a century and a half later. Marey’s delight in shifting between media, as demonstrated in his Analysis of the Flight of a Seagull (1887) sculpture, also suggests that the notion of motion as something that can be extracted from a body and shifted into another has been around much longer than contemporary motion capture techniques.

One of the key differences between motion capture and traditional cinematic techniques is that motion capture seeks to render 3D models based on “deep” (rather than “surface-level”) scans of bodies in motion. Digital effects firms like WETA produce 3D models with physiological and anatomical depth, scanning not just the surface of the body but also the body’s “interior,” including muscles, tendons, and bones.

As Momcilovic noted in his talk, some of the actors in WETA’s productions have undergone MRI scanning to make their 3D renderings of movement as realistic as possible. Although MRI may not be a standard component of motion capture yet, it does suggest that the overall trajectory in this area of visual effects is toward “deep” scanning of bodies.

Information from MRI and other kinds of scanning allow the production of virtual models of the actor which perform gestures and movements and respond to similar conditions and events (e.g., impacts) as the actor’s biological body. Motion capture today effectively enables bio-mechanical reconstruction of the actor’s body in digital code.

Looker might have missed the manner in which digital techniques would be integrated into acting and film production more generally, but it does anticipate changes in creative labour facilitated by 3D motion capture and other virtual production techniques.

There are at least two ways of looking at the impact of virtual production on creative labour.

On the one hand, labour that was previously considered “merely” technical is now considered to be creative. Visual effects artists like Momcilovic are increasingly recognized as creative contributors to the film’s aesthetic quality (as well as to its marketability).

On the other hand, motion capture reinforces long-standing hierarchies of labour in the film industry. Momcilovic noted that virtual production gives directors like James Cameron unprecedented control over the entire fictional world of the film. All Cameron needs to do is say, “make that hill bigger,” and the designers make it happen by altering the virtual landscape.

In this way, virtual production, as innovative as it may be, reinforces a very traditional sense of the film as the artistic expression of its director, reminiscent of auteur theory. Even effects artists sometimes regard the film they are working on as the director’s work of art. As Momcilovic put it, “Every decision on Avatar would at least go by [Cameron] and he’d have something to say about it … In a way, it was like watching a genius do what he does.”

Virtual production, as it is currently practiced by filmmakers and interpreted by critics and at least some effects artists, bolsters the sense (however distorted it may be) that films like Avator spring forth from the inspired mind of the director. The actors’ contributions, and those of the effects artists, are thus placed in the background.

Rather than “replacement by computers,” as predicted by Looker, motion capture and the broader array of techniques that enable the virtualization of production are bringing about some very real shifts in creative labour and in the way audiences interpret films produced in this way. Unfortunately, in the short term at least, these shifts appear to be backward, to the 1950s to be precise, when auteur theory, or the notion that the film is the personal expression of its director, became popular among film scholars and critics.

How can the return of the film auteur in the context of an increasingly computerized mode of production be explained?

One possibility is consider the way digital techniques are linked to other artistic practices. It may be, as Lev Manovich suggests in his 2001 book, The Language of New media, that digital cinema is more like painting than photography. As Manovich writes,

“The manual construction of images in digital cinema represents a return to the pro-cinematic practices of the nineteenth century, when images were hand-painted and hand animated. At the turn of the twentieth century, cinema was to delegate these manual techniques to animation and define itself as a recording medium. As cinema enters the digital age, these techniques are again becoming commonplace in the filmmaking process. Consequently, cinema can no longer be clearly distinguished from animation. It is no longer an indexical media technology but, rather, a subgenre of painting.” (p. 295)

What Manovich didn’t anticipate was that techniques like motion capture would be used to bridge animation and live-action. Nevertheless, his suggestion that digital techniques allow filmmakers to manually construct (or program) every detail of the cinematic image, much like a painting, provides a partial explanation for the appeal of the auteur theory in this context. Motion capture is industrially organized and represented as a way for the director to have more control over each “brush stroke” of the cinematic “painting.”

As film production becomes increasingly complex, notions of cinema-as-art may become more simplistic. While the notion of the director as the lone genius toiling away to make the magic happen seems hopelessly out of date, it is precisely that idea which seems to be gaining currency among those who are closest to the action in virtual production.

John Shiga

Advertisements

Blogs: A Self-Motivating Medium?

After having a few years to think about blogging, I’m finally ready to give it a try. But as a researcher and instructor in communication studies, I’m writing a lot anyway. So why would I add this blog to the list of things to write, especially since, according to a Pew study conducted last year, the perceived importance of blogs relative to other media is declining, at least among younger Internet users. So, why blog?

One might find inspiration in the idea that, as a form of social media, blogs are contributing to the transformation of media production, especially news production. For some writers, like clay shirky, bloggers (along with micro-bloggers and Facebook users) are a key part of the vanguard of the social media revolution.

And shirky has many examples of blogging and other forms of digital content creation which had a significant impact on politics and the news media.

For proponents of the view, which could be called “social media optimism,” social media have a democratizing effect on societies, and it’s clear that blogs are helping to open up journalism and other types of media production to “everybody” (i.e., non-professionals).

Social media optimists often point to the Memogate scandal in 2004 as an example of the way blogging can effectively challenge mainstream news media in the U.S. There is no shortage of more recent examples of bloggers who have successfully used blogs to challenge the official media in authoritarian regimes (and there are many such regimes around the world). This includes the Iranian-Canadian blogger, Hossein Derakhshan, as well as Hamza Shargabi in Yemen.

 

In all of theses cases, blogging was an important source of information about political conflict and worked as a catalyst for organized political action.

For those who believe that (1) social media are inherently more democratic than mass media and that (2) a more democratic media system will inevitably lead to a more democratic society, these blog success stories seem to be early indications of a major historical shift in media and politics.

But according to other media theorists, the impact of blogging on politics and the mainstream media tends to be vastly overestimated. Jodi Dean, for example, argues in her 2010 book, Blog Theorythat blogging is a part of a fantasy of “communicative capitalism” in which bloggers believe that “circulating messages” (posting, sharing, etc.) is an effective form of political participation. Political and economic elites are happy to let bloggers think along those lines because blogging diverts dissent safely away into cyberspace.

It’s a rather somber view. But after a decade of blog-topian rhetoric, Dean is correct to point out that it’s time to critically evaluate what blogs actually do in particular political, social and cultural conditions rather than assuming they are agents of democratization.

Setting aside the “big picture” issues of blogging for a moment, there are perhaps some less revolutionary but nonetheless important motivations for blogging.

Most bloggers can probably answer the question, “Why blog?” without hesitation. They do it because they enjoy it.

For cognitive psychologists, enjoyment is an “intrinsic motivation.” An intrinsic motivation comes from the activity itself. If I enjoy blogging because it gives me some degree of pleasure or gratification, then my blogging is intrinsically motivated. The engine for action comes from the activity itself. In this case, blogging is like a locomotive which propels itself and the blogger.

 

Extrinsic motivations come from outside the activity. These motivations may refer to outcomes of the activity, but not to the activity itself. Extrinsic motivations include obligations and responsibilities as well as the desire for money, recognition and other kinds of reward which motivate activity. The engine for action comes from other people or things. In this case, blogging rolls along, but it is being pushed (or pulled) by someone/something else. The blog and the blogger become passenger or cargo cars.

Does this give me a better sense of why I am blogging?

A bit.

Many actions have both intrinsic and extrinsic motivations, so it’s no surprise that I am writing this post because I enjoy it (so far) AND because there’s a possibility (even if it’s remote) that I will eventually get some feedback which will reinforce the value I perceive in blogging and act as an extrinsic motivation to keep me writing and posting.

The intrinsic/extrinsic dualism seems to be cognitive psychology’s way of talking abut the individual and the social levels of action. If that’s true, then there’s plenty of overlap between the two categories. How do you categorize the pleasure of writing? Is that pleasure purely “intrinsic,” or is the pleasure of writing (and the desire to write) shaped by social norms about the value of certain kinds of writing over other cultural practices?

What I’m suggesting is that the intrinsic/extrinsic dualism can gloss over the social origins of desires and gratifications. That’s a pretty significant downside to these categories. But with some tweaking, they can still be useful for understanding why bloggers blog.

In this short inquiry into the raison d’être of my blog, I read an interesting study of bloggers’ motivations. The study builds on the intrinsic/extrinsic categories but goes further by outlining how motivations change over time.

In a 2010 article in New Media & Society, Brian Ekdale, Kang Namkoong, Timothy K.F. Fung and David D. Perlmutter examined the reasons why political bloggers blog. They built on the intrinsic / extrinsic categories of motivation, but the researchers developed 13 motivations that are specific to political blogging (see below). Then they asked 154 of the top political bloggers to rate how much influence of each motivation had their blogging, using a 0 (not at all) to 10 (very much) scale. 66 of the bloggers responded and completed the survey.

Extrinsic motivations

  • To provide an alternative perspective to the mainstream media
  • To help society
  • To inform people about the most relevant information on topics of interest
  • To influence public opinion
  • To help your political party or cause
  • To influence mainstream media
  • To serve as a political watchdog
  • To inform people about the most recent information on topics of interest
  • To critique mainstream media
  • To critique your political opponents

Intrinsic motivations

  • To formulate new ideas
  • To keep track of your thoughts
  • To let off steam

The researchers were interested in how the motivations for blogging change over the course of the “blogspan,” that is, the lifespan of a blog. So they asked the bloggers to give each motivation a rating for the influence it had on their initial blogging and on their current blogging.

The results of the survey are a bit surprising. The researchers expected extrinsic motivations to become stronger over time, and I can understand why. One might assume that the more time, energy and money invested into a blog, the more one will be motivated to maintain reputations, generate income, keep one’s career going, etc. It might also be assumed that such extrinsic motivations would become stronger over time particularly for writers who have made careers out of blogging, like the top political bloggers in the NM&S study.

Interestingly, the results suggest a different picture of blog motivation. All motivations (both “intrinsic” and “extrinsic”) for blogging increase in influence during the “blogspan.”

(There is one exception to this rule. What the researchers called the “Let off steam” motivation decreased over time. The more experience the bloggers acquired, the less “letting off steam” was the key motivation for blogging.)

Another interesting finding in this study is that the most significant increase in ratings between initial and current blogging occurred in the two categories of motivations related to the mainstream media:

“To influence mainstream media”

“To critique mainstream media.”

Over time, the desire to influence and critique the mainstream media became increasingly strong motivations to continue blogging. These political bloggers began with a fairly jaded attitude about the potential of blogging to affect mainstream media. One blogger told the researchers that, early on, it seemed to him/her that most political blogs were “vanity projects” dressed up as challenges to the media and political systems.

Gradually, the more that they worked on their blogs and paid attention to other blogs, the more the bloggers viewed blogging as way of contributing to alternative ways of thinking about issues in the news media and drawing attention to events and issues that are left out of mainstream news media.

The study was done on the “top bloggers.” These are bloggers who have successfully established a reputation and perhaps even a career in the political blogosphere. So, the findings of the study – that blogging increases motivation to blog over time – probably shouldn’t be generalized to all blogs or all bloggers.

But it’s short-term motivation for this post at least.

John Shiga