motion capture vs. biological flow

Next week the Department of Biological Flow will visit the Balance and Gait Laboratory at Brock University to continue our trajectory of research-creation on the moving human body as it is integrated with broader information networks of signal and noise. To this point the project has primarily been about the walking body in surveillant public and quasi-public spaces, the idea of gait as a uniquely identifying feature, and what we have understood to be an emergence of gait-based surveillance.

Our Kino-Gait Study No.3 was basically an attempt to ask what would happen if the skin-as-volumetric-construct could "see back" in surveillant space — in other words, if gait became our method of seeing rather than the eye (kino-gait). The video above was an experiment to demonstrate that concept.

Following the twin legacies of Muybridge and Marey, the motion capture studio exists today as both clinical instrument for biomechanical analysis and productive apparatus for entertainment spectacle. Its surveillance function is located somewhere in between. Our goal during the study next week is to have an embodied experience of "motion capture" and to creatively play within these tensional intersections.

The objectives for this study include:

  1. Develop a "normal" full model of walking body.
  2. Decode or "scramble" the markers by changing their locations on the body joints and walk again — what would this do?
  3. Have the two Department of Biological Flow members "glued" together front-to-back. The forward person has markers on front and left sides of the body, the rear person has markers on right and back sides of the body. The two bodies match strides, glued together, and then "split" apart halfway and veer off in opposite directions, as if "tearing" the subject in two or radically reconfiguring its relationality.

After these objectives are completed, the next phase of the study will have us wrap a simple videogame flesh texture map around the mocap model and then see what happens to the flesh and its gestural qualities when scrambled (as in scenario #2) or when the body splits in two directions (as in scenario #3).

Marginal Notes on Notes on Gesture

Motion capture. Captured motion.

It is no coincidence that in his essay "Notes on Gesture" Giorgio Agamben only provides the reader one concrete exemplar of what actually constitutes a gesture, and that is gait. Recall that Muybridge and Marey became godfathers of not only the art of cinema but also the science of biomechanics, the relation becoming more apparent over the course of the twentieth century insofar as both serve to capture motion. Or, more specifically, as they both serve to capture gesture: walking and gait have become as important to the processes of consumption as they have to those of production.

It is gait that provided the basis for some of Muybridge and Marey's early cinematic works, but is also the foundational human movement that has driven most innovations in biomechanical measurement during the past century, from stroboscopic photography to force plate analysis to high-speed videography. As Francesco Careri suggests, walking is the "first aesthetic act" of humans in that it assumes a "symbolic form" shaping our very being in the world and our relationships to landscape and architecture. Gait is integral to this symbolic form and thus integral to our built environment both real and virtual. While Careri argues convincingly that the built environment of humans emerges from nomadic walking peoples, eventually it comes to mark the character of the sedentary city in both material and immaterial fashion: the polis and the walking subject enter biunivocal relations of naming the other. Walking is not simply an aesthetic act, then, but a political one as well.

Courtesy of Gabriel Orozco

Gabriel Orozco
digitally-manipulated photographic print

And while Agamben devotes his attention to cinema for the remainder of the essay, perhaps we ought to follow the twin genealogies created by Muybridge and Marey to consider parallel developments in biomechanics as well. Extending an argument from Deleuze's book on cinema, Agamben suggests that "the element of cinema is gesture and not image." If Agamben and Deleuze are correct, then the reason gesture has been obscured in cinematic analysis appears to be simple, as it is literally a matter of appearances. Until recently, cinematic scenes were always shot from a single perspective at a time, from a single camera, and many of these single shots (perhaps from different cameras) were edited together to form a final filmic image — with the audience member, as Benjamin points out, assuming the position of the camera and the gaze of the director.

With this flattening of the perspectival gaze to the two-dimensional surface it appears that the image constitutes the foundational element of cinema, but this is due to the technical limitations of the input device rather than to any truth of the form itself — if we can consider "cinema" to be an assemblage of bodies and technologies that produces the final filmic image. Given such an input, one can never see all sides of a volume from a single point in Euclidean space — and gesture is volumetric.

What technical vision wants is to see the subject from all directions at once — in other words, to become omnidirectional or omnipresent (and here we can explain the "replacement" for an idea of God, in a technocratic sense of becoming-secular). Following Agamben and Deleuze, this is because technical vision wants to represent gesture rather than simple image.

The goal of omnidirectionality had been accomplished to some degree in biomechanics with motion capture technology, an apparatus that features multiple simultaneous camera angles synthesized together to identify the position of markers located on key anthropometric sites of the body. In doing so, it became possible to create volumetric models of gesture for the purposes of measurement, analysis and optimization.

But omnidirectionality has truly taken off with videogames, which took the practical fruits of biomechanic research and made them profitable for the industry of integrated spectacle. Financial gain may now accrue by capturing and expropriating the gestures of athletes and actors to create identity-constructs that are tried on like well-made Armani suits. While playing these games the user reduces one's own gestures to a programmed and nearly-pure electromagnetic impulse almost unrecognizable in comparison to those movements taking place on the screen.

Motion Capture Collage - Courtesy EA Sports

And since it is the integrated spectacle we are describing it is no surprise that innovations in the videogame medium were fedbackforward into cinema, as with the bullet time effects in The Matrix. It is perhaps most impressive, then, that Deleuze recognized cinema's gestural character without ever having seen Trinity levitate to raise holy hell on two units of simulated police.

On Massumi's Logic of Relation: Rules

striated, smooth

We begin our translation of Massumi's soccer ball to sportsbabel's basketball with an opening passage on rules and their retrospective coding of the conditions of possibility.

To the question of what founds a formation like a sport, or what its conditions of existence are, an obvious answer would be "the rules of the game." But in the history of sport, as with virtually every collective formation, the codification of rules follows the emergence of an unformalized proto-sport exhibiting a wide range of variation. The formal rules of the game capture and contain the variation. They frame the game, retrospectively, describing its form as a set of constant relations between standardized terms. A codification is a framing derivative that arrogates to itself the role of foundation. It might be argued that all foundations are of this nature: ex post facto regulatory framings rather than effective foundings. Once they apply themselves, the rules do effectively frame and regulate the play, taking precedence. Their precedence is retrospective, or fictional, but effective. It has all the reality of a formation of power, of which usurpation might be argued to be the model — usurpation of variation (Parables for the Virtual, p.71).

The history of basketball would suggest otherwise. James Naismith invented the game specifically to find an indoor winter activity as an outlet for aggressive masculine tendencies at the YMCA in Springfield, Massachusetts. In other words, the rules did not form later and apply themselves retrospectively, but were rather present from the outset (and the soccer ball became the first basketball!). Indeed, play in general would not be possible were it not for the loosely formed constraints that condition possibility in the space of ludic movement. Otherwise we would find ourselves squarely in the zone of the virtual.

I also think outside of sport to computer videogames. Julian Kücklich discusses the ruledness of a game environment and how in greater degrees it may also constrain the conditions of possibility. In effect, what he is describing is how linguistic codes can striate a space of play. But the spaces of computer videogames — which are rapidly becoming sites of collective formation not unlike sport — are always already ruled spaces by dint of their linguistic form, or put differently, their programmed nature. There is no proto-game from which Grand Theft Auto emerges to contain variation.

(It should be pointed out that ruledness is not quite the same as the striated-smooth dichotomy proposed by Deleuze and Guattari: a space may be striated in material form without firm rules necessarily being attached to the effect of those gridlines. That said, in the foreword to Baudrillard's In the Shadow of Silent Majorities Lotringer, Kraus and El Kholti state that "Félix Guattari may have answered that it is no longer necessary to maintain a distinction between material and semiotic deterritorializations and that there is no more absolute primacy of one system over another." We will not understand them as synonymous, then, but rather as forms that modulate the conditions of possibility in similar fashion.)

But we are in the realm of sport for this discussion. And so we are more fully in the realm of the moving body, in all its sporting forms. Does this imply that Massumi's broader concerns are invalid? No, I don't think so: the point was still to capture and contain variation, but variation of a different order, the roughhousing of young male physical education instructors, rather than an earlier proto-version of basketball. Through the formalization of rules, variation is captured and contained across space and time: it implies that the sport will be played again on another occasion, in the same place or at a different location, and that there will be some consistency between the two events.

If the rules are ex post facto captures that take precedence, what do they take it from?: from the process from which the game actually emerged, and continues to evolve, to the extent that circumstances arise that force modifications of the rules. The foundational rules follow and apply themselves to forces of variation that are endemic to the game and constitute the real conditions of the game's emergence. The rules formally determine the game but do not condition it. (They are its formal cause, not its efficient cause) (Parables for the Virtual, p.72).

Do not forget that this "proto-sport" continues to exist (in basketball we have called it "pickup") alongside the more formal regulated versions of the "sport" (that is, league basketball). In other words, we do not want to get caught up in a linear "stages of progress" model in which pure play becomes proto-sport, which becomes sport, which is thereafter refined by variations in the style of play that are captured or modulated by new rules. The informal pickup versions continue to exist in parallel and in fact play significant roles in creating the variation that flourishes in (and challenges the formality of) "sport" proper.

Let us not become bound up in the search for origin and instead be more cognizant of the process. For in capturing variation — the motor of sporting emergence — we are capturing the possible of the body athletic. Herein lies the micropolitical moment of sport.

Homo Ludens, Deludens, Transludens

Elements of Style: Homo Ludens, Deludens, Transludens

(submitted by sean smith to the 2009 artfulness of play conference at the university of western ontario)

Transludic

Play is a fundamental component of human cultures, one that has infused other structural forms of society such as art, philosophy and law (cf. Huizinga, Homo Ludens). Inherent in play to greater or lesser degrees are sets of rules that channel the conditions of possibility for the ludic subject, whether towards a particular goal or in freer forms of expression. But in the contemporary digital age, these rules of play become more implicitly rules of a system, algorithmically and architecturally so. Julian Kücklich suggests that cheatingdeludology as a strategy — is one way to understand such systems, to learn what constitutes their conditions of possibility and opportunities for agency. Though the theory of deludology is born of the computer game medium, there are still other forms of game in which we play more directly with and against human competitors: is there a violence to these human others in executing strategies of the deludic? Here we turn to Brian Massumi, who suggests style as a third way of approaching the situation. In his analysis, style exists in the liminal space between rules-based technical proficiency and deliberate strategies of cheating, the space where the possible always emerges as a provocation to the game's governing authority. This paper will interrogate play, cheating and style together in trialectic form, as well as in resonance with Paolo Virno's notion of virtuosity, in order to investigate its applicability for a potential micropolitics.

wii/nous/??

In America, Jean Baudrillard suggested that the mirror phase had "given way" to the video phase and the contemporary era of the screen image. But have we not changed again, reverted back to the mirror or at least mutated into a new hybrid of mirror and video?

There is a model for what we are attempting to describe here: the two-way mirror so adored by psychology practice. As children we play in these special rooms while the medical gaze and its recording devices sit quietly behind the silvered glass. Eventually, we learn of the duplicity, not unlike those moments in which we discover the fictions that are Santa Claus, Tooth Fairy, Easter Bunny, and Michael Jackson's whiteness. Whenever in the same situation again, we are subconsciously aware of the mirror and wonder what lurks on the other side.

The regime of the screen intensifies, both in quantity and quality. The sheer number of screens increases beyond even that which Baudrillard could imagine. There is a viral proliferation of the screenal, vectoring beyond home (television) and work (computer) to infect every public space (monitor, jumbotron, electronic billboard, arcade game, etc.), and even the very flows of human movement themselves (laptop, PDA, cellphone).

But the nature of the input interface has changed as well, "democratized," a contagion of interactivity to match the proliferation of the screenal. Now we are all "creators," all able to see ourselves extended into the data networks of the ludic-virtual. In other words, all complicit in the creation of a new mirror — a slightly kaleidoscopic mirror, mind you — but one that captivates us like Narcissus long beyond that mirror phase of childhood.

Like the two-way sort used in psychology, however, this new era of the interactive is at once mirror and screen, at once opportunity for enclosed self-contemplation and open performance. For we all know what lurks behind the silvering of this new mirror and that is the gaze: sometimes manifest as benevolent glance and sometimes as cold, clinical, unblinking stare. Always performance.

Narcissus never suspected that Echo was swimming below the surface of the pool, but we know better.

* * *

There is a certain congruency here with videogames that allow one to toggle between first- and third-person perspectives. Vilém Flusser discusses the difference between line and surface and its implications for perception and thought, but, writing before the videogame revolution, neglects to consider the volumetric. All three appear in the planar form, but since Flusser distinguishes between line and surface, or text and image, it seems important to understand that the videogame is also of an altogether different character, for one actually enters its non-space to control the avatar during one's play.

This is not the same as a three-dimensional setting being reduced to the two-dimensional planar surface through perspectival optics, as with photography, film or television. In that case, one's vision identifies strictly with the point-of-view of the camera and one must imagine the depth of field that is represented on the surface. With the contemporary videogame, on the other hand, there is literally a three-dimensional non-space that has been mathematically modeled "behind" the screen. While the screen thus appears as a site of reduction, this is not due to the nature of cognitive engagement with this non-space, for we are continually monitoring multiple points-of-view as our bodily expertise increases in these ludic environments.

Admittedly, an entire history of static versus scrolling versus spatial gameplay environments needs to be told here, but suffice it to say that the emergence of the ludic subject from the primordial digital ooze of the surface to become volume is the most significant challenge to perception and thought since the invention of photography.

The split of the two-way silvering between mirror and screen is perhaps one way to understand this challenge to perception and thought, manifest in the ludic environment as the ability to instantaneously switch between the subject and object, between the I/je/? and the one/on/??? pronoun positions.

What of the you/tu/??

This was the opening in which wii would like to play // we don't have tickets found its niche. In "sprinting" the videogame 100-metre dash against a local, embodied competitor there was an explicit engagement with the you/tu/? at the nexus of I/je/? and one/on/??? positions. No, people didn't actually run, but yes, they did flail their arms, breathe heavy, and perhaps even shed a bead of sweat. No, people didn't face each other, but yes, through a Japanese interface both Chinese and English engaged amicably, not in translation but rather as a mediation.

And yes, in the process a temporary we/nous/?? was established: a micropolitics of the social body that first began with a politics of the moving and sensing animal body.

(a work-in-process between elaine w. ho and sean smith towards "17 days in beijing: screen of consciousness on the micropolitical," a text for public issue 40)

A Springtime Love Letter to RECL 4P21

Self-organize!

Don't wait to be told what to do, how to share information, how to think!

Thank you for the invitation, but I shall decline. Too much surveillance already, no? ;)

Good work. You're getting it.

S.