Notating Affect!?

A Consensual Elucidation

"'Panther Moderns,' he said to the Hosaka, removing the trodes. 'Five minute precis.'"

* * *

And so in Neuromancer William Gibson introduces the concept of an autonomous intelligent agent that can scan video data archives and return with a report on a particular subject, which (most importantly) may be set to a desired length of time.

Of course autonomous intelligent agents have been in various stages of existence and development for a long time. The ability of an agent to successfully accomplish the task above lies primarily in the emerging semantic web, in which the meanings of data and information on the web is explicitly defined such that it may be better understood and connected by human and machine agents. In other words, many of the pieces are in place or are being designed now for the Hosakas of the future.

At least when it comes to simple news reports. What about in sport?

(Carolina Panthers, Pitt Panthers, Black Panthers: database entries in the semantic web.)

We see this type of temporal compression take place all the time: Sportscenter, league-produced weekly news shows, plays of the week, plays of the year, plays of the century. Raptors NBA TV has a feature called Game in an Hour, in which they edit out all the "boring" parts of the original game (usually 2 1/2 hours in length, including commercials) so that it will fit in a one-hour TV slot (including commercials).

But what if I want 14 minutes of footage? Or if I tell my Hosaka I want a 5-minute precis?

(Or if the human editors become cost-prohibitive for the manufacture of sporting spectacle?)

As discussed earlier, professional sport is at its core about the commodified manufacture of affect through uncertainty of outcome. Hence it is not as simple as saying get 5 minutes of video from the game — which 5 minutes is important as well. This is not simply a problem of semantics, as we can, in basketball for example, determine significant events by asking filters to just extract those baskets in which the lead changed hands between teams. Rather, it is a problem of affect: which highlights generated the greatest qualities of felt intensity?

It is not simply a problem of notation, of saying what happened during the game action. There is a quality component (or sign value) for each element of the game that is captured in the statistical flow. How to communicate that to the Hosaka?

Chess offers us a clue. As we have discussed earlier, chess is a game with a highly striated playing space, with pieces that are ranked and coded in such a way that they may only circulate in a constrained, prescribed fashion. Hence we are able to notate a game alphanumerically in such a way that it is easily understood by a computer: in algebraic form, The white king commands his owne knight into the third house before his owne bishop becomes Nf3. These notations constitute the facts of the game, the objective record of what piece moved where and in what sequence. They form the incontrovertible fabric of the game in its archival form.

But there is an element to chess notation that is more subjective in nature. This is not subjective in the sense of a judgment as to if or how a particular statistic should be assessed, but rather subjective in the sense of an aesthetic or emotional consideration layered upon the objective facts of movement.

When archiving a chess game one can notate — or literally punctuate — a particularly brilliant move by appending an exclamation mark to the original notation of the move (ie. Nf3!). In the rare case of an exceptionally brilliant move, one adds a second exclamation mark (Nf3!!). Similarly, in the case of a dubious or questionable move one appends a question mark (Nf3?) or two (Nf3??).

So who determines the aesthetic quality of a particular chess move? For now suffice it to say that this responsibility lies with the "expert" authority who is notating the game.

Returning to our original question about autonomous intelligent agents that can scan the data archives and return a video report of any desired length, how might we accomplish this with a more fluid sport such as basketball?

First we need to be able to merge the video data feed with the statistical data feed such that the former may be tagged for cutting along any desired parameter. Extract all Raptor three-point baskets or all LeBron James shots from further than 15 feet, or just those baskets when the lead changed hands between teams.

This technology currently exists.

So the hypothetical Hosaka can now extract the requisite amount of video footage for a customized report, but if the specifics of the footage have not been clearly articulated at the outset (in other words, 'five minute precis'), then the Hosaka cannot determine which footage should be used. For this we need a third feed of data, one that captures a representation of the affect felt by the spectator of the game in real-time.

Though chess provides us a model of this subjective notation it suffers in that it represents an individual, "expert" point of view. The affect of the sporting experience, on the other hand, is that of the crowd and hence its notation is likely to be sourced in such a fashion, indeed, crowdsourced.

Courtesy of CNN

We saw an example of such crowdsourcing with the "audience reaction meters" used during the 2008 U.S presidential debate coverage on CNN. Each member of the televised debates live studio audience — composed of a blend of Democrat, Republican and undecided voters — held a device called a Perception Analyzer, which allowed them to toggle a dial in response to the talking points and rebuttals made by the two candidates. The television audience then saw a graphic overlay tracking the real-time approval ratings from each of the voter segments present in the studio. In other words, a new data stream had been created, one of embodied visceral and intellectual response to complement other data streams such as video, audio and closed-captioned text.

Now instead of a few dozen people at a political debate, imagine a few thousand fans in attendance at an NBA basketball game, each armed with a perception analyzer, perhaps interfaced in return for perks from some corporate sponsor or as part of an exclusive fan VIP club. Call it the privilege of presence.

[I]n the capitalist regime, surplus labor becomes less and less distinguishable from labor "strictly speaking," and totally impregnates it. … In these new conditions, it remains true that all labor involves surplus labor; but surplus labor no longer requires labor. Surplus labor, capitalist organization in its entirety, operates less and less by the striation of space-time corresponding to the physicosocial concept of work. Rather, it is as though human alienation through surplus labor were replaced by a generalized "machinic enslavement," such that one may furnish surplus value without doing any work (children, the retired, the unemployed, TV viewers, etc.). Not only does the user as such tend to become an employee, but capitalism operates less on a quantity of labor than by a complex qualitative process bringing into play modes of transportation, urban models, the media, entertainment industries, ways of perceiving and feeling — every semiotic system (Deleuze and Guattari, A Thousand Plateaus, p. 491, emphasis added).

Now we have a stream of aggregated fan perception data to complement other data streams such as video, audio, time and alphanumeric game statistics. Now the Hosaka may go to work.


2 responses to Notating Affect!?

- rss feed for this comment thread
  1. sportsBabel » instant karma's gonna get you says:

    [...] the coding of the chess game moves almost strictly to the archival databank the pieces and squares cease to possess an "identity" in any traditional sense, save for [...]

  2. sportsBabel » Wolfgang Schirmacher: In Memoriam di Imagum says:

    [...] the automated chess archive this situation does not prove a problem as every piece in the game is inscribed into biunivocal [...]