Audiovisual Coherence and Physical Presence

I am there, therefore I am [?]

by Louise Harris

The following is an attempt at both documentation and discussion of my personal audiovisual practice to date; in particular my attempts over the past four years to bring a complex, largely algorithmic, fixed-media method into a live, improvisatory performance context.

Historically, my work has been primarily concerned with making fixed-media compositions that attempt to avoid any sense of media hierarchy — that is, that neither the visual nor the auditory component be perceived as being of primary importance in the work. Indeed, I have always used the term “audiovisual” (as one word, not a two-word hyphenate) to describe my pieces, though my work has been described as “visual music” by more than one reviewer. I do very much identify with the visual music tradition and the works of, for example, Oskar Fischinger, Norman McLaren and, more recently, Mick Grierson and Joseph Hyde, however, I personally do not find the term “visual music” particularly useful as it, for me, could be seen to imply that music — or moreover the “musicality” of the work and how it relates to the visual component — is of central importance. 1[1. For an extensive discussion of the various historical and contemporary practices in the various fields of visual music, videomusic and audiovisual work see eContact! 15.4 — Videomusic: Overview of an Emerging Art Form / Vidéomusique : aperçu d’une pratique artistique émergente.] For my own purposes, this isn’t of primary concern, for a number of reasons, but perhaps most fundamentally because I consider myself to be an audiovisual composer, composing with auditory and visual media simultaneously to create works in which the sound and image function as part of a unified, cohesive system — what John Whitney described as a “complementarity” (1994, 2) and Bill Alves has subsequently referred to as the “digital harmony of sound and light” (2005, 1). The fixed nature of these works, existing as delimited audiovisual artifacts to be installed in a gallery or played back during a concert, were historically an essential aspect of this attempt at cohesion — an attempt to limit additional demands on the audioviewer’s sensory experience. Some short examples of some of my fixed audiovisual compositions should serve to better illuminate some of these ideas (Video 1).

However, this aspect of my practice has only been of central concern in recent years — my musical background extends back considerably before this, indeed from the age of ten, as a performer. Be it as a classically trained flautist, performing in musical theatre or singing in jazz clubs to put myself through university, I have always “performed” — that is, I have always encountered my audience in a “live” context. However, when attending “performances” of my audiovisual works, typically in a concert hall setting, I began to feel uncomfortable — this no longer felt like a performance. Instead, I typically sat in an audience whilst either I, or someone else, pressed “play” and the work was played back. For me, this simply wasn’t a performance; it was a screening. Yet, these playback events being described as performances gave rise in part to my feeling the desire to negotiate ways to perform my works in a live context — to be able to improvise aspects of the work in the moment and for the work to exist slightly differently in each subsequent iteration. It is the process of reorienting my practice towards live performance that forms that basis of this discussion.

This process, however, raised some interesting questions for me, namely how the ideas concerning perceptual unity that had historically guided my work were shaped by the introduction of a physical presence into the audiovisual space; how the sensory cohesion I sought in my practice was altered by my physical bodily presence within a live performance context. If I wanted my audience only to concentrate on the sonic and visual structures I was creating, what happened when I inserted my own physical presence into that performance context? Could I, wittingly or unwittingly, serve as a distraction, an additional point of visual focus that might take the attention of the audience away from the visual component on screen? Or would the addition of a physical bodily presence, rather than distracting from the visuals on screen, result in my exploring other forms of agency and context for my work, navigating the resistance inherent in negotiating a fixed practice within a live performance environment?

Performative Audiovisual Networks (PAN) and Anxieties of Failure

In most of my performance work, I have adopted the strategy of having a varied set of musical resources before me which I have structured as an arena for activity; a performance ecology. (Bowers 2002, 57)

Ideas concerning performance ecology have been particularly informative in navigating both the initial construction and subsequent conceptual deconstruction of my live performance work. Initially proposed by John Bowers (2002) and subsequently discussed by authors and practitioners such as Agostino Di Scipio (2003), Simon Waters (2007, 2013) and John Ferguson (2013), performance ecology can describe or denote the performance environment created by the artist and the physical, tactile components often inherent in technologically mediated live performance practice. For Bowers, openness to a wide variety of technologies and a broad range of potential human-machine interactivities affords flexibility in the negotiation of the performance space and performer/audience relationships.

Waters considers that contiguities between performer, instrument and environment can engender complex interactions that are in danger of being glossed over through rhetorical categorization into these delimited designations (Waters 2007). Indeed, he suggests:

The terms reify the corporeality (bodilyness) of the first, the goal-orientedness of the second and the otherness of the third. What is lost in this set of distinctions? What is masked, covered, generalized away in the mute acceptance of these separations… (Waters 2007, 2)

These contiguities in my own practice both existed and continue to exist through and between a number of roles, such as composer, improviser, performer, designer, technologist and educator, and certainly engender complex and somewhat nebulous interactions, a few of which will be teased out here.

The construction of my live audiovisual performance work began in 2012 and has been on-going ever since, initially in the work intervention:coaction (2014) and more recently in IC2 (2015), which builds on and expands intervention:coaction in a number of ways. As discussed previously, the initial impetus for the work came through my personal frustrations at attending “performances” of my work in which I felt little performance took place. However, the initial genesis was informed through my work with John Ferguson, Diana Salazar and Oded Ben-Tal at Kingston University, and in particular our founding of KUDAC (Kingston University Digital Arts Collective) in 2011.

In their article “Rethinking the Musical Ensemble: A model for collaborative learning in higher education music technology” (2014), Ben-Tal and Salazar have discussed some of the possibilities for collaborative musicking afforded by students’ participation at KUDAC:

In this collaborative context, understanding the technical (programming/hardware), æsthetic (sonic potential) and social (potential musical role in an ensemble) aspects of an instrument all require development in order to achieve mastery. The principal aim for each participant becomes not to build a complex instrument, but an instrument that facilitates meaningful contribution to the ensemble. This requires students to (re)evaluate their tools, to consider what is most appropriate in the ensemble situation, and to develop the listening skills to use the instrument effectively. (Ben-Tal and Salazar, 11)

Whilst it is, of course, somewhat redundant to discuss how formative relationships with like-minded colleagues can be in the development of one’s creative practice, it is important here not to underestimate just how key the live practice of particularly Salazar and Ferguson were in developing my own live performance work. Becoming better acquainted with their individual practice during the course of 2010–11 facilitated a desire to push into new and, for me, previously uncharted territories, but it was through founding KUDAC that I was first able to work in a live context collaboratively, not only with these established practitioners, but also with students currently studying music technology at Kingston and often with radically different musical experiences and ideologies to my own.

Increasingly relevant, therefore, were the possibilities of failure, in a range of contexts. Kim Cascone has, of course, discussed the æsthetic appropriation of technological failure in some detail:

It is from the “failure” of digital technology that this new work has emerged… Indeed, “failure” has become a prominent æsthetic in many of the arts in the late 20th century, reminding us that our control of technology is an illusion, and revealing digital tools to be only as perfect, precise, and efficient as the humans who build them. New techniques are often discovered by accident or by the failure of an intended technique or experiment. (Cascone 2000, 13)

Whilst the use of failure as an æsthetic strategy is something that became increasingly relevant to the sonic, and visual, outcome of my own live performance work, the anxieties surrounding failure of various forms came into relief during early work as part of KUDAC. The necessity to have “something” to play with at each session was something that was key to the development of one’s place within the ensemble — indeed, as Ben-Tal and Salazar have discussed, one of the central functions of KUDAC was to encourage those involved to develop individual approaches that utilized a range of tools from whatever was available to them 2[2. It is perhaps worth noting here that we provided a number of laptops, controllers and interfaces etc. for students to work with if they didn’t have their own.] and that through “removing many of their usual compositional constraints, students are then challenged to design their own, and in doing so, construct a role for themselves as part of the group. Here the roles of performer, composer and conductor become much more fluid” (Ben-Tal and Salazar, 2014, 9). This kind of fluency reflects Waters’ “complex interactions” (2007), often present in this type of performance practice, but also presents the possibility of failure — from frustrations of being unable to get something to “do what you want” or to articulate the kind of sound one is trying to make, to full technological breakdown.

Initially, working with KUDAC required a complete reconfiguration of my ways of working, which necessarily required a reconfiguration of the way in which that work was mediated. Though largely functioning around a laptop and software with which I was familiar, this quickly developed into an “infra-instrument” (Bowers and Archer 2005), featuring a range of controllers, a Victorian Piano Harp played with DPAs attached to my fingers, a large squeaky plastic mallet and, more recently, a range of circuit-bent toys. My personal performance ecology, which evolved initially through KUDAC, necessarily engendered fragility — an unpredictability in numerous of the components of the performance network that could, and sometimes did, lead to a failure to do what was expected. This initially lead to personal anxiety, due to relinquishing control over the minute details of the outcome of the performance, an anxiety that, through developing an idiosyncratic and personal ecology for performance, ultimately facilitated a kind of creative freedom I had rarely experienced to that point (and which, I feel, was subsequently reflected in my approach to working with fixed media). Indeed, the creative relationships developed here could quite neatly be related to both Adam Parkinson (2013) and Atau Tanaka’s (2010) ideas on the affordances of working with computers, and the creative possibilities engendered through the “unique and singular capacities” (Parkinson 2013) of this type of performance ecosystem.

Subsequently, my live performance work (which from here on I will refer to as PAN [Performative Audiovisual Network]) has developed into an often complex, unpredictable milieu in which sonic and visual structures exist simultaneously and symbiotically, one informing the other in an unpredictable system, where chaotic behaviour is coded into both the sonic and visual structures. The audiovisual performance ecosystem in PAN engenders fragility, ambiguity and failure, often working with failure as an æsthetic strategy both sonically and visually, and resulting in performances that are unique and significantly varied with each subsequent iteration.

Sensory Perception and Audiovisual Space(s)

Another concern underpinning my approach to fixed audiovisual composition has been the nature of the confines of the audiovisual frame. Michel Chion has remarked that “what is specific to film is that it has just one place for images” (Chion 1994, 67), yet what is often central to narrative film is the intent to imply the existence of the physical world beyond the confines of that frame, and this is something that I have sought to explore in my fixed audiovisual work. Often, the visual component has been intended to give a snapshot of a larger whole, suggesting an environment that extended beyond the confines of the frame (Figs. 1 and 2). However, this becomes more difficult to negotiate within PAN, as the audience is not so free to imagine the environment continuing beyond the confines of the frame; they are instead confronted with a physical environment and a bodily presence existing and inhabiting that previously imagined space.

This has lead to some interesting negotiations of the relationship between the physical and virtual within my own audiovisual performance ecology, and indeed some reconsiderations of the resistance between physical, virtual and embodied space within my own performance ecosystem. Susan Sontag has stated that “If an irreducible distinction between theatre and cinema does exist, it may be this. Theatre is confined to a logical or continuous use of space. Cinema… has access to an alogical or discontinuous use of space” (Sontag 1966). Though I do not necessarily consider my performance to be a form of theatre, there is nonetheless an implicit tension between the inhabited, physical space of the environment in which the work is performed and in which the audience is present, and the computer-generated visual component of the work projected on screen. These two visual spaces exist differently from one another; although they inhabit the same performance space, one is essentially physical and present — a comprehensible, logical and continuous space — whilst the other is essentially virtual — alogical, delimited and, in a sense, discontinuous.

One way to consider and attempt to reconcile these tensions might be through considering phenomenological theories of perception, and one in particular has guided my own approach to working with audio and video simultaneously:

The sight of sounds or the hearing of colours come about in the same way as the unity of the gaze through the two eyes: in so far as my body is not a collection of adjacent organs, but a synergic system, all the functions of which are exercised and linked together in the general action of being in the world, in so far as it is the congealed face of existence. (Merleau-Ponty 1996, 237)

Taken from Maurice Merleau-Ponty’s Phenomenology of Perception, the ideas presented here have led to my previously asking questions about the nature of successful non-narrative or non-representational audiovisual work, and in part guided my intention, to date, to create audiovisual work in which there is no sense of media hierarchy — to attempt to engage the senses equally within this synergic system. However, viewed from a contrary perspective, there could be seen to be an implication here that the nature of our audiovisual perception is such that the two are synergic regardless of the stimulus with which they are presented. It might be useful to consider some of these ideas further from the perspective of James Gibson’s ecological approach to vision: that we perceive an environment “not just with the eyes but with the eyes in the head on the shoulders of a body that gets about” (Gibson 1979, 211). Gibson uses this embodiment of viewing experience to discuss the relationships between ambient and ambulatory visual perception, but here we might relate it to the nature of the perception of different visual environments and spaces within a single performance work — that the physical and virtual visual spaces become part of a single, coherent visual environment, perceived (to combine Merleau-Ponty and Gibson) through the unity of the gaze through the two eyes in the head on the shoulders of a body that engages with the performance.

What, then, about the relationships between the visual space(s) and the auditory ones? This is further complicated in this particular performance context by the recent addition to PAN of the eloquently named “pink thing” — a circuit-bent toy I have incorporated into my performance array (Fig. 3). The “pink thing” is capable of outputting sound through the Pd patches that run the audio component of the work, but also has a built-in speaker, allowing it to make sound in an arguably different space to that which is played back through the loudspeakers (it also has a number of flashing red LEDs that contribute to the physical, visual space). Waters’ ideas concerning sound “touching at a distance” might be useful here, particularly his assertions that “our hearing allows us to structure relations with our acoustic environment meaningfully — affording the possibility of intimate, local and environmental zones of heard experience” (Waters 2013). The audience, by implication, is capable not only of perceiving and reconciling differences in visual space and relationships between physical and virtual media, but also of combining this visual perception with auditory perception to identify and navigate a range of audiovisual spaces within a single performance context. Indeed, as, Joseph Anderson has discussed in Moving Image Theory:

[T]he arrays of energies to which our senses are attuned are constantly changing, yet we are able to perceive a stable world.… [W]e do not passively catalogue random properties of the world as they are revealed to us through our senses; instead we actively look and listen… for the things that the environment might afford us. (Anderson et al. 2007, 3)

The nature of this affordance, then, is dynamic and constantly shifting within a live audiovisual performance context; there is tension, conflict and resistance in the complex interactions between the performer, the audience, the visual and the auditory spaces. Yet the audience perceives a coherent performed whole, not in the sense of the fixed, delimited audiovisual artifacts that are manifest in other forms of my practice, but in the live performance context in which the demands on the audioviewer are shifting and dynamic, both visually and aurally.

The primary difficulty here, then, in discussing and disseminating work of this kind is the same as that which I have come up against numerous times in the past: that while there exist numerous theories to account for our visual and auditory experiences of audiovisual work, in sensory isolation, there are very few that effectively reconcile the audio and video within a cohesive sensory whole. My continuing hope, as a practitioner, is that through foregrounding these issues in the discussion of creative practice, they will begin to be more widely discussed and, hopefully, eventually, reconciled.

Performance/Performativity and Documentation

Finally, almost as a coda to the discussion above, I would like to consider the documentation of this work as a peculiar and very specific addition to the relationship between performer and audience, and performance space and the audioviewer. I was asked by Butch Rovan to contribute a document of this performance work to a special edition of Computer Music Journal centred on interactive audiovisual performance, to which I gladly agreed. However, this raised further questions about the nature of the work — namely the attempt to translate a fixed audiovisual practice into a live performance context and subsequently, somewhat perversely, back into a fixed audiovisual document. The dynamic relationship between physical and virtual visual spaces, in combination with diverse acoustic spaces, would be condensed to a fixed document of the performance that, once again, inhabited a single delimited screen with 2-channel stereo audio.

For me, perhaps the most interesting and relevant question that emerged was this: What and where was the work? When editing the document together, I cut between three different angles of myself with the screen showing the computer-generated imagery, and a render of the visual component that was taken during the performance itself. Consequently, there were times when the performance environment, and myself as the performer, were not visible in the document; the performance at that point being effectively the same as one of my fixed audiovisual works.

In navigating these particular and rather idiosyncratic considerations, I found Philip Auslander’s ideas on “The Performativity of Performance Documentation” particularly illuminating:

[T]he act of documenting an event as a performance is what constitutes it as such. Documentation does not simply generate image/statements that describe an autonomous performance and state that it occurred: it produces an event as a performance and, as Frazer Ward suggests, the performer as “artist.” (Auslander 2006, 5)

What resulted from the documentation of the performance was an artifact that was not only illustrative of my live performance work, but also of the fixed audiovisual practice that preceded and surrounds it. The hybridization of physical and virtual I had sought in my performance work extended to a hybridization of multiple facets of my creative practice within a single artifact, one which, ironically, is probably most reflective of all aspects of my audiovisual practice to date whilst simultaneously not really being reflective of any of them. To return to Auslander:

The purpose of most performance art documentation is to make the artist’s work available to a larger audience, not to capture the performance as an “interactional accomplishment” to which a specific audience and a specific set of performers coming together in specific circumstances make equally significant contributions.… In that sense, performance art documentation participates in the fine art tradition of the reproduction of works rather than the ethnographic tradition of capturing events. (Ibid., 6)

There might be some interesting conversations to be had here, in terms of discussing and relating some of the ideas concerning perception covered earlier to the experience of audioviewing a performance document as opposed to a live performance. But this will have to wait for another paper.

Bibliography

Alves, Bill. “Digital Harmony of Sound and Light.” Computer Music Journal 29/4 (Winter 2005) “Visual Music,” p. 54.

Anderson, Joseph D. and Barbara Fisher Anderson (Eds.). Moving Image Theory: Ecological considerations. SIU Press, 2007.

Auslander, Philip. “The Performativity of Performance Documentation.” PAJ: A Journal of Performance and Art 84 (2006), pp. 1–10.

_____. Liveness: Performance in a mediatized culture. Second edition. Routledge, 2008.

Ben-Tal, Oded, and Diana Salazar. “Rethinking the Musical Ensemble: A model for collaborative learning in higher education music technology.” Journal of Music, Technology & Education 7/3 (December 2014), pp. 279–294.

Bowers, John. “Improvising machines: Ethnographically informed design for improvised electro-acoustic music.” ARiADATexts 4 (September 2002). Available online at http://cid.nada.kth.se/pdf/CID-195.pdf [Last accessed 1 April 2016]

Bowers, John and Phil Archer. “Not Hyper, Not Meta, Not Cyber but Infra-Instruments.” NIME 2005. Proceedings of the 5th International Conference on New Instruments for Musical Expression (Vancouver: University of British Columbia, 26–28 May 2005). http://www.nime.org/2005

Cascone, Kim. “The Æsthetics of Failure: ‘Post-digital’ tendencies in contemporary computer music.” Computer Music Journal 24/4 (Winter 2000) “Encounters with Electronica,” pp. 12–18.

Chion, Michel. Audio-Vision: Sound on Screen. Trans. Claudia Gorbman. New York: Columbia University Press, 1994.

Di Scipio, Agostino. “‘Sound Is the Interface’: From interactive to ecosystemic signal processing.” Organised Sound 8/3 (December 2003), pp. 269–277.

Ferguson, John Robert. “Imagined Agency: Technology, unpredictability, and ambiguity.” Contemporary Music Review 32/2–3 (2013) “Resistant Materials in Musical Creativity,” pp. 135–149.

Gibson, James J. The Ecological Approach to Visual Perception. Classic Edition. Psychology Press, 2014.

Merleau-Ponty, Maurice. Phenomenology of Perception. Trans. Colin Smith. Delhi: Motilal Banarsidass Publishe, 1996.

Parkinson, Adam. Embodied Listening, Affordances and Performing with Computers. Ann Arbor MI: Michigan Publishing, University of Michigan Library, 2013.

Tanaka, Atau. “Mapping out instruments, affordances and mobiles.” NIME 2010. Proceedings of the 10th International Conference on New Instruments for Musical Expression (Sydney, Australia: University of Technology Sydney, 15–18 June 2010). http://www.nime.org/2010 Available online at http://www.nime.org/proceedings/2010/nime2010_088.pdf [Last accessed 1 April 2016]

Tanaka, Atau, Alessandro Altavilla and Neal Spowage. “Gestural Musical Affordances.” SMC 2012. Proceedings of the 9th Sound and Music Computing Conference (Copenhangen, Denmark: Aalborg University, 11–14 July 2012), pp. 318–325. http://smc2012.smcnetwork.org

Sontag, Susan. “Film and Theatre.” The Tulane Drama Review 11.1 (1966), pp. 24–37.

Waters, Simon. “Performance Ecosystems: Ecological approaches to musical interaction.” EMS: Electroacoustic Music Studies Network, 2007. Available online at http://www.ems-network.org/IMG/pdf_WatersEMS07.pdf [Last accessed 1 April 2016]

_____. “Touching at a Distance: Resistance, tactility, proxemics and the development of a hybrid virtual/physical performance system.” Contemporary Music Review 32/2–3 (2013) “Resistant Materials in Musical Creativity,” pp. 119–134.

Whitney, John. “To Paint on Water: The audiovisual duet of complementarity.” Computer Music Journal 18/3 (Fall 1994) “Composition and Performance in the 1990s (2),” pp. 45–52.