Interactivity

by Danny Dursley

… the art work … is no longer a static object or a pre-defined multiple choice interaction but has become a process-like living system. (Sommerer and Mignonneau 1998)

Interactivity in the electronic arts is a word that appears to have an unclear meaning. For the sake of this essay I am going to relate the term interactivity to that between a human and computer or other electronic device. The word tends to refer to an artwork or piece of technology that has been created that can or does require some form of human interaction but this is not always true. Many researchers and musicians who attend the NIME (International Conference on New Interfaces for Musical Expression) conferences publish papers that claim to document the evolution or concept of some form of interactive system but there is a small group of people who claim that this is an inaccurate use of the word interactivity. Bert Bongers suggests that many interactive systems these days are in fact “reactive” (Bongers 2000) and he is not alone in this view. Garth Paine and Tod Machover also subscribe to this view and Machover states that too many interactive systems rely on complex and elaborate controllers to ‘attract easy attention.’ (Machover 1992) The aim of this paper is to give a clear and justifiable meaning to the word interactivity and to establish whether many interactive systems have correctly used the term. Firstly, I will give a brief history on the development of interactive music systems

One of the foremost pioneers in the development of interactive music systems is Joel Chadabe. In 1972, Chadabe created a digital system known as Daisy. Daisy controlled a number of different analog sound processors that would manipulate real-time sonic events and the spatial location of the sounds being emitted from the speakers. Chadabe later formed the Intelligent Music Company in the mid 1980s and was responsible for the development and publication of a wide range of innovative software including Max. Another notable contribution to interactive music was the San Francisco Tape Music Centre (SFTMC) that was established in 1959 by Morton Subotnick and Ramon Sender. SFTMC has gone on to produce such composers as Pauline Oliveros and Terry Riley. MIDI (Musical Instrument Digital Interface) has become a dominant force in interactive systems since the 1980s. Composers such as Dexter Morrill and Gary Nelson developed programs that controlled MIDI information coming in and rerouted this information to different synthesisers and effects units. Nelson created a MIDI horn for Warps of Time (1987) that triggered different effects and algorithmically generated musical output to various synthesisers (Schwartz 1993). In more recent years, the emphasis has been pushed towards Artificial Intelligence (AI) and gestural controlled elements of performance in the development of a truly interactive music system. Gestation (2003)by Garth Paine is an interactive installation that uses human movement to generate unknown results. The installation is split into two integrated galleries. The First gallery is where the user interacts with sound while a motion sensor tracks their movements. Algorithms are used in real-time to tightly configure the movement of the user with the sounds that are being generated. In the other gallery a background of images is displayed on a wall that represents the birth of new life in response to what the user is generating in the other gallery. Ultra-sound videos are used to form the bases of these new life-forming cells that have varying rates of growth depending on the gestural movements of the user in the other gallery. The two galleries are detached to illustrate the hidden outcomes of one’s actions whereby the user is only half aware of what is being created. This creates a sense that the user is truly the creator of the work being produced even though he/she is unaware of it. It are installations like this that try to move beyond that of the complex and elaborate controller and focus on the relationship of the user and the system that surely should be the main focus of any interactive installation. Whether this installation is truly interactive is an issue I will try and clarify.

The Oxford English Dictionary defines interactivity as “pertaining to or being a computer or other electronic device that allows a two-way flow of information between it and a user, responding immediately to the latter's input.” This clearly suggests that the computer or other electronic device has an equal part in the interaction. Each responds to the other’s output and this in turn affects the subsequent output. I think it might be fair to say that the majority of papers published at NIME do to some degree comply with the above definition. What the definition does not make clear is whether the contingency view has an effect or bearing on the definition. Sally MacMillan states that in the contingency view, there are three levels of interactivity. The first is noninteractive, where a message is not related to the previous message. The second is reactive, where a message is related only to one immediately preceding message. This could relate to the process of cooking a meal in the microwave whereby the user inputs a time for however long she wishes to cook her meal and then the microwave informs her when the time is up. The third is interactive, where a message is related to a number of previous messages and the relationship between them. It is at this point that our definition of interactive starts to merge with the word reactive. As stated by Bongers, many interactive installations could be described as being reactive rather than interactive. In “Interactive Public Sound Art: A Case study,” David Birchfield et al. describe an interactive installation called Transition Soundings. The installation consists of a wall containing a large bank of sensors situated behind the bench of a bus stop. As a user approaches the wall, sound emerges from the wall as his movements are detected. With time the sounds emerging from the wall will change as day turns to night and the same movements now generate new outcomes. This installation clearly shows an interaction between the user and the system but is this strictly interactive? If we refer back to MacMillan’s definition of interactive we can see that in order to be an interactive installation there must be a two-way flow of information whereby the response of each participant is related to numerous other responses and the relationship between them. Transition Soundings does not conform to this definition as the actual installation has no knowledge or memory of previous responses and bases its output purely on the last known message received from its sensors. According to MacMillan, this installation would be classified as reactive. In order for their to be a two-way flow of information between the user and the system there must be some form of cognition, no matter how rudimentary, in order to process the information and use it in response to other bits of information coming in. This suggests that all interactive compositions must have some form of artificial intelligence (AI) in order to be truly interactive. It is for this exact reason that Bongers believes that many of today’s interactive installations are actually reactive due to their lack of cognition.

Shortly after joining the cec-conference email list in 2007, I posted the question: “What is your definition of the term Interactivity?” What followed was not what I had expected as a number of participants seemed to be quite passionate about their answers and defended any possible objection anyone made towards them and their opinions. I have compiled some of the definitions and edited them for clarity for this paper (see below: List of Definitions of “Interactivity”). The reason I asked this question was so that I could get a modern day view of the term and to see if it had changed in any way over the years. One general consensus is that for an installation to be interactive there has to be some form of human interaction and “without interaction, it cannot come into being” (Pelletier). This is a fair point and was at first overlooked by myself as I presumed that all interactive installations had to have some form of human participation. Carey Dodge went on to say that it is the “audience” that has to interact with piece before it can become “active”. This suggests that for a piece to become interactive, it must be interacted with by anyone apart from its creator. This would rule out live performance as interactive art and focus the definition on public art alone. This though may not be the point Dodge was trying to make as the use of the word “audience” is quite ambiguous in this context. Both of these views contradict that of the Oxford English Dictionary as they do not confine interaction between that of a human and computer/electronic device. If we focus on MacMillan’s definition of interactivity there is one point that seems to be overlooked. While I initially agree with this definition it does not take into account the nature of choice. While a participant may interact with an installation, sending and receiving information, he has the choice of whether he wishes to accept this information. If he does not accept this information the outcome would be similar to that of a reactive system. This suggests that interactivity would be more accurately described as a quality as it covers a scale of interaction rather than a single polar extreme. This scale could be said to be between that of a reactive system and that of an interactive system with everything in between all classed as interactive. If we focus, again, on the comment made by Jean-Marc Pelletier that a “truly interactive musical piece must be such that without interaction it cannot come into being,” we can see that his views are not too dissimilar. I very much agree with his view as it provides a level of control for the user in which they can make the system interactive or not. Again it is choice that makes a system interactive and not merely the level of cognition each participant possesses. I feel that these two factors, choice and cognition, when combined make up that of an interactive system. Only when the element of choice is included into a system can it become interactive. A reactive system may come into being with a chaotic form of interaction but it will never posses an interactive quality if neither agent has the choice of accepting the information exchanged by the other party. Andrew May concurs with this view and goes on to state that the information exchanged must add “meaning to the system”. This meaning is directly related to the choice each agent makes on how they process the received information. Without meaning, a choice is no more than a random element. I feel Kevin Austin helped sum up the nature of the word interactivity:

I consider that “interactivity” doesn’t exist, but rather there are a set of practices which the community (and those outside) have started to group together and label them “interactive”. Maybe before interactive, there was (simply) reactive: I turn a knob, the door opens.

I feel this view explains how the definition of interactivity has evolved over time. As technology has progressed more and more people are willing to classify their work as interactive. I feel this is an unjust use of the word and while I do not agree with Austin that the word interactivity does not exist, I do agree that as technology advances the set of practices used to define this term will increase. Due to this, it is very hard to pinpoint an absolute and all-encompassing definition of the term interactivity. While I believe that for an artwork to be classified as interactive it must contain some form of cognition, there is no possible way of achieving a general consensus of this view or any other. Until there is a significant change in the general consensus, the debates will have to continue.

List of Definitions of “Interactivity”

Interactive art is that in which a viewer can take some measure of the extent to which the artwork is aware of the viewer. (24 wds).

The viewer might go along with art which seems responsive to one’s intuitions about how things, either in reality or in art, tend to go. (all mass culture is interactive in that sense).

The underlying idea is to arrange for synchronized thinking, good or bad.

Interactive electronic art is, on this view, is about the poetry of a supplemental consciousness in which I can merge my existence.

— Eliot Handelman (22 March 2007)

Interactive. 1. reciprocally active; acting upon or influencing each other. 2. (of a computer or other electrical device) allowing a two way flow of information between it and a user.

— Rob Godman (22 March 2007)

Interactivity describes a system in which two or more agents (connotatively, a combination of human musicians and computers) exchange information, and in which each provides information that adds meaning to the system. This implies that each agent processes input information and generates output information that is partially, but not entirely, derived from input.

— Andrew May (22 March 2007)

Musical interactivity is an information exchange system, conversational and bidirectional in nature, between musician and data/computing device, which transforms the function of the two agents in equal measure.

— PerMagnus Lindborg (22 March 2007) [Also see article by Lindborg in this issue: “Reflections on Aspects of Music Interactivity in Performance Situations.”]

I think the problem is that interactive is a buzz word, a marketing term, and is used to mean many different things, ranging from artificial intelligence to “more flexible in performance than tape.” As technical strategies I don’t think there is anything wrong with any of these; the problem lies in the way in which the æsthetic implications of all these things become blurred and misunderstood. A flexible triggering interface or some minor (perhaps perceptually or musically unimportant) variation in each performance of a piece, are very different from free improvisation, or an Anthony Braxton composition, or a piece by Christian Wolff.

— Scott Wilson (23 March 2007)

interactive usually means that the audience has to “interact” with the piece for it to be “active”.

— Carey Dodge (23 March 2007)

I consider that “interactivity” doesn’t exist, but rather there are a set of practices which the community (and those outside) have started to group together and label them “interactive”. Maybe before interactive, there was (simply) reactive: I turn a knob, the door opens.

— Kevin Austin (23 March 2007)

A truly interactive musical piece must be such that without interaction, it cannot come into being.

— Jean-Marc Pelletier (24 march 2007)

Interactivity may be more of a quality than a form or style. That is, a piece might include some interactivity without having to be defined as an “interactive piece”. Also, if we are looking for stylistic and form related definitions, there seems to be a significant difference between pieces where performers are interacting with each other, and where performers are interacting with an audience, and then again where these lines become blurry (performers interacting with an audience who are interacting with another audience who are interacting with a device designed by performers/composers to interact with another device, etc.).

— Digs Dorfman (25 March 2007)

Bibliography

Balaban, Mira, Kemal Ebcioglu, and Otto Laske (eds.). Understanding Music with AI: Perspectives on music cognition. Cambridge MA: MIT Press, 1992.

Birchfield, David, Kelly Phillips, Assegid Kidane and David Lorig. “Interactive Public Sound Art: A Case study.” Proceedings of the 2006 Conference on New Instruments for Musical Expression (NIME-06, Paris, 4–8 June 2006).

Bongers, Bert. “Physical Interfaces in the Electronic Arts: Interaction Theory and Interfacing Techniques for Real-time Performance.” Trends in Gestural Control of Music. Marcelo Wanderley and Marc Battier, eds. Paris: IRCAM—Centre Pompidou, 2000. [CD-Rom]

Honavar, Vasant and Leonard Uhr (eds.). Artificial Intelligence and Neural Networks: Steps toward principled integration. Neural Networks, Foundations to Applications. Boston; London: Academic Press, 1994.

Leman, Marc (ed.). Music, Gestalt and Computing: Studies in cognitive and systematic musicology. Lecture Notes in Computer Science Vol. 1317. Berlin: Springer Verlag, 1997. [With CD-Rom]

McAdams, Stephen and Irène Deliège (eds.). Music and Cognitive Sciences. London: Harwood Academic Press, 1989.

McMillan, Sally J. “Exploring Models of Interactivity from Multiple Research Traditions: Users, Documents, And Systems.” Handbook of New Media. Edited by L. Lievrouw and S. Livingston. London: Sage, 2002, pp. 162–82.

Machover, Tod. “Instruments, Interactivity, and Inevitability.” Proceedings of the 2002 Conference on New Instruments for Musical Expression (NIME-02, Dublin Ireland, 24–26 May 2002).

Paine, Garth. “Interactivity, Where to from here?” Organised Sound 7/3 (December 2002). Cambridge MA: Cambridge University Press, pp. 295–304.

Robertson, Andrew N. and Mark .D. Plumbley. “Real-time Interactive Musical Systems: An Overview.” Proceedings of the Digital Music Research Network Doctoral Researchers Conference (London, 22–23 July 2006.

Rowe, Robert. Interactive Music Systems: Machine listening and composing. Cambridge, MA: MIT Press, 1993.

_____. “The Aesthetics of Interactive Music Systems.” Contemporary Music Review: Live Electronics 18/3 (1999), pp. 83–97. London: Harwood Academic.

_____. “Personal Effects: Weaning Interactive Systems from MIDI.” Proceedings of the Spark Festival 2005.

Russell, Stuart J. (1995). Artificial Intelligence: A modern approach. Prentice Hall series in Aicial Intelligence. Englewood Cliffs NJ: Prentice-Hall International, 1995.

Schwartz, Elliott and Daniel Godfrey. Music Since 1945: Issues, Materials, and Literature. New York: Schirmer Books, 1993.

Sommerer Christa and Laurent Mignonneau. " Art as a Living System: Interactive Computer Artworks." Leonardo 32/3 (1999), pp. 165–73.

Temperley, David. The Cognition of Basic Musical Structures. Cambridge MA: MIT Press, 2001.