Conversation with Marco Donnarumma, Claudia Robles and Peter Kirn

Performing biological bodies at Body Controlled #4 ‘Bio Interfacing’

by Pedro Lopes and jef chippewa

The following discussion with the invited artists at Body Controlled #4 — Bio-Interfacing took place at LEAP (Berlin) on Sunday, 15 July 2012. Lopes led the discussion and eContact! Editor jef chippewa was also on hand and periodically interjected. The Friday evening concert featured solo performances by Donnarumma, Robles and Kirn, and was preceded by a Keynote Lecture given by Lopes. Due to scheduling conflicts, Marco could only be present for the second half of the discussion.

The full discussion has been transcribed below.

Body Controlled #4 — Bio-Interfaces
11–15 July 2012
Berlin, Germany: LEAP — Lab for Electronic Arts and Performance
http://www.leapknecht.de

[Pedro Lopes] The first thing I wanted to ask you guys, because this upcoming issue is about biosensors and revolves around bioart, and in that sense, the first preliminary question is: what got both of you interested in using sensors for your art, instead of using a more standard controller, using biosignals?

[Claudia Robles] I’m interested in biodata, and in the imperceptible things that we cannot perceive during the normal day; for example, the heartbeat is running all the time but you never really pay attention to it, you are not aware of your body. That was the point of using biodata, to make the body visual, or audible.

[Peter Kirn] Since I was in on the curatorial discussion [for Body Controlled #4], I can probably speak to that better than about my own work, since this is the first time I’ve done a piece like this [using biodata]. My interest as a journalist has always been in how we design musical interaction, how we design musical instruments, scores and technology in general — this certainly fits under the larger umbrella [of biomusic]. Once you are in the realm of the computer, everything is numbers, so I think part of why you see artists branching out into other fields, part of what enabled me to go this direction (when it’s not normally the direction I go), is because everything in the digital space is numbers, and it allows you to translate any input into any output. It is enabling composers, sound designers and performers to experiment with those kinds of new directions. Because you have this sense of “well, I could just throw out this MIDI keyboard and replace it with my heartbeat.” And that’s encouraging lots of experimentation, the neutrality of the computer. In terms of this panel, we have had a series of Body Controlled that I have been able to be involved with at LEAP. With the curators here, there has been an attempt to look at all of those different areas. So it’s nice to be able to have this “bio-event” in that larger context.

This was absolutely the first time that I had ever done anything like this, and I am not an expert on it in any way, but I am always interested in how you design interaction with the machine — for music — and how it works for performance. And I was interested in this case in exploring the role of music and mental state; that is something that is relevant to us all in everything that we do. The opportunity to use musical feedback in a bio-feedback system means that you can really create a sound world that is involved directly with how you feel, how your internal state is.

[PL] So, you’re saying that musical interaction nowadays is mostly digital — any input is translated into 0 and 1’s. I remember a talk by Atau Tanaka — who is also a big mentor, a pioneer of biomusic — at the Sound and Music Computing Conference in 2009 He said one of the big reasons why people are just turning to this, is that, unlike any other controller you can play with, any type of discrete controller, the body acts on its own, you cannot determine everything. Is that something that you are really interested in as performers, in the sense that it branches out into something that the digital realm cannot do by itself?

[CR] Something for me that is important is that you can control your body data more or less, but certain things you cannot control at all! This is why I really love [using brainwaves captured via EEG]. I can really try during the performance to go from meditation to a trance state but I cannot control all the parameters. For example, I have worked with muscle sensors, too, and I remember that the first time I used them, I was impressed that even if I was really relaxed, I had tension in my muscles all the time. That was great, that I can’t really completely control my body. That is something that is very important for me.

[jef chippewa] There is a constant state of flux already existing in the body…

[CR] Exactly, of tension.

[JC] … regardless of any action that you might do. Like when Marco is tensing his muscles [captured via MMG], I wondered the other day, what would happen if he was not moving at all. Because during his entire performance, he was tensing the muscles.

[CR] You have tension there all the time, that is what I discovered when I used it. Is not possible to have zero [tension].

[JC] If you would get zero, basically you would basically get the sounds of your blood flowing.

[PK] Yeah, this self-awareness is so important in performance. You constantly face this challenge anywhere, that you want to be self-aware but not too self-aware. I was listening to a radio investigation of baseball pitchers and what happens when major league guys when they suddenly lose the ability to pitch, because they start over-thinking! When you bring that element directly into the performance, you really challenge yourself to be doubly aware [of yourself]. For me, part of the appeal of the bio-feedback system is, as you were saying, that you can’t simply think about what you’re trying to do. A lot of people asked me about that after my performance, were you intentionally trying to control what was happening?

[CR] I try to be in control, when I perform.

[PL] I see [your work] as particularly good examples of what we have been talking about. The GSR and EEG are both highly complex to control, and if you are nervous [in the performance] 1[1. On the Friday evening programme, Peter performed Seas of Tranquillity, a new generative composition using a Galvanic Skin Response (GSR) sensor, which measures skin conductivity, itself influenced by many body factors. Claudia performed her work INsideOUT, for which electroencephalographic (EEG) measurements of her brain activity are used to control audio and images.], it is going to appear in the sound [the sensor will display higher values]!

[PK] Which is also kind of interesting!

[CR] Yes, for example, I remember the first time I played this piece, I programmed all the things while working at home, so I could really control the meditation states. But then, on stage for the first time, all the values were so high that I couldn’t really control them in the performance, it was impossible, the values were high all the time! I couldn’t do anything about it.

[PK] I tried to increased the range [the sensitivity, of the sensors] but that happened, because I am completely new to this, so when I came in for the sound check, the levels were way above what they were supposed to. Really, I wasn’t aware or conscious of being that much more nervous, but there was that extra excitement in the sound check that I hadn’t had before. So I did try to make the sound range of this piece such that it would work regardless of my mental state. Of course, it’s not only about trying to keep those levels low and be relaxed, the GSR will respond if you are excited or aroused. I said that somebody could try to creep up on me from behind and scare me, that would work, or naked women walking around, but it doesn’t necessarily have to [do with me being in a] positive or negative [state], whether I am relaxed or not, it does tap into some level of awareness; this additional input beyond what you are mentally conscious of.

[JC] There is an interesting issue implicit in all of this, from the start of the use of these technologies in a performance situation, even though it has been recognized that there are implicit signals in the body, there has been always an attempt to mediate them. Even with Lucier’s piece [Music for Solo Performer], there is a really basic form of mediation. I guess my provocative question is: if these signals are in your body already, and can be transmitted through [or mapped onto] a reception system, why not just let the thing run [by itself], and call it an installation?

[PK] Ah! But if you can hear it or see it, you will start to mediate it. That was the other reason that I was interested in the bio-feedback angle and I think there is an element of that in all three pieces [on Friday’s programme]. If you have any kind of feedback at all, you may consciously or unconsciously begin to mediate what’s happening. My parents always observed that when you go to a nurse and they check your temperature, very often they will leave the temperature gauge leaning towards you, if you can see the numbers when someone is taking your temperature, you can very easily — again, consciously or even unconsciously — begin to control your body temperature.

[CR] This is the way bio-feedback works!

[PK] So really, artistic intentions are none. If you have any output at all from what your are doing, you can intentionally or unintentionally change what is happening.

[PL] Apparently one of the most controversial things I said in my talk on Friday was that when you are observing [an artist performing using biosignals], we are actually a bit closer to him, to his mind, to his body, than when we see somebody who is not doing anything with biosignals, but is rather playing whatever score [for an acoustic instrument] he wants to perform. If somebody is using EEG data, like Claudia, she cannot lie, if she gets nervous…

[PK] Well, you can, but your whole body is lying; there is essentially nothing between your body and what is happening [sonically].

[PL] Its very interesting how people are somehow still not comfortable with this idea, that [the artist is “playing his or her body”]. Before he played, Marco said, “I am going to give you the sounds of my inner body.”

[PK] That is a different issue, the concert programme describes this as something from “the inside of the body,” but two of us are actually using skin-based data. Claudia and I are both [working] on the level of the skin, and the microphone is listening from the skin as well as below the skin.

[JC] I think the issue came up during Pedro’s talk as well, when you were showing slides. Especially when you get to somebody like Stelarc, when people aren’t familiar with that stuff, they see it and immediately think of chips implanted in [the human] body, and have this sort of [fear that] we’re all going to become cyborgs.

[PK] And it taps into various phobias, too.

[JC, to CR] What has your experience been, talking with audience members after your performances? It’s not the same as having an implant, it’s still an external thing, but there are still incredible amounts of technology [used in recording and presenting] biological functions.

[CR] Sometimes people have said they feel a little bit of stress in their body towards me, probably when, as I am performing, I try to go into [a stressful state]. People feel that. Probably the sound [affects them] too. But this is in fact what I want to do — I want the sound and video to affect the audience too. They go with me, in this feeling.

[JC] But what sort of comments have you had regarding these kinds of technologies? Or do people just see this as another approach to performance, using electronic media?

[CR] Yeah, there are sometimes people who are really scared by it, like “What’s happening there?” But it’s not really common.

[PL] I think, as Peter said, it is common ground, all of these phobias. We talked previously about Stelarc [and Arthur Elsenaar], they might be the extreme, but still, phobia is a nice description. Because people are still concerned with these type of performances. For example, I organized a concert for Marco in Lisbon earlier this year [at the 2012 NetAudio Festival] and one of the concerns was that the sound in Marco’s performance comes from within his body; people could not believe that the sound could come from the muscles. I know that in your case, you are mapping, so it’s different… To some extent, people become scared that the sound — unlike visuals — actually comes from within his body, and I don’t know where this concern comes from.

[PK] How did they express concern?

[PL] You can see recurrences of this question [about the source of the sound], even in the media when they wrote about the event.

[PK] Was it fear or just not understanding?

[PL] I think it was a non-understanding… well, sometimes fear and non-understanding blend.

[JC] I see that in Marco’s explanations [of his sensor-based works], he talks about using the muscle sounds, but very quickly he also [mentions the use of] algorithms and mapping strategies. So he immediately brings up the language that the milieu is accustomed to.

[PK] But it is part of the process, what you are hearing is not raw…

[JC] Pure muscle sound.

[PK] Well, nothing is pure, it is coming through the microphone, but it is not unadorned. Some of it is mapped and some of it is more heavily processed. That serves an æsthetic purpose, I mean he is not building medical equipment.

[JC] No, but the description of it very quickly goes towards things that people are familiar with, to be able to say, “Well, it’s out there, but actually it’s not that far out.”

[PL] Yeah I think it’s a fall-back.

[PK] Right, but otherwise I think people might feel cheated, somehow. You feel obligated to disclose the things that you are doing that are not purely biological. Even with medical equipment, you have to add filters and you have to amplify the sounds and you have to process the signal. What is different for us [artists], is that we can do whatever we like, we have æsthetic ideas and desires to fulfil.

[JC] Another difference is that a medical patient in a hospital isn’t going to ask about the mapping strategies. [laughs] He just accepts it!

[PK] True, but then again, it seems to me that there is no reason why a bio-feedback system shouldn’t be æstheticized. Why not? I guess that’s what differentiates us in the music world from people in the medical world, that we want to think more about the quality of the experience, to have it not just be clinical, but to — why not? — enjoy it. Maybe with our massively aging population, it will become increasingly important. Why does everything have to give the appearance of being pristine or clinical, or unæsthetic? Medical facilities, medical equipment… the sounds that they make are ugly [laughs].

[PL] Yep! Why did they choose that mapping? [laughs]

[PK] But then you do feel obligated to disclose — just in case someone accuses you of cheating somehow — that you are adding these extra layers.

[PL] This is actually something that [recalls other practices such as free jazz or free improvisation, where the musicians may disclose that there is no explicit musical planning or score for the performance]. This starts to happens with biomusic and biophysical music [the “lie”].

[PK] We can lie to ourselves, that’s the beauty of these systems.

[PL] It’s the same thing [for questions like] “Where does the sound come from?” Or, “What is she ‘thinking’ during the performance?” Do people ask you that? Are they concerned?

[CR] I read or think about a text, that I wrote many years ago about spaces: spaces that produce pressure on me and spaces that are open [which provoke some kind of relaxation]. [During the performance] I play with this, I think about this text, I speak this text and I am really thinking about these spaces. [I may be thinking about, on the one hand,] something that is very, very dense, or, [on the other hand, something that is very] light, for example.

[JC] About the psychological implications of these spaces?

[CR] Yes, because it directly affects my state, I feel that. I try to go into this “stress” trance, where I really think about spaces in which I feel pressure, and I try to play with both states [“meditative” / “stressed”].

[PL] The practice of playing with signals from the body [using bodily signals for audio-visual or other types or live performance, such as GSR or EEG data] has been around for many years. As always, once things have been around in the art world for some time, [artists, curators and audiences] start asking questions like “Are they still valid”? “Why are you people still doing this?”… Why are you still using GSR, EEG data? Where do you guys think this is going?

[PK] Well, that is one of the things that I brought up when we had this conversation about doing this event. Where is it going? There is going to be more of it, and I think that’s a safe prediction. Because everybody is carrying around these mobile computers, called iPhones and other things.

[PL] Availability…

[PK] There is going to be a lot more data collection, because somebody is going to try to exploit that. Whether they are successful or not, they are going to try to do it; they are going to see an opportunity and they are going to try to make it happen. There is also tons of investment in medical and health areas, it’s the one area of the US economy that is growing, so there is going to be a lot of investment there; Europe has an aging population, so there is going to be a lot of investment here. It’s pretty safe to say that these technologies are going to be out there. I guess I’m encouraged that artists are intervening and themselves getting involved, because you know that people are going to be coming at this from economic areas, from medical areas, It’s kind of exciting that artists already have a head start — as artists often do — in human-computer interaction, in thinking about some of these things. And dealing with them in æsthetic and cultural ways, and not just what’s perceived as a purely functional approach to these mechanisms. So I think there is going to be a lot more of it.

Now, how I got into it was I was tricked into it because it was part of the discussion for this event, and I accidentally volunteered to make a piece and didn’t have any direct background or research in this myself, so I tricked myself — or was tricked — into doing this piece. But I am really, really happy that I did, and I’m honoured to be part of a programme with people who have done more actual research than I have, and have been doing this for longer than… since Friday! Because I had this tremendous experience in doing this piece that I didn’t really expect until the performance. There is this sense of being aware of my own internal state, body aside, being aware of my internal mental state.

[PL] Is this really like [learning] a musical instrument, in the sense that there is virtuosity, there is taking months and months and years and years to control? Did you feel that?

[PK] Well, my piece is not virtuosic at all, so no! [laughs]

[PL] Did you feel that there were so many things that you did not control, there are so many things you have to learn about your own body… ?

[PK] Well, the input [of my system] is pretty simple — it’s just a very basic gauge, it’s one of the three or four most basic things possible you could do, “bio-wise”. You can be a musical instrument without being a virtuoso. But it’s the simple feeling of being an oscillator, the way that I set it up. I’m not “doing” anything, I sit in place for 15–20 minutes just being the oscillator source 2[2. Recordings of the performances from Friday’s programme can be heard on LEAP’s Soundcloud site: Marco Donnarumma — Music for Flesh II (10:27), Peter Kirn — Seas of Tranquillity (21:58), Claudia Robles Angel — INsideOUT (21:43).]. It was really satisfying and I had some surprises in regards to how I felt and in my awareness of how I felt. I had to be aware, but not aware; think about it, but not think about it.

[CR] This is why I decided to start to work with biodata, I was really interested in these things that we are not aware of in our lives. This is the leitmotif in my work, looking at the unperceivable things. If I explore all these unperceivable things in the world, why not in my own body? Instead of looking outside…

[PL] I really like the description of INsideOUT, where you state that you are looking at “the invisible world”.

[CR] I am inspired by butoh, the Japanese [dance/performance art]. I have in fact worked with butoh dancers and muscle tension [Seed / Tree (2005)]. They danced very slow and because of the muscle tension it was very good to work with muscle sensors. I’m really very inspired by this kind of work, about slowness and awareness, really going deep. For example, in butoh there is this kind of trance during the performance that is really fantastic. I try to [produce a similar type of trance using other means] in INsideOUT. 3[3. More information on this work and INsideOUT can be found in Robles’ paper published in the NIME 2010 Proceedings: “Creating Interactive Multimedia Works with Biodata” (PDF).]

[PL] Actually that is where I find your performance interesting, because both you and Peter are more static and more “invisible” than Marco in a performance situation. When you look at Marco, it’s very physical, you can determine [when] something is going to happen; you follow his movements, there is predictability, you understand what is going to happen (although the mapping [of the biodata] is his own). Whereas with you guys, I could sense that people were looking at you and…

[PK] “What are they doing?!” [laughs]

[PL] … something completely “odd” happens. Which is beautiful. It’s invisible, the mapping is invisible.

[JC] There was a guy that got up in the middle of your [Peter Kirn] performance and was standing almost right behind you, looking over your shoulder, and came back with his hands in the air, as if to say, “I don’t know what he’s doing!” [general laughter]

[PK] That is great! There is a huge movement from the DJ scene right through to the experimental classical scene, about wanting to make everything visible — as if the most important part of the performance was to be able to see what people are doing. [Marco Donnarumma enters] So it’s kind of nice to thwart that intention and make it completely invisible. Because I am not sure that that is the most important part of a performance. […] As an audience member, you have to have faith in the performer [that he is performing]. And of course with the computer, performers can fool you about what they are doing, maybe in a way that isn’t possible with acoustic instruments.

[PL] You mean the metaphor of “checking the email.”

[PK] Yeah, absolutely, but at the same time, I am not a violinist, I am a composer. So I know quite a bit about how those instruments work, but a lot of audience members don’t know a whole lot about how instruments work. If everything has to be seen, that would suggest that you could be at a live show and close your eyes and the performance would cease to be. So I like the idea that some things are invisible and require an act of faith. For example, I programmed one thing in the timing of my piece, a change from a sine wave oscillator to a saw wave, and that was the only thing that was based on timing, so everything else was really “what was happening”… but I could have programmed all of it.

[JC] But I think in the end, it’s also simply a personal choice. Marco could have used muscle tension from the inside of his thighs and just stood there staring at the audience and performed using the different tension [that could be created by using those muscles], but he chose to have [the microphone sensors] on his arms. [As with many of the performers who have used sensors like those Marco is using — e.g. the BioMuse.]

[CR] Because there are more values and you can control them better with the arms.

[PL] I think it’s two things at the same time. [The arm is clearly a highly trained muscle region that can be easily captured and controlled] but I think it is also for the audience. Because it is so visible, it is our arms — the big limbs of our daily tasks [the ones we use daily to gesture].

[CR] And Marco is Italian! [general laughter]

[PK] This does now make me want to try something with my toes!

[PL] That is invisible! [laughs]

[JC] I saw a dance piece once, the performer was a very well built guy, probably did some mild form of body building, an extremely fit dancer. He came and placed a chair [on the stage] while facing the audience, sat backwards on the chair and took his shirt off. The entire performance was just the muscles, it was like a choreography of all the muscles on his back. He had learned to control all of these different muscles in different ways with the different positions of the shoulders. Maybe it is that the arms are more immediately accessible. Whether you are a performer or not, you can immediately [get results]. For example, the people who have tried the installation 4[4. Marco had set up an installation at LEAP that ran during the entire weekend and gave visitors an opportunity to try out his Xth Sense instrument.], you tense your fingers and immediately you have a response. You don’t have to train for that.

[CR] In my butoh piece, we tried many muscles, but the best was really here [indicates upper arm] and the calves. Here you can control it very accurately, because of the fingers.

[JC] Is there a much more complex muscle system here?

[MD] I guess it’s the best position to start experimenting with such instruments. And also there is an obsession with a lack of touch. While playing in the air it’s easier (at least for me) if I still have to use my arm to do something: I can deal with it better and produce better gestures, that can be meaningful [to the audience]. I don’t think it’s about the complexity of the muscle specific to the arm. I think, again, it’s about the ability and the skill to actually manipulate it better, because we are just more used to it.

[CR] In my experience, for example, in my piece for butoh dancers and muscle sensors, if you have the sensors here on the arms, you can move the fingers and have a lot of values there. On the legs it is more difficult to control, it is just pure tension; on the arms it is more accurate to play with.

[MD] It depends on the body…

[PL] It’s a question of dexterity.

[CR] It’s more subtle, you can find more values [on the arm] using the fingers than on the thighs, for example.

[PK] Did you use your toes? What about your toes?

[JC] Can you move the toe beside your pinky toe? By itself?

[PK] Yeah… well, sort of. Not as well.

[PL] No, they tend to move like a group, at least mine.

[MD] Working with dancers I realized that they can move and display muscles that I didn’t even know existed. So they prefer to have the sensors on their legs. I was surprised at that but it actually works better for them. I don’t know why, but I can guess that just because of all the movements they do, there is all the weight — both literal and metaphorical — on the legs. They have to move dynamically around the space, whereas a musician generally just stays static in one place. The dancers could do many different things with the legs; I think it’s really just about getting used to it.

[CR] That was my experience with Butoh dancers, we can get more interesting values with the arms than with the legs.

[MD] I don’t know about you, but I don’t really do much with my body. We don’t really know much about our bodies, that is why I always think about dancers, also mimes. Actually, at the place where I am staying, there is a girl who is studying mime, and what is so cool about mimes is that not only do they have to produce a really specific tension in their body to visualize some “invisible” thing, but also they have a really tight rhythm in what they do, because there is a whole narrative which is actually quite rhythmic. Because they are not speaking, they don’t make any sound, they don’t have any object, so it’s all about moving, tensing their bodies and creating this kind of rhythm. I’m sure there are thousands of other ways to use the body that we as musicians have no idea about. That is why I try always to work with different people, and see what comes out of it.

[PL] When you played the second piece [on Friday], the one you created specifically for this event, there was a big difference from Music for Flesh II, which I have already seen you perform twice before. That piece was a bit more of a visual performance, there was some tension between your two hands, like you were holding an invisible object that you didn’t want to touch. It’s very nice that you present Xth Sense as a biophysical instrument in your performance, but there is also a big gestural side [to your performances]. How do you balance this? [Peter and Claudia] play in a more static manner, but you are clearly more physical.

[MD] Yes, I do that on one hand because the goal of the Xth Sense is actually to outline and open up some inner capabilities of the body. It’s not about interfacing the body with something else, a situation in which the attention of the audience is inevitably focussed on that “something else” and the body is just the medium by which we manipulate it. Instead, the goal of the Xth Sense is rather the opposite, focussing the attention of the audience on the body. That said, I don’t know my body, so what I am trying to do is just a natural evolution in my practice with the instrument itself. I don’t think I could have ever done a piece like that second one at the beginning [of my work with the Xth Sense], because I had no idea about many things. Firstly, what does it really mean to create a gestural narrative for an audience — something that is meaningful, not just waving some Buchlas around or something like that? Secondly, I also didn’t know how the instrument would react. That piece is based on an idea that I got from a sculpture by Giacometti called Holding the Void. Basically, there is this really scary creature, really thin, just “holding the void” [mimics the sculpture “holding the void” between his hands]. Thinking about that I got this idea of creating some kind of incarnated sound sculpture. But that came to my mind just after I realized how powerful a specific gesture can be, if it is well-executed, well-designed. So yeah, it’s a very different approach but still focussing on what the body can do and not what the instrument can do. My question is, as always, whether the Xth Sense would be the same without the body… probably not! So what is really the instrument? I keep asking myself this — not in public, because that’s risky! [general laughter]

[JC] A really banal question, but while you were learning this [instrument], did you have to sit down over the course of a month or whatever and try to find out what each muscle does? There was probably a fairly long learning process involved in this, in conjunction with developing the software that read the data and transformed it and played it back?

[MD] Yes, definitely. Literally, just sitting there and moving fingers and listening to the sound that would come out. Although, I have to say this was only in the beginning, because now that I have gotten more familiar with it, it is a more intuitive process. There are some kinds of gestures in Music for Flesh II that were not there in the beginning, they just came out while performing. And actually it was quite interesting at the Pure Data Convention in Weimar 5[5. 4th International Pure Data Convention (Weimar–Berlin, 8–14 August 2011). Marco, Peter and Pedro all use Pure Data for their programming.], that was probably the best performance I ever did; usually there is something about my performances I don’t like and that is probably the only one in about 30 or so that I did so far [that I was satisfied with]. What was interesting is that — I realized afterwards that due to the excitement of being there and having to show my work to my community for the first time, with all this emotional and psychological pressure to do it right — there, for the first time, I started also moving the rest of my body. And while doing that, in real time, I realized that new sounds were coming out, sounds that had never appeared before in the same piece, using the same programme.

[JC] So the rest of your body had an effect on the stuff you were used to being able to control, and changed the piece?

[MD] Yeah, because in the earlier performances I was just standing, pretty static, not moving my leg, not moving my torso, not bending over… just waving my arms. And there at the PD convention, I don’t even know why, at some point I just found myself like that [bends over, upper body against thighs] and there was this amazing “swooosh” sound that came out by itself. Afterwards, I tried to reproduce the effect, working again and training and I realized it was not due to the movement of bending down, but to a really, really strong tension at the onset of the movement — we are talking about a gesture of like 30 seconds — which would excite the software in a certain way. And then releasing it immediately let all the control values just fall down and then as I bent down there was a natural “bounce”, and that would somehow cause this really interesting feedback. Now, without getting into the technical details, that was pretty amazing for me because I realized then that this thing can do things I have no idea about and it just depends on how I move. So from there, as you see, I started to go the other way…

[JC] Throwing your arms all around.

[MD] Yeah, and from this kind of observation I got the idea of working on a kind of a sculpture, kind of a space. This is another topic, but we tend to forget that we work with the body, but the body is [situated] within a space. So we always have to deal with space, too.

[PK] But you can’t process the space, at least in your system, but you could add something that does.

[MD] Yeah.

[PK] But people can see you, I was going to say, when we talked about things being “invisible”. But it’s quite nice to watch you, to see this movement. The spatial [aspect] for you is more about what people see in the performance they watch.

[MD] Yeah, in this sense, maybe it’s about presence, if you want.

[JC] Claudia, maybe you could also talk about the learning process that you had to do, because it’s a fundamentally different instrument [than Marco’s].

[CR] With the EEG I cannot move, because the interface is so sensitive that if you move you get values [noise] from other sources than my brain [such as body movement]. 6[6. For more on the dynamics of Claudia’ system, see her article “EEG Data in Interactive Art,” published in the online proceedings of ISEA 2011 (Istanbul, Turkey: 14–21 September 2011).] Yes, I had a process and I am still learning about it: how I can go really deep in meditation or which kinds of things should I think about. For example, last year, I performed Music for Solo Performer by Lucier, and you must not [as a performer] really do anything, you just open and close your eyes. I learned a lot of things [with this piece]: by just opening and closing the eyes I can activate the alpha waves, or if I read something I activate something else… There are many things to learn, like in which position do I put the electrodes…

[JC] There are something like 28 [EEG electrode placement] positions or so, do you change that?

[CR] There are a lot, but I use only two channels, the frontal and occipital.

[JC] Is that because of the different kinds of [brain] activities that you want to explore?

[CR] In this case I have only two channels and I can have more control with the frontal. I can use the frontal for closing and opening my eyes, for example. I connected the sound with the frontal and the occipital is connected to the video. It’s difficult for me to really control the occipital, I know that. For example, during the performance on Friday, I could go into a deep meditation with the sound [frontal] but not with the occipital part of my brain. If I am very, very relaxed, I could have generated blue colour, but I know on Friday I only arrived at white; I was not in a really deep relaxed state.

[PL] It is like what Peter was saying, you get there and the range [of the biodata] changes.

[CR] Yes, I couldn’t get to that point on Friday. Sometimes, it depends on the situation. Normally, I prefer to be at the start [of the concert programme] and to be there half an hour before people arrive and not do anything before performing. But the setup was different here. And this is something that I really like, it’s never the same, it depends on the space, it depends on the audience, it depends on the day you have!

[JC] That’s a direct outgrowth of your emotional / physical state, which an audience doesn’t expect to see when they go to listen to a string quartet or a Beethoven Sonata. They are listening to the piece! The performer is really important, but in the end, it is really first and foremost about a specific piece: the notes and harmony don’t change over different performances…

[PK] The notes don’t change…

[JC] The piece itself doesn’t change. But the interpretation of the piece of course changes a lot.

[CR] Some friends of mine were here on Friday and they had seen the piece in Dortmund last year, and they told me it was a totally different performance. And yeah, it was another situation, it’s as simple as that. It is really great that it’s not the same.

[JC] But there are certain aspects of both your performances that are pre-planned to an extent. First of all there is the mapping which is pre-determinant to a certain extent. And Marco specifically used the term composition in talking about the second piece that he presented the other day, that he had “composed a new piece for the Berlin people.” Marco, you have said that you have performed with the Xth Sense many times, for Music for Flesh II. How much is predetermined in your pieces and how much is — let’s say, without using the word too lightly — improvised?

[CR] In my case, I have a structure for the piece, because it’s a composition, in fact. I wanted to go at this point from meditation to trance state. But there are some points that are totally free, it depends on the state I am in at that moment. How I get to the trance state depends [on the performance], that is open, but there is a structure.

[JC] So the dramaturgy is fixed, more or less…

[CR] Exactly, at some points I know I want to go to the meditation or to the trance state, other [aspects] are open. But the structure is [fixed].

[MD] Well in my case, what I compose is more the context of activity I want to create with the system. Which means I literally create some scenes in which the sound forms can vary with a really, really big range of different nuances, dynamics, amplitude and everything. What I compose really is maybe the order of these scenes. But then what I do within a scene depends on how I want to perform on that day. This is because I still want to let myself be free to improvise, somehow; it’s not literal improvisation. For instance, I played that piece at a noise=noise evening in London, which are “noise nights”, and Music for Flesh is usually not that “noisy” but if I want to make it noisy, I can. In that case, I was playing it harsh, really pushing it along. At other conferences, for example, you are in more of an academic environment, so you just try to make it look more like a “composition” or something… [general laughter]

And then, there are exceptions, for instance, when I played at NIME in May 7[7. NIME 2012 — 12th Conference on New Interfaces for Musical Expression (Ann Arbor MI, USA, 21–23 May 2012).], well, it was NIME, that’s the conference about new musical instruments… But I was playing in a club and they had these subwoofers that were two meters tall, this massive PA, in the NIME conference! They didn’t want to make me do a sound check because there was too much bass and the offices were complaining. So I just arrived [for the performance] and played it really loud, and it was great. This is what I like about the instrument also, it gives you this degree of freedom.

At the same time, I also like to compose, so I try to find a balance. Although, I would like to find a way — which I haven’t found yet — to really improvise. I don’t know… some kind of way to create mappings in real time, without me really designing how to create that mapping.

[PK] You mean to re-map, in real time?

[MD] Yeah, dynamic mapping, according to the muscular energy or something… this is something I’ve been working on. Also when using the timeline you have time going from A to B in a linear order, and what I am working now is moving into the timeline in a non-linear way and not based on the time, but actually based on my effort. Say, if I push it very hard for two minutes, then I will find myself at point C; and if I get quiet for one minute, I get back to point A, or something like that. To make things more dynamic. Its kind of a leap into the void, but I think it could be interesting.

[PL] It’s kind of like composing with the body, that surely is [creating the structure with the body].

[MD] Theoretically…

[PK] I was thinking about this, as well, in my piece. The one change that happens is time; it almost felt, I could almost imagine I was bringing it about, so I could lie to myself about that. I need some more specific feedback in order to do this, like a number, just so that I can actually work out when it will happen, but I wanted to summon this one change…

[PL] You mean that you would like to have a specific feedback so you could observe yourself?

[PK] Well, the sound is good. It gets you out of this conscious mental state and into something intuitive. But to be able to move from one part of the piece to the other, would be nice to have a way of summoning the second half of the piece.

[PL] That is a important question for the three of you: Marco, you perform with your back to the computer, there is no way you can visualize if the levels are “ok” and if you are going to trigger the next part of the piece; Claudia, you are completely inside the thinking process and the projection is on top of you, and you, Peter, had only one pre-determined change and nothing else. So, how do you guys envision a new type of feedback?

[PK] All my system is feedback, so everything that is happening is giving me aural and visual feedback.

[JC] I had the impression in Peter’s piece that there is only so much that you can actually control and pre-plan, but it did come across as being very consistent throughout and having its own contour, and it did seem “closed”, it did feel like this [the duration of the performance] was the time that it needed. And as [BC#4 Curator] João said (I think quoting somebody else?) those kinds of pieces have to last 15 minutes or 4 hours, because of the conceptual underpinning of the piece. But it did feel closed for me, in a “composed” sense, even if it wasn’t fully composed.

[PK] Yeah, that was the intention, and the shape, internally, was me. But I did kind of compose with my excitement level, I tried to use that to give it extra shape. But there is one cue that is counting milliseconds, it would be nice, instead of having it count milliseconds to somehow have to trigger it myself.

[PL] Triggered it emotionally?

[PK] Yes. Sort of summon the move from the first section to the second. On the other hand it’s the one time where the piece says something back to me, when I’m waiting for that to happen.

[PL] Closing up, a bit of a futuristic question for everyone. There is a lot of availability [of technologies] today. There are very good GSR sensors that you just plug into your iPhone. Even the EEG [have gotten much cheaper] in the last five years, for $50 you can have one. And Marco’s system is very poetic in one sense, because a lot of people, including many scientists, thought that microphones were dead, replaced by accelerometers (as a sensor technology, not as a recording device!). You offer proof that they aren’t, and they are very cheap, so they have great availability. You publish [the Xth Sense] as open source, so people can build one at home. So in terms of availability, what do you think is going to happen with biomusic? Can we expect — like [they do of themselves playing] guitar, is everybody going to start playing bio-instruments and post YouTube videos of these things?

[MD] That is hard to say… I don’t think so. That is because before that happens we really need to change our relationship with technology and technological innovation — which is happening, at a good pace too. But it is always at risk of falling back to where it started. There is the DIY community that has become something standard now [in comparison with] a few years ago. But now as it becomes standard, it is just getting into a really interesting crossover with corporate business. And corporate business is actually learning that they don’t have to fight DIY anymore and they actually have to compete to get to the same level, because DIY stuff is getting better than what the corporate [businesses] can do.

[PK] Are you thinking of any specific example?

[MD] In general, I mean look at Arduino 8[8. Open source platform developed to serve the creative artistic community.], the most obvious example we can think of. No company was able to do something like Arduino. It came out of an academic environment, but it was one of the first open source projects [adopted worldwide].

[PK] Although you could say — just to play devil’s advocate — there would be no Arduino without Atmel. Although the tools that Atmel created were kind of unusable… So there was some kind of symbiosis that happened there. The company making all the hardware was not doing the best job in making the tools. Obviously the open source community had level of intuition about what people actually wanted, that the company who should have [understood], missed.

[PL] I think that in the sensor world that plays a big role. Marco, you released Xth Sense as open source because (I think) you understand that, in the field of biophysical music and biosensors, it will get to people faster through these channels than if you talk to some patent guys and try to make a patent to then sell it.

[MD] Honestly, I didn’t really think about that. I just looked around and everything was closed and I thought to open this up for people, because I work with free tools…

[PK] You are saying that for this revolution to happen, for everybody to start doing this, having that open relationship with the hardware is a prerequisite for people to really begin experimenting with biodata?

[MD] No, the [hardware] revolution already happened. [general laughter]

[JC] We’re having tea now…

[MD] We all talk about the future all the time, but the future is past! We have to think about what we are doing now, which is more interesting. The revolution already happened, and it was not only because of Arduino, it was not only because of Atmel, but it was the two principles together. What I think is that both the corporate business and the open source community have contributed to actually getting the people out there closer to technology. And that was the past. So what is going to happen now is a big question mark, because it doesn’t depend on the creators anymore, it actually depends on the people now, because the people have access to the technology, the people can have knowledge.

[PL] Are you talking about spectators, or also active participants?

[MD] Well, we are all active participants, I mean in the moment you take your tablet out, you are an active technologist at the same time, even if you don’t think so or do anything with it. But you have a really powerful tool in your hand and if some friends of yours come up with a nice app you will start to make music in the street.

[PK] I can see your controller becoming more popular. It seems like we are going to see a lot of biodata collection, somehow. Whether people use that for art is more obscure… I don’t know if that many people necessarily want to do that, to be honest. But I could see people using your interface because we know that amongst the music community there is a strong interest in new gestural controllers, and that’s what drives people to keep coming back to NIME and asking the same questions year after year. And I could see it becoming popular especially if the software becomes more accessible to people. The fact that the Kinect has taken off — sometimes with tools that are not very easy to use at all, sometimes unreliable — suggests to me that the Xth Sense could take off. It’s a lot cheaper. You lose some of the spatial interaction but you gain this physical interaction and you get levels of control that are really missing from things like the Kinect applications…

[PL] You also gain [some levels of control that the Kinect does not have, such as muscle tension].

[PK] Yeah, you gain a lot. Kinect is very good spatially but it has a lot of latency (in the way that most people use it) and it is very disconnected from the way that you actually move your body which frustrates people once they start playing with it. So the fact that you are offering something that has a greater degree of physical control, that is also much cheaper and runs more reliably on all three operating systems, suggests that it could be quite popular. Maybe not in a mass market sense, but we could see other people picking it up and “run with it”.

[MD] Yes. The really last thing about if bio-stuff will go on or not. Another reason I released the Xth Sense as an open source project is because, as Peter said, biometric data can be mined from our body, and this is just becoming incredibly common in advertisement and security. The Department of Homeland Security has programmes already that run on the iPhone and can learn the temperature of your body, how tall you are, how much you weigh, what your heart rate is, and all this is done from a telephone at a distance, without you knowing. They are testing this since 2008. And in 2008, the Japanese company NEC was already using digital walls that would recognize age and face, etc., to serve you individually targeted ads; we are talking about four years ago.

I release this for free, because there is a need to demystify the black box of bio-technology. Secondly, we need to understand what this biological data is, so we can actually do something with it. We can also better understand what others are getting from our bodies. These are the main reasons. So, if the Xth Sense will get used more, this could possibly be one of the outcomes outside of the musical application, and I really hope so. I also hope that other developments will come out of the Xth Sense, maybe completely different, but with the same mindset.

[PL] Thanks everybody!