Bodily Listening

by Satoshi Morita

This paper was presented at the Toronto Electroacoustic Symposium 2010, during the Tangible Music session, chaired by David Eagle.

In 2006 I was traveling in Switzerland. One day I built a tent at a lakeside and during the night it started to rain, when I was lying in a sleeping bag in the dark. Tiny raindrops were hitting the roof of the tent softly. I could also hear the sound of rain dropping on leaves of trees around the tent. I also felt like I was able to listen to the sound of raindrops falling onto the surface of the lake. Although I didn’t even touch or see the raindrops, the feel of the tiny drops was intimately tangible. The membrane of the roof of the tent was synchronized in my imagination with my feel of skin and became a sort of second skin. It was warm and comfortable in the sleeping bag. I was physically divided from the environment, however my awareness of the bodily sensation was expanded to the outside of the sleeping bag and into the environment around me. The liquidity of the raindrops and the tension of the membrane were vividly tangible. The only trigger for this sensation was sound. It was a highly intimate bodily experience of the sound. From this inspiring experience I developed the piece Sound Capsule (Fig. 1).

When lying in the capsule, the visitor can listen to the clear sound field from the loudspeakers beside both ears. Vibrotactile stimuli can be perceived at different locations of the body through six vibrotactile transducers.

Project Description

Object Design

The basic motif for the shape is a person lying on his back. Design of the shape must also reflect the consideration of actual contact with the body for the inner space, and the sound field is designed around one’s head for clear spatial perception. Aesthetic concerns from a sculptural point of view were a strong influence on the final decision of the shape of the work, which also fills the functional requirement.

Technical

Vibrotactile stimulation is transmitted through the areas where the body and the object come into contact, for instance the back of the head, the back and heels. I installed six transducers in the capsule: one under the head, two on the back, two in the area where both palms are and one under heel (Fig. 2).

The vibrotactile transducer manufactured by Acouve Laboratory has a wide frequency response, from 16 Hz to 15 kHz. The response can differ depending on the material on which the transducer is mounted. For Sound Capsule, hard polyurethane foam (density: 120 kg/m3) 20 mm thick is applied to support the transducers (Figs. 3 and 4). Softer polyurethane foam 10 mm thick covers on top. The idea with this structure is to diffuse the vibrotactile stimuli around each transducer and provide comfort. Visitors between 165–180 cm tall are well-suited for the capsule regarding the distribution of vibrations from the heels to the head.

Two coaxial loudspeakers are installed beside both ears (Fig. 3). Due to the positioning of the sound source, it creates an interesting stereo effect, which is different from a stereo field heard through either headphones or a frontal two-loudspeaker system.

Sound Recording and Composition

The focus of the sound material is based on sound of bodily action, which is generated from the inner and outer space of my own body. In order to record the sound around and from inside of my body, I put eight miniature microphones (DPA 4060 omnidirectional microphone) in eight locations: the back of my head, on my chin, on my chest with a stethoscope in order to record my heartbeats, on my abdomen, on my left and right wrists, and on my left and right heels. Each microphone works as a sort of auditory magnifying glass which focuses the environment and my body from different listening perspectives. This simultaneous eight-channel recording provides the possibility to build a unique listening perspective: by mixing the desired channels the imaginary space crosses between the inner and outer spaces of the visitor’s body (Audio 1).

These recordings are also sources for the vibrotactile stimuli. The six-channel sound material for six transducers was composed regarding the topology of the different parts of body. The audio signal from each single channel is amplified and transmitted to the transducers. The whole sound material with eight-channel sound material is operated by DAW software (Fig. 5).

Feedback

I got very different responses from audience about the experience. Many of them got strong reaction by the tactile stimulation. For some audience auditory sound got more attention than the vibrotactile stimulation. In each case, the balance of focus between the each sensory modality was diverse in each individual. There were some people, who had difficulties to stay in the capsule for the whole length of the sound collage, as they recognized strong de-synchronization between the rhythm of their heartbeats and the pulse, which appears at the end of the sound material. One of them needed nearly 10 minutes to recover from the experience and return to the “normal” perception in reality. When I talked with her afterwards, she told me that she is in general really sensitive and aware of her bodily perception. The audio-tactile experience with direct association to physical activity from “someone else” was too overwhelming for her.

After experiencing the work myself and getting response from the audience, my awareness about the audio-tactile perception was raised even more and started to research about haptic, which was rather unconsciously my interest since years. In order to get basic knowledge, I have been reading literatures about haptic, tactile perception etc. The following chapter describes a quick overview about tactile perception, which is relevant for my artistic approach.

Haptic, Tactile and Kinaesthetic Perception

There is a basic definition about haptic and related terms by Jack Loomis and Susan Lederman from a functional point of view. The tactile sense, which can also be called as cutaneous sense, provides an awareness of the stimulation, which is provided through skin. The kinæsthetic sense provides an awareness of the inner space of our body through neuromuscular feedback. The information of haptic perception is based on the both tactile and kinæsthetic perception (Loomis and Lederman 1986).

Basic elements of tactile perception are touch, pressure, vibration, temperature and pain. The first three sensations — touch, pressure and vibration — are the interest of my work for bodily listening. These stimuli cause mechanical deformations of the skin area and are perceived by mechanoreceptors. The mechanoreceptors are located in different layers of skin. The feel of pressure and the vibrations are difficult to distinguish from each other. From a sensorial point of view, both stimuli sometimes overlap, depending on the intensity level.

Pressure and vibratory stimulation are sensed by four mechano-receptive units in the superficial skin and deeper tissue area. The units are categorized by their adapting speed and receptive field:

• FA: fast adapting and no response to sustained stimulation;
• SA: slowly adapting and responding to sustained stimulation;
• I: small receptive field;
• II: larger and diffuse receptive field.

FA I with Meissner corpuscles, FA II with Pacinian corpuscles. SA I with Merkel cells and SA II with Ruffini endings. Each of these units corresponds also to four physiological channels:

• NP I (non-Pacinian) channel with FA I;
• NP II channel with SA II;
• NP III with SA I; and
• P (Pacinian) channel with FA II.

This four-channel model was established by Bolanowski as a result of the investigation of sensitivity threshold in frequency and intensity of vibratory stimulation (Bolanowski et al 1988). The threshold curve shows the combination of involvement of different channels 1[1. For a representation of the four-channel model of sensitivity to vibrotactile stimuli based on research by Bolanowski et al, see E. Bruce Goldstein (Ed.), Blackwell Handbook of Sensation and Perception (Oxfors: Blackwell Publishing, 2005) p. 553.]:

• 0.4–3.0 Hz without change (NP III channel);
• 3–40 Hz with gradual decrease of threshold (NP I channel);
• 40–500 Hz U-shaped curve with maximal value around 250 Hz (P channel).

As Weinstein pointed out, the sensitivity of tactile perception on our body differs depending on the location (Weinstein 1968). He investigated the threshold of tactile perception with various parameters and measured, for instance, pressure discrimination threshold on the whole body both male and female. 2[2. See Weinstein 1968, for diverse tactile sensitivity thresholds for men and women.]

Wilska examined the sensitivity of vibrotactile perception using 200 Hz vibrotactile stimuli and found that hands and soles of the feet are the most sensitive. Head, larynx region and abdomen are also sensitive (Wilska 1954).

Temperature also influences the sensitivity of the vibrotactile stimuli. When the skin temperature is between 25 and 40° C, no change under 40 Hz is recognized and the U-shaped curve is shifted to higher frequencies with a maximum value at 40° C of 400 Hz. The maximum value with the U-shaped curve between 15 and 20° C was 200 Hz. This is the result of different involvement of the Pacinian corpuscle, which is located in deeper tissue, and non-Pacinian channel. When the skin temperature is higher, the involvement of Pacinian corpuscle is more notable. With lower temperature, non-Pacinian channels pull the U-shaped threshold curve to the lower frequency (Uchikawa et al 2008).

Outlook

These studies encouraged me to develop the idea of audio tactile work more precisely and more radically.

Other studies about the interaction between multiple modalities also show some hints for the integration of the auditory and tactile perception. Physical, physiological, neural and cognitive science etc. provide clear guidelines for the consideration of this approach indeed. I see the development of the sonic object for bodily listening as a platform for a new way of perceiving sound, which involves corporeal awareness. Two developed versions of Sound Capsule are planned.

Sound Capsule 2 is the direct improvement of the first version of Sound Capsule introduced above. This version of Sound Capsule tackles the issue of the diverse vibrotactile sensitivity around the body. More precise localization of the source of the vibrotactile cue as well as wider diffusion of the tactile sound are intended, in order to achieve free and specific mediation of tactile stimulation with variation. For instance, a combination of both vibrotactile transducer and subwoofer would create both pointy and diffused tactile sonic fields, due to the characteristics of each sonic wave mediation.

Sound Capsule 3 is filled with water. Most of the body will be in the water and an audience feels the sound through water. There will be external loudspeakers around one’s head, similar to the first Sound Capsule. The characteristic of sound mediation and diffusion in the water is a highly interesting issue for tactile sound experience. Further research and experiment about the mediation of vibrotactile information in the water and its response on the human skin surface is required.

For the sound material, the project is open to collaborations with electroacoustic musicians and composers. This would also be a great opportunity to expand the quality of the audio-tactile sound experience with diverse directions of compositions.

In addition, the bodily experience in Sound Capsule often evokes linkage to the user’s physical and imaginary memory, which might be stored not only in brain but also in the skin. The ability to have memory in / on skin itself and to become aware of the memory of the skin is an interesting issue.

Bibliography

Bolanowski, Stanley J., George A. Gescheider, Ronald T. Verrillo and Christine M. Chechosky. “Four Channels Mediate the Mechanical Aspects of Touch.” Journal of the Acoustical Society of America 84/5 (1988), pp. 1680–94.

Gescheider, George A., Stanley J. Bolanowski, J.V. Pope and Ronald T. Verrillo. “A Four-Channel Analysis of the Tactile Sensitivity of the Fingertip: Frequency Selectivity, Spatial Summation and Temporal Summation.” Somatosensory and Motor Research 19 (2002), pp. 114–124.

Loomis, Jack M. and Susan J. Lederman. “Tactual Perception.” Handbook of Perception and Human Performance, Vol. II. Edited by Kenneth R. Boff, Lloyd Kaufman and James P. Thomas. New York: Wiley, 1986.

Shimizu, Yutaka and Masami Shinohara. “Hearing, Touch, Equilibrioception.” In Science of Perception 3. Edited by Keiji Uchikawa. 2008.

Weinstein, S. “Intensive and Extensive Aspects of Tactile Sensitivity as a Function of Body Part, Sex, and Laterality.” The Skin Senses (pp.195–222). Edited by Dan R. Kenshalo. Springfield IL: Charles C. Thomas, 1968.

Wilska, A. “On the Vibrational Sensitivity in Different Regions of the Body Surface. Acta Physiologica Scandinavica 31 (1954), pp. 285–289.