A Composition of the “Things Themselves”

Visual music in practice

by Maura McDonnell

This article is based on a conference paper originally presented at Sounding Out 5 (8–10 September 2010), Bournemouth University (UK). it has been slightly revised and expanded for publication in eContact! 15.4.

Visual Music and Other Terminologies

Current use of the term “visual music” has arisen mainly from recent exhibitions and events, which present contemporary visual music works and, in some cases, document these exhibitions with a catalogue or book (Brougher et al. 2005; Ox et al. 2006; McDonnell 2007; Rainer et al. 2009). Individual artists, academics and composers who have been working closely with sound and image material in their own practice have been doing so, nonetheless, for quite some time, and indeed for quite some time before some of these exhibitions have taken place. An earlier discussion arose amongst practitioners in the audio-visual field about the definition of the term on the iotaCenter Discussion Group list (Ross 1999). Earlier again in 1986, William Moritz uses the term “visual music” in his research and writings. Almost two decades later, he documented the start of a history of visual music where he recognised — in the area of historical absolute film or abstract cinema, and in particular his analysis of the filmmaker Oskar Fischinger — that there was a particular æsthetic of visual music arising in these works that provides the beginnings of principles of æsthetics for new artists setting out to compose visual music (Moritz 2004). Roger Fry first used the term in 1912 to describe the particular “closely knit structure”, the colour oppositions of the abstract paintings of Wassily Kandinsky (Spalding 1980, 168). Some contemporary artists / filmmakers / composers, who would describe their main body of audio-visual work and approach with different terms — for example, animation cinema (Hébert 2010), audiovisual animation, audiovisual composition, cross-media, electronic cinema, electroacoustic movies (Cope), live multimedia, intermedia performance, intermedia composition (Kapuscinski et al. 2009), object animation and motion painting (Maxwell), videomusic (Piché), audio-visual artworks (Battey), visualisation of music as an abstract phenomenon (Ox) — would consider some of their practice and works to be visual music works, and certainly agree to having them described as such, even if they do not deploy the term “visual music” to describe all of their works. 1[1. For definitions of visual music see Ox et al. (2006), McDonnell (2007) and Lund et al. (2008).]

Defining Visual Music — Identifying Visual Music Practice

A survey of recent visual music exhibitions and symposiums indicates that the practice of visual music is growing, for instance, the Visual Music Marathons held in Boston (2007) and New York (2009), the Punto y Raya Festivals held in Madrid (2007) and Barcelona (2009), the See This Sound exhibition held in Austria (2009) and Seeing Sound, a practice-led visual music research symposium held bi-annually in the UK since 2009, not to mention many others. 2[2. For useful historical discussions of precursors in the field see Elder (2008), Keefer (2005) and Fred Collopy’s website <rhythmiclight.com>.] As the practice of visual music is growing, so too is research and writing on the field (Computer Music Journal 2005; Daniels et al. 2009; Lund et al. 2008). It is hard, nevertheless, to pin down and to agree precisely what visual music is, as some practitioners question the term, while others debate its definition. This is due to a number of factors, such as, for instance:

• the technology employed to create the work;
• the background the artists / composers / practitioners come from, and whether it is (primarily) a music composition, filmmaking, vj-ing, visual art, computer engineering, computer programming background;
• the method(s) used for its presentation to audiences (e.g., fixed media video and sound screening, performance of musicians and video, installation, live improvisation of visuals and music, real-time interactive rendering of images and sound…).

Some practitioners also focus on the sensory aspect of combining visuals and audio, where the focus of the work is on its final appeal to the senses. This, in a way, adds some confusion in relation to defining visual music.

All these discussions, however diverse they may be, do help, nonetheless, to define and to understand a phenomenon of art and music activity that is actually being created and taking place and that is an active area of practice. And as more artists and composers who create “visual music” works document their approaches and thoughts on the area, the more this will contribute to the developing area of visual music practice. One example of such formalised research is taking place at the CCRMA / Intermedia Performance Lab at Stanford University led by Jaroslaw Kapuscinski, where detailed studies investigate the intermedia areas of exploring the relationship between visual art and music (Fig. 1). Two organisations based in the US, the iotaCenter and the Center for Visual Music, promote both historical and contemporary practice in the visual music field and are forums for the gathering of resources, curating exhibitions and events, promoting contemporary practice, preserving historical films, and building a community of all those interested in the field of visual music and “experimentation, abstraction, and animation” (iotaCenter).

Jack Ox and Cindy Keefer provide a useful, broad definition of contemporary visual music, defining visual music practice as being characterised into four differently formed visual structures. This definition arose from their curating of New York Digital Salon’s Abstract Visual Music Project. In summary, they distinguish one visual structure to be “a visualization of music which is the translation of a specific musical composition (or sound) into a visual language”; the second is a “time-based narrative visual structure that is similar to the structure of a kind or style of music”; the third is defined as a “direct translation of image to sound or music”; and the fourth and final visual structure is defined as a “visual composition that is not done in a linear, time-based manner, but rather something more static like a 7' x 8' canvas” (Ox et al. 2005). This is a good starting point for any enquiry into visual music.

Visual Energy

For the remainder of this paper, the discussion in relation to the things themselves in a visual music work will concentrate on the “visual things”. The visuals in the final product — regardless of how and in what way they were composed, that is to say, whether they were composed before, after or alongside the music — seem to be directed towards the music; music is the impetus and the ground that drives the energy of the visual. The main energy of a visual music piece is this visual energy. The visual energy of a visual music piece bursts out of the screening frame — as an audience we are no longer bound to a chronological rendition of frame images. The visual energy is then to be understood as the things themselves in the image, each of these individual things having their own trajectory of appearance and disappearance, their own dynamics, their own rhythm, their own life span and momentum, and their own manner of being directed into the depth of the frame. There can be multiple things made visible to us, and each of the things contributes to the overall visual effect as they change and evolve over time, interact with each other and interplay with the elements of music in the soundtrack. As in absolute film, the imagery is objective. The visuals are objects; they are things that reference themselves — the things themselves of the image.

Visual Things

What, exactly, are these visual things with which one composes in a visual music work? The visual things can be simple elements of a picture drawing and graphic design, or they can be more complex figures crafted from the simple elements of picture and graphic. They can also be more painterly, focusing on filling the screen space with a multitude of colours and textures. Digital technologies offer the user a precise control of computer-generated image parameters; these parameters become new visual things that the artist can access and use in the creation of images that can consist of precise patterns, ordered shapes or images consisting of a multitude of dots and lines, forever changing and transforming across time.

Examples of Pictorial Elements

Simple pictorial, drawing and graphic elements may comprise one or more of: straight lines; curved lines; dots; simple shapes; use of colour, drawing marks of all shapes, lengths and sizes; scribble and scratch marks; pictorial figures; geometric figures, flat and gradient blocks of colour. Historical works used a range of mechanical devices and techniques to record pictorial and graphic elements for their films, for example, Oskar Fischinger’s film Studies (1929–34), Hans Richter’s Prelude (1919) and Kurt Schwerdfteger’s Reflektorische Farblichtspiele (1922). More modern works use different technological means to realise their visuals, for example by drawing directly on film (Lye; McLaren), or using video and early computer graphics experiments for generating dots of light and graphical blocks and building these into geometric shapes and patterns (Whitney; Cuba).

Contemporary visual music works visualise similar pictorial image elements, and have access to both traditional techniques of image and filmmaking, as well as having access to a whole plethora of digital technologies to realise their image elements. Javier Sanchez’s video Unorthogonality (2009) uses simple lines to explore spatial and musical movement (Fig. 2). In a style similar to historical works of Len Lye and Norman McLaren, Oerd van Cuijlenborg uses direct film animation techniques in Jazzimation (1999) to align the visuals — strokes, squiggles and scribbles — to the musical elements in a jazz composition (Fig. 3). Through experiments with one-bit technology, Chiaki Watanabe generates computer images for 1/3 [one over three] vol. 1 (2006), the emphasis being on using a single bit of information for all levels of the creative process: one-bit colour, one-bit code and one-bit sound (Fig. 4). Neil O’Connor’s video The Glistening Bride (2004) uses simple, digitally created curved graphic blocks of colour to change the scale parameter of the blocks over time, in accordance with the dynamics and intonation of the music instruments in the piece (Fig. 5). In the visual music work The Night After I Kicked It (2008), Jane Cassidy uses a series of lines to create a minimalized spatialized image composition that interacts with the spatialized sounds in the music composition (Fig. 6).

Examples of Complex Figures

More complex visual figures and forms can be created with these pictorial and graphical building blocks, and these pictorial forms can be used to form new visual objects or figures that we come to know and recognise as they are transformed over time, as in Viking Eggeling’s film Symphonie Diagonale (1924). Eggeling creates visual forms from basic graphic elements and used the idea of musical counterpoint to elaborate their arrangement in time. He calls these forms “themes”, or “instruments”. Larry Cuba’s computer-animated film Calculated Movements uses computer programming to create solid areas and shape volumes (Fig. 7). The film consists of a “choreographed sequence of graphic events constructed from simple elements repeated and combined in a hierarchical structure.” The work has a precise mathematical structure in the design of the graphic events and their unfolding in time (Cuba 1985).

Pierre Hébert’s recent film Triptych (2009) was made from a series of live animation performances. Using lines he created three simple figurative forms, positioned as a triptych, that become more complex over the temporal evolution of the piece (Figs. 8a, 8b). Each of the figures consisted of several lines that are part of the visual form yet evolve independently from it. Each element had its own action and the various elements coalesced to form figures that then transformed and revealed motions and actions that were aligned to the dramaturgical flow of the music.

LIA’s video Construction 76 (2008) builds beautiful shapes and patterns with lines and simple graphic elements; the sounds in the video are synchronised with these graphic elements (Fig. 9). Stephanie Maxwell uses hand-painted experimental abstract animations in films such as All That Remains (Fig. 10) in order to create “an intricate mosaic of sequences of animated abstract images and musical passages that create a chaotic yet coherent and tightly choreographed portrayal of figurative matter in perceptual decomposition” (Maxwell 2006).

Examples of Painterly Approaches

A more painterly fine art approach can also be taken to creating the elements of image. The whole screen space becomes a ground and surface for full screen activity. The screen can, for example, be filled with washes and textures of colours, merged forms, a multitude of lines, a sparsity or density of image particles, that change, merge, coalesce, collide and interact like a large abstract painting canvas that has come to life over time and with the addition of music. The final result is one of a whole visualscape of imagery, a visual environment — and a visual odyssey in some instances — that is made incredibly powerful through the addition of time. Digital processing techniques have allowed us to bend and mould imagery to become the basis of the most complex visualscapes and create both 2D and 3D visual environments. Jean Detheux develops his digital abstract expressionism to create incredible imagery as it unfolds in time and interconnects in a multiple of ways to the music. The energy in his works is very tightly aligned to the music it follows, and his work provides incredible examples of visual music. His work Visual Abstract (2010) explores this energy with a multitude of changes and variations, which is similar to the multitude of changes and variations in the timbre of the music instrument or the melody or rhythmic pattern or the dramatic changes in dynamics in the music (Fig. 11).

Examples of Computer Image with Electroacoustic Music Composition

Technological advances have let us access microparameters of image, through a wide range of techniques from image processing and image manipulation, to generative images that are created from code and programming or from a mapping of data, to even audio-data-generating images. Such levels of parametric access provides us with the means to create image elements and manipulate these image elements into whatever form we wish — insofar as the software or programme is capable of meeting our demands. The following examples are works in which a single composer composes both the images and music. For his videomusic work Patah (2009), Diego Garro creates image particles to match the granular grains in his music composition; similar processes of transformation and speed are applied to both images and sound. His music composition is an electroacoustic composition, and his image composition has a similar feel, an electro-visualscape (Fig. 12).

Dennis Miller also creates complex visual forms as well as composing the electroacoustic music composition. His video work Echoing Spaces “explores a number of virtual environments” in which echoing, repeating lines and forms emerge (Fig. 13). “The visual imagery employs a number of similarly shaped elements that appear in overlapping, morphing configurations” (Miller 2009). Bret Battey’s computer-realised Sinus Aestrum (Fig. 14) is an incredible unfolding of “12,000 individual points, which are continually transformed and warped, restrained and released, without cuts, to form compound, multi-dimensional waves of activity” (Battey 2009).

Examples of Real Things as Visual Objects

The things of image refer, in many cases, to these more abstract marks and basic units of drawing and picture-making and painterly effects, but the things of image can be any visual object defined by the artist, for example, real-world photography, or moving image elements, or montages of images are used as a thing in the visual composition. The photograph or piece of live-action footage, nevertheless, is not being used to reference the content of the image; they become rather the building blocks of a composition that references the composition of image elements in time. For example, Jaroslaw Kapuscinski’s Juicy (2009), a performance / installation for Disklavier piano and computer-controlled image and sound created in collaboration with John Edmark, uses images of pieces of fruit as the image object (Fig. 15). With a close alignment to the music and inter-media interplay of images and sound, “sounds and images are given equal importance and are developed either simultaneously or in constant awareness of each other” (Kapuscinski 2001). Huckleberry Lain’s Parallel (2010) also uses real imagery and units of moving image in this work, mixed with a variety of image elements. Here, the live-action footage and real photographs are assembled on screen as a thing-in-itself and as a thing-in-the-composition (Fig. 16).

Composition

The compositional task for the author(s) of a visual music work involves making choices about both the visual and music things themselves as they are in each of their own domains and also about how the visual and music elements interplay and interconnect over time. An essential characteristic of a visual music piece is the unfolding of a temporal interaction between visuals and music. How this is structured over time is at the hands of the composer, filmmaker, animator or visual music artist. Music composition strategies work well in a visual music work, and many music composers who also create their own visuals have also applied their compositional æsthetic to their imagery. These composers can work with both visual and music material in their compositions. Cinematic, fine art and animation techniques and computer animation techniques, however, can also serve to bring other approaches to composing with visual and music things. Having such knowledge makes for a visual, and its temporal evolution over time with music, that is more like a moving art — fine art that has moved out of the constraints of the static frame and into the dynamic action of the temporal screen. Cinematic and animation techniques use their technology and knowledge of screen dynamics to explore the things of image themselves and to compose with these basic building blocks of image, re-contextualizing the image when the music composition is as important as the images on screen. The visual music composer — a particularly new breed of video artist who uses programmed and generated images and music, or one or the other, in their work — brings another æsthetic dimension to the creation of visual music works. The strategies of composition here involve making decisions about the parametric control of the visual and musical material, setting up computer procedures and applying mathematical formulæ to the creation of the building blocks of their material, and structuring the unfolding of the computer imagery over time. Regardless of the visual music compositional æsthetic at work, the core common element of the compositional task in composing a visual music piece is structuring time, and the core task is to make connections between visual and music material.

Visual music works are compositions. They may be presented in different formats and with different processes, but essentially they are compositions arising from deliberate æsthetic choices and intentions by the visual music artist working alone or in collaboration. Music composition provides a model for structuring time and for composing with the abstract basic building blocks of music making. Visual music works can re-work music composition techniques into the image component of a visual music composition, focusing on building compositions with the things themselves of image — applying choice of colour, screen direction and dynamics; working with simple elements to complex image elements, rendering these elements over time, transforming and changing parts, re-mixing older parts in changed form, and basically crafting the simple things of image over time. Cinematic, fine art and computer art have all a part to play in creating compositional approaches and strategies for a visual music work and in identifying the best principles for creating a visual music work. Visual music does not reference anything beyond itself. It exists as visuals and music in an interplay of simple things and complex forms, from both visual and music material. The increasing amount of contemporary practice in this area of visual and music interplay is fast unfolding and, arguably, an indication of the start of the journey towards a shared language of composing with visual and music material. By suggesting that one such approach is to see and hear a visual music work as a composition of “the things themselves”, we give due recognition to the “things themselves” that announce this exciting new visual art form for and in the sonic arts.

Bibliography

Brougher, Kerry Jeremy Strick, Ari Wiseman and Judith Zilczer (Eds.). Visual Music: Synaesthesia in Art and Music since 1900. Los Angeles: Museum of Contemporary Art. London: Thames & Hudson, 2005.

Center for Visual Music. Website. http://www.centerforvisualmusic.org

Computer Music Journal 29/4 (Winter 2005) “Visual Music.”

Elder, R. Bruce. Harmony and Dissent: Film and Avant-garde Art Movements in the Early Twentieth Century. Film and Media Studies. Waterloo ON: Wilfrid Laurier University Press, 2008.

Fischinger, Oskar. Studie Nr. 1 – Nr. 13 (1929–1934). Oskar Fischinger Filmography. http://www.oskarfischinger.org/OF_Filmo.htm [Last accessed 23 April 2014]

_____. “My Statements are in My Work.” Orig. published in Art in Cinema catalogue. San Francisco Museum of Art, 1947. Available online at http://www.oskarfischinger.org/MyStatements.htm [Last accessed 23 April 2014]

Hébert, Pierre. “The Idea of Animation and Instrumental Expression.” Posted 7 May 2010. Translation from original French, “L’idée de l’animation et l’expression instrumentale,” 2009. Available at http://pierrehebert.com/index.php/2010/04/21/180-the-idea-of-animation-and-instrumental-expression [Last accessed 23 April 2014]

IPL — Intermedia Performance Lab: Research and Education in Interdisciplinary Art (CCRMA). Research available at http://ipl.stanford.edu/Research.html [Last accessed 23 April 2014]

Kapuscinski, Jaroslaw. “Basic Theory of Intermedia Composing with Sounds and Images.” 1997, 1998, 2001. Available on the author’s website at http://www.jaroslawkapuscinski.com/writing.html [Last accessed 23 April 2014]

Kapuscinski, Jaroslaw and Javier Sánchez. “Counterlines: Studies in Interfacing Graphic and Melodic Lines.” ICMC 2009. Proceedings of the International Computer Music Conference (Montréal: McGill University — Schulich School of Music and the Centre for Research in Music Media and Technology [CIRMMT], 16–21 August 2009). Available on the author’s website at http://www.jaroslawkapuscinski.com/pdf/studies-counterlines.pdf [Last accessed 23 April 2014]

Lund, Cornelia and Holger Lund (Eds.). Audio. Visual — On Visual Music and Related Media. Stuttgart: Arnoldsche, 2008.

McDonnell, Maura. “Visual Music Lecture Notes.” 2002. Available on the author’s website at http://soundingvisual.com/visualmusic/index.htm [Last accessed 23 April 2014]

McDonnell, Maura. “Visual Music” [2007]. In Visual Music Marathon [programme book]. Boston: North Eastern University, 2009. Available on the VMM 2009 website at http://www.music.neu.edu/vmm/VMM_booklet.pdf [Last accessed 23 April 2014] Revised version published in eContact! 15.4 — Videomusic (April 2014). https://econtact.ca/15_4/mcdonnell_visualmusic.html

Miller, Dennis. Visual Music Marathon 2009. Programme booklet. Boston: North Eastern University. Available from: http://www.music.neu.edu/vmm/VMM_booklet.pdf [Last accessed 23 April 2014]

Moritz, William. “Towards an Aesthetics of Visual Music.” Asifa Canada Bulletin 14/3 (December 1986). Montreal: ASIFA Canada. Available online at http://www.centerforvisualmusic.org/TAVM.htm [Last accessed 23 April 2014]

_____. Optical Poetry: The Life and Work of Oskar Fischinger. Bloomington IN: Indiana University Press, 2004.

Ox, Jack and Cindy Keefer. “On Curating Recent Digital Abstract Visual Music.” Abstract Visual Music Project and website. New York: New York Digital Salon, 2006. Revised version (2008) available online at http://www.centerforvisualmusic.org/Ox_Keefer_VM.htm [Last accessed 23 April 2014]

Piché, Jean. Personal website. http://www.jeanpiche.com/videomusic.htm [Last accessed 23 April 2014]

Rainer, Cosima, Stella Rollig, Dieter Daniels and Manuela Ammer (Eds.). See This Sound. Germany: Konig, 2009. Web archive available at http://beta.see-this-sound.at [Last accessed 23 April 2014]

Ross, Aaron. “visual music: definition?” Discussion thread on the iotaCenter Discussion Group posted on 5 August 1999. Available at http://groups.yahoo.com/group/iotacenter/message/49 [Last accessed 23 April 2014]

See this Sound — Promises in Sound and Vision. Exhibition, symposium and web archive in Linz (Austria), 28 August 2009 – 10 January 2010. http://www.see-this-sound.at/seethissound/en

Seeing Sound 2009 — Practice-Led Visual Music Research Symposium (Bath, UK: Bath Spa University, 19–20 September 2009). http://www.seeingsound.co.uk

Spalding, Francis. Roger Fry: Art and Life. Berkeley CA: University of California Press, 1980.

Stadnik, George. The Art of Lumia. Website. http://www.photonlightguitars.com/digitallumia.html [Last accessed 23 April 2014]

Watanabe, Chiaki. Personal website. http://www.vusik.net

Creative Works

Battey, Bret. Luna Series #3: Sinus Aestum (2009), computer-realized video and sound. Available on the artist’s website at http://www.mti.dmu.ac.uk/~bbattey/Gallery/sinus.html [Last accessed 23 April 2014]

Cassidy, Jane. The Night After I Kicked It (2008), visual music for 10.1 surround sound. Video. Available on the artist’s website at http://www.janecassidy.net/video.html [Last accessed 23 April 2014]

Cuba, Larry. Calculated Movements (1985 / 6:00). 16 mm Film. Information available at http://www.well.com/~cuba/CMovements.html [Last accessed 23 April 2014]

Detheux, Jean. Visual Abstract (2010). Film. Music by Pierre Jalbert. Performed by the Pittsburgh New Music Ensemble, conducted by Kevin Noe. Available on the artist’s website at http://www.vudici.net/movies/Visual_Abstract/Visual_Abstract_final.html [Last accessed 23 April 2014]

Eggeling, Viking. Symphonie Diagonale (1924 / 5:00). Film.

Garro, Diego. Patah (2009). Vimeo video “PATAH” (15:26) posted by “Diego Garro” on [?] 2011. http://vimeo.com/14112798 [Last accessed 23 April 2014]

Hébert, Pierre. Triptyque (2009). Film. Excerpt available on the artist’s website at http://pierrehebert.com/index.php/2009/09/02/138-triptyque [Last accessed 23 April 2014].

Kapuscinski, Jaroslaw. Juicy (2009 / 10:00), for Disklavier and computer-controlled image and sound. Collaboration with John Edmark. Excerpt available on the artist’s website at http://www.jaroslawkapuscinski.com/work-juicy.html [Last accessed 23 April 2014]

Kapuscinski, Jaroslaw and Javier Sanchez. Counterlines (2009), for Disklavier and Wacom Cintiq. Excerpt available on the Intermedia Performance Lab (IPL) website at http://ipl.stanford.edu/Research.html [Last accessed 23 April 2014]

Lain, Huckleberry. Parallel (2010 / 32:00). Video. Available at: http://www.huckleberrylain.net/cinema/parallel [Last accessed 23 April 2014]

LIA. Construction 76 (2008 / 4:46). Video. Available at http://vimeo.com/1729281 [Last accessed 23 April 2014]

Maxwell, Stephanie. All That Remains (2006 / 6:00). Film. Music by Michaela Eremiasova. Available online at http://people.rit.edu/sampph/W_AllThatRemains.html [Last accessed 23 April 2014]

Miller, Dennis H. Echoing Spaces (2009). Video. Available on the artist’s website http://www.dennismiller.neu.edu [Last accessed 23 April 2014]

O’Connor, Neil. The Glistening Bridge (2004). YouTube video “Somadrone — Film 1” (5:33) posted by “neil o connor” on 22 May 2006. http://youtu.be/0KLkk0-0E9g [Last accessed 23 April 2014]

Richter, Hans. Prelude (1919). Film.

Sanchez, Javier. Unorthogonality (2009). Video. Text and video clip available online at http://ccrma.stanford.edu/~jsanchez/Music_Intermedia.html [Last accessed 23 April 2014]

Schwerdfteger, Kurt. Reflektorische Farblichtspielz (1922). Film.

van Cuijlenborg, Oerd. Jazzimation (1999). Film.

Wilfred, Thomas. 5 Thomas Wilfred Lumia Compositions. DVD. 2007.