Visual Music

by Maura McDonnell

This article, originally penned in 2007, was updated and published in the catalogue of the 2009 Visual Music Marathon 2009. Some minor changes have also been included for its publication in eContact! 15.4.

The Visual Music Artist

Today, technology provides the opportunity to create and mould parameters of image and sound in imagined and unimagined ways. A new type of hybrid artist who works across media and technologies is emerging. Walter Ruttmann foresaw this in 1919 when he remarked that technological progress would lead to the acceleration of the transfer of information between sound and image, leading to a “constant state of being swamped with material” and thereby to an altered state of perception. As a result of this, a “new, hitherto latent type of artist would emerge, approximately half-way between painting and music” (Daniels 1997). This is very much the case today: musicians and music composers can craft visual music compositions either with or for music, and artists and filmmakers can craft original music and soundtracks for their visual compositions.

The computer in particular has become an instrument for forging connections between music and image. John Whitney Sr., whose work with filmmaking across three broad domains (optical to analogue to digital), states: “Technical innovation is thus providing the means to begin a fine art for eye and ear.” He adds, the “computer [is] the only instrumentality for creating music inter-related with active color and graphic design, and though the language of complementarity is still under-examined and experimental, it foretells enormous consequences and offers great promise” (Whitney 1994, 46). The visual music practice of today is diverse and wide-ranging, assisted by computer and digital technology tools. 1[1. For a definition of the types of visual music in contemporary visual music practice, see Jack Ox and Cindy Keefer, “On Curating Recent Digital Abstract Visual Music” (2008).]

Visual Music Composition

Visual Characteristics and Musical Characteristics

What are the visual characteristics available to the visual music artist that are akin to the musical characteristics available to the music composer? How are they used? There are the broad strokes and consideration of, for example, music tradition, style, time, structure, form, space, rhythm, duration, relations, harmony and Gestalt. Then there are the more specific strokes of orchestration, phrasing, line, colour, contrast, shape, pattern, repetition, consonance, dissonance, tone and dynamics. Added to this are the artistic style and intentions of the composer, where there is a consideration for the expression of concepts, ideas and emotions. The result is a temporal visual artwork that exists in time and whose constituent elements evolve over time just as music elements evolve and exist over time. Some broad categories from which to examine visual music compositions are aspects of the language, grammar and syntax of music composition that are either used in a similar manner or shared in the visual domain, or both. Artists, filmmakers, technicians, instrument builders and system designers alike seek their own connections between music and sound and use these parameters and characteristics of sound in different ways.

The first part of this essay draws a path through historical works that connect images and sound, with a view to bringing forward some of these connections. Notwithstanding the difference between visual and sound mediums, the common ground with music in visual music work is a consideration for composition; it is in the composing part — the craft of creation — wherein resemblances to music composition reside. Each visual music artist has an idea and approach to working with his or her chosen visual material. The visual material is pliable and formless; it can be taken from many sources, just as contemporary music takes its sound material from many sources, and it is shaped in many different ways. What is most striking about visual music works, however, is that in order to put some shape onto this visual material, the focus rests on using concepts from music, focusing on the structures and language of music, yet reworking these concepts for a visual production. One of the common properties between music and moving image is motion. All the artists and works discussed here consider motion in their work. At some level, the essence of visual music composition is this composition of motion.

Is it the case that a new grammar for a new art form — which the early pioneers of music and visual art (particularly those who worked with film) had envisioned and sought in their works — is being created and advanced today?

Painting According to a Language of Music

Since the late 19th century, the forms and language of music composition have been used by various painters, often in a metaphorical manner. They are also a source for creating a visual language that helps the artist construct images that focus on creating visual forms and translating concepts and structures of music into concepts and structures of the visual. The resulting imagery is often non-narrative, non-representational and abstract, bringing visual imagery into a similar position as music. The non-representational nature of music and its emotional expression is mirrored in the non-representational nature of the resulting imagery that also expresses and appeals to emotions. By exploring the visual with musical thinking, artists create new visual forms, new patterns and new relationships between visual elements. Abstract painters such as Wassily Kandinsky (1866–1944), Paul Klee (1879–1940) and Roy De Maistre (1894–1968) worked with music concepts and ideas, translating them into their own ideas and principles for artistic practice.

Roy De Maistre, a musician turned painter, devised a system of colour-music codes based on Sir Isaac Newton’s theories of colour and the correlation of specific colours to the seven diatonic pitches of the Western musical scale. The underlying assumption in his colour-music codes was “that a mathematical relationship of frequencies, or vibrations, unites the physical phenomena of light and sound” (Hutchison 1997). De Maistre devised his colour charts and aligned them to the white notes of the piano keyboard starting with the note A for the colour red, and from this he created his colour-music code of pitch correspondence to different hues. He used this colour code and his pictorial composition æsthetic to compose his paintings. This can be seen in his painting Rhythmic Composition in Yellow Green Minor (1919). Interestingly, his later work focuses more on a chord-based, rather than a note-based colour-music code.

Kandinsky was also interested in the analogous relationship between music and painting. He recognized that there was an emotional appeal in abstract art as in music. His artwork imagery is like a counterpoint in music where there is interplay of form, colour and expression. He had a structural method of composition. 2[2. For more information on this method, see Judith Zilczer, “Music for the Eyes: Abstract Painting and Light Art” (2005, 32).] Judith Zilczer quotes Kandinsky from his book On the Spiritual in Art:

Color, which itself affords material for counterpoint, and which conceals endless possibilities within itself, will give rise, in combination with drawing, to that great pictorial counterpoint, by means of which painting also will attain the level of composition and thus place itself in the service of the divine, as a totally pure art. (Zilczer 2005, 32)

Kandinsky was influenced by the atonal music of composer Arnold Schoenberg and found the dissonance in atonal music to be comparable to the freedom and creative energy in his abstract painting.

Paul Klee’s paintings, whose titles often refer to musical terms, used music as a framework with which to explore his artistic ideas. Unlike Kandinsky, Klee was interested in the traditional harmony and counterpoint of 18th-century music composition. The Classical music period developed disciplines and rules of composition across a range of forms and structures. Klee’s interest in working with music structure was an important part of his painting composition. He used colour to create harmony in his work, eventually focusing on working with the musical analogy of polyphony and counterpoint in his painting, transforming the formal elements of musical polyphony into an equivalent form of visual polyphony for his paintings. He was interested also in formulating a vocabulary of abstract art comparable to the rules and structures of music composition. One of his visual polyphonic methods was to overlay colours and intermingle shapes and forms to suggest polyphonic forms and create a rhythm and direction with pictorial elements.

Principles of Counterpoint — Orchestration of Form

Viking Eggeling and Hans Richter researched and experimented with the æsthetics of the visual and through their art new principles explored technical devices and techniques to create non-representational works. One of their ideas was to explore the many visual forms of relationships that used the principle of the “equivalence of opposites” (Richter 1952); but they approached this idea with the principle of musical counterpoint. This led to a number of drawings of “themes” or “instruments” consisting of the transformation of the contrasting relations of visual elements, which were orchestrated through different stages, bringing about a dynamic arrangement that they felt as being “the music of the orchestrated form.” This can be illustrated in Richter’s first scroll painting Prelude (1919), described as the orchestration of a theme developed in eleven drawings, which demonstrates such oppositions among the visual elements where “a vertical line was accentuated by a horizontal, a strong line connected with a weak one, a single line gained importance from many lines, etc.” (Turvey 2003).

Eggeling and Richter made the unexpected discovery that transformation of the visual relationships across the large, elongated, horizontal painting of the scroll forces a type of rhythmic expression into the painting which, in turn, creates a form of dynamic expression that produces a sensation for the eye as it transverses and memorizes the sequence of visual elements across the scroll.

Orchestration of Movement

Eggeling and Richter realized that the kind of accumulated energy that took place in the orchestration of the visual forms across the scroll needed to be released into actual movement: “movement implied film” (Richter 1952). Film was a new medium for them, yet each took the ideas and principles used in his scroll paintings and worked with the constraints and potential of this new moving art medium. Eggeling continued to work with the orchestration of form and applied motion to the forms that he had developed for his scroll paintings. His visual forms for film were comparable to music instruments that now had their own defined ways of being articulated through different stages, with motion, over time.

This availability of time and continuity through the medium of film brought to full realization the release of the movement that had accumulated in the scrolls. Eggeling’s forms still retained what Richter calls their “graphic elegance”. In his film Symphonie diagonale (1924, first publicly shown in May 1925 at Der absolute Film [Absolute Film] series in Germany), the graphical lines and shapes that appear and disappear, and that evolve and transform over time, were created using paper cut-outs and tin foil figures that were photographed one frame at a time. Symphonie Diagonale has a tremendous musical feel to it, in its use of rhythm, motifs, themes and forms. The film has no soundtrack, but it has the most evocative musical quality to it. The animation of the visual elements of line, figure and shape brought about rhythmic sequences and a sense of dynamics in the progression of these rhythmic figures. Visual “instruments” could now evolve, transform and progress in visual rhythmic sequences.

These parameters of “instruments”, rhythm, dynamics, figures and shape are analogous to the parameters of rhythm, pitch, phrasing and timbre available to music composition. In abstract animation, the now hard-to-define artist could compose his or her animations like musical compositions, orchestrating the visual elements, creating motifs and repetitive elements, transforming a visual element’s shape over time, and creating a sense of harmony and symmetry in the use of screen space and screen time. All the non-representational strategies for composing music were now available to the abstract filmmaker.

Orchestration of Time

Richter realized that time was the basis of this new art form. Continuing from his ideas about counterpoint and the equivalence of opposites, Richter moved his focus from orchestrating form to orchestrating time relationships:

The simple square of the movie screen could easily be divided and “orchestrated.” These divisions or parts could then be orchestrated in time by accepting the rectangle of the “movie-canvas” as the form element. Thus it became possible to relate (in contrast-analogy) the various movements on this “movie-canvas” to each other — in a formal as well as a temporal sense. (Richter 1952)

Richter’s first abstract film Rhythmus 21 (1921) 3[3. Richter’s dating of 1921 has been often questioned. Some believe the film was actually made later, ca. 1924.] is an excellent example of his ideas about time relationships. The screen being the form of the image is most interesting. The visual forms that are created for the screen exhibit two main properties: the static visual composition of each frame of the screen and the temporal composition of both the screen image and the visual forms in the screen image over time.

Expressive Moving Paintings and the Composed Score

Walter Ruttmann’s Lichtspiel Opus I premiered in Germany in 1921, the first abstract film to be publicly screened. In the film, Ruttmann mastered the technical means to realize his abstract imagery in film. He patented his particular technical methods in 1921. Dr. William Moritz provides an interesting description of his method:

[Ruttmann’s] first animations for Opus No. 1 were painted with oil paints on glass plates beneath an animation camera, shooting a frame after each brush stroke or each alteration because the wet paint could be wiped away or modified quite easily. He later combined this with geometric cut-outs on a separate layer of glass. (Moritz 1997)

Ruttmann’s visual style is considered to be more expressive than Eggeling and Richter’s and overall has a painterly feel to it both in technique and in the use of screen, colour and movement. Indeed, his technical methods are also painterly and would have had a definite bearing on the resulting imagery. His Opus films have been described as paintings that move in time. While Richter and Eggeling focus on figures, forms and time relationships between visual elements, Ruttmann focuses on a more expressive visual æsthetic for his imagery. He exploits “movement and color to create choreographies, where entrances and exits, collisions and complementary trajectories establish a linear, cumulative scenario or development in which new configurations, colors and shapes appear right up to the last moments of the film.” And he uses colour as an element in the choreography that helps not only to structure the film and “differentiate certain shapes, movements or repetitions, but [also] sometimes to establish general mood or atmosphere” (Moritz 1997).

Ruttmann also envisioned his Lichtspiel Opus I film to be closely related to music and commissioned the composer Max Butting to compose a string quartet for it. In the music score, Ruttmann provided many indicators for the musicians to ensure that the music precisely synchronized with the visual elements of the film.

Colour Sings — Coloured Rhythm

Léopold Survage (1879–1968) connected his ideas about colour with music and foresaw the potential of film to bring forth his ideas about colour — colour and rhythm in particular. Because sound is the primary element of music and colour is the primary element of painting, he believed that when rhythm, by means of movement, is applied to colour, the resulting coloured rhythm becomes an abstract form that is superior to the use of colour in static painting, and is more like music. Colour sings because it is in rhythmic motion; the principle of mobility brings forth the rhythmic motion (Putnam 1929, 112–119). The alternating series of colour that occurs when colour and rhythm is realized with motion can exert a psychological influence on us that is similar to the way the alternating series of sound in music exerts a psychological influence.

Survage conceived that rhythm existed independently of colour, but that through the mobile animation of colour, rhythm could be captured and æsthetically harnessed, and a coloured rhythm would result (Ibid.). He conceived that the film medium could be the means for providing the mobile animation of this coloured rhythm. Survage prepared a series of drawings, his Colored Rhythm: Study for the Film (1913), for the purposes of having them realized in film. Unfortunately, not being able to secure the funding or a patent for it, Survage never made the film. Nevertheless, the individual pictures for this film and the idea of using film or cinema to realize the movement of these stills were in place by 1913.

Similar to Richter and Eggeling, Survage recognized that time was the necessary component to bring forth, in particular, the dynamics of rhythm and movement of visual elements. His intended film Rhythme Colore and his series of drawings for the film didn’t simply illustrate or interpret music; rather, he believed them to be an autonomous art, based upon the same psychological premise of music. It is the “mode of succession of their elements in time which establishes the analogy between music, sound rhythm, and that colored rhythm of which I am announcing the realization by means of the cinema” (Ibid.). His colour was an essential part of his pure abstract image æsthetic.

Composing Motion — Figures of Motion

Eggeling, Richter, Ruttmann and Survage all came to the realization that motion was needed to fully realize their visual æsthetics. Their works and ideas crossed the “glistening bridge” from still to moving art, as Survage had foreseen in the medium of film (Guillaume Apollinaire quoted in Russett and Starr 1976, 35). Len Lye (1901–1980), a kinetic artist and filmmaker, was interested in the kinetic potential of visual forms for film, which he perceived to be like music. He worked with the techniques of direct film and colour processes to create films that enhanced the kinetic and motion potential of imagery. He identified his visual forms, which were often set to jazz-calypso music, as being like figures of motion. Composing visuals set to music was a task in composing motion, a task similar to that of the music composer who composes music.

Len Lye discovered the potential of composing with motion. “If there was such a thing as composing music, there could be such a thing as composing motion. After all, there are melodic figures, why can’t there be figures of motion?” (Lye 1984). His film A Colour Box (1935) is the first film in which he painted and scratched directly onto the filmstrip, creating a visual mass of complex and jumbled movements. As one commentator notes:

This creates a sense of off-screen space, as if the patterns are streaming in and out of the frame. Also, the dynamic abstract shapes seem to dance to the popular Cuban music that was used as a soundtrack. Lye used the soundtrack as a creative base by associating particular shapes with certain sounds, so that there is a loose relationship between sound and image. (Sexton)

Though “loose”, as Sexton remarks, there is, nonetheless, “a [musical] relationship between sound and image” in the image.

Music and Image Synchronization — Acoustical Laws and Optical Expression

Oskar Fischinger (1900–1967) saw the first performance of Walter Ruttmann’s Lichtspiel Opus 1 in 1921 and as a result was inspired to work with the absolute cinema of abstract filmmaking and synchronized music. 4[4. Acknowledgment and thanks to Cindy Keefer, director of Center for Visual Music, for her expertise and advice in relation to Oskar Fischinger.] Music was used in his films to make the absolute nature of the visuals more understandable. As William Moritz explains: “In the spirit of non-objective art, [Fischinger] maintained, correctly, that his films were absolute experiences in and of themselves, not representations of some other object or experience.” However, even when the music would suggest a flow or structure for his film, his visual compositions were composed carefully to “represent visual structures and dialogue with some sort of meaningful conclusion” (Moritz 1976). For example, in his Studies series of films, started in 1929, he explores in each film a specific visual task. In Studie No. 9, he explored streaking afterimages, which were also explored in several other Studies films (Moritz 1979).

Music adds another dimension to Fischinger’s films, where the tightly synchronized non-representational graphics and music appeal directly to the feelings of the viewer.

The flood of feeling created through music intensified the feeling and effectiveness of this graphic cinematic expression, and helped to make understandable the absolute film. Under the guidance of music, which was already highly developed, there came the speedy discovery of new laws — the application of acoustical laws to optical expression was possible. As in the dance, new motions and rhythms sprang out of the music — and the rhythms became more and more important. (Fischinger 1947)

By focusing on the rhythm and dynamics of the music to enhance the experience of the abstract elements, the visuals and the music at times seem to fuse. The rhythm and the dynamics in each medium have a togetherness and unity.

Fischinger selected music from classical and jazz traditions for his music and image films. Some of the music tracks were chosen by his patron, Baroness Rebay. For example, Rebay commissioned Fischinger to create a film using Bach’s Brandenburg Concerto No. 3, which resulted in Motion Painting No. 1 (1947).

Fischinger was an innovative and versatile filmmaker and inventor. In his work, he straddles several visual music traditions and filmmaking techniques, even inventing his own devices to realize his ideas. He invented the Lumigraph, a device to perform colour, as well as a wax-slicing machine, which he used to create a temporal transformation of both soft and hard geometric imagery for his films — he also built one for Walter Ruttmann 5[5. For a description of Fischinger’s wax cutting machine, see William Moritz, “Oskar Fischinger: Artist of the Century.”]. He created synthetic sound by modifying a camera that was able to photograph his ornament drawings and other geometric shapes right onto the film’s soundtrack. His Ornament Sound (1931) was able to turn his visual shapes into actual sounds (Fischinger 1932).

Synthesis — Simultaneous Composition of Sound and Image

The Whitney brothers, John Whitney Sr. (1917–1995) and James Whitney (1921–1982), sought to bridge the gap between animation and sound and their actual realization. 6[6. Acknowledgment and thanks to John Whitney Jr. for his helpful insights.] They were interested in the combination of graphic arts and music and in building devices to realize the synthesis of music and image. An important part of their method was to meticulously plan, compose and score their films and investigate the “complementarity” of music with filmmaking. In order to do this, however, they departed from looking at the cinematic potential and focused more on methods to create a more simultaneous composition of sound and image and a more immediate rendition of the temporal dimension of sound and image. They also strove to create a more instantaneous harmonic union of abstract art and music (Moritz 1979).

The Whitneys were influenced by the music of Schoenberg and in particular Schoenberg’s twelve-tone system of music composition, for example in composing the films Catalog and Arabesque. In 1975, John Whitney Sr. invited the contemporary visual music computer animator Larry Cuba to be the programmer on Arabesque. Just as Schoenberg is often acclaimed to have reduced music to the serial row, so too, some argue the Whitney brothers reduced the image “down to its most fundamental state — essentially a point of light, which could then be ordered like a tone row” (Brougher et al. 2005, 125).

While working out the ideas for his visual-music artworks, John Whitney Sr. simultaneously created the equipment — at first optical, then analogue and later digital (the computer) — to implement them in both his and some of James Whitney’s films (Whitney Jr. 2007). For example, he developed one device that consisted of an optical printer and another that used a system of pendulums to photograph and control the light to expose areas of the soundtrack. This allowed him to create sounds directly on the soundtrack section of the film, “translating oscillations into synthetic sounds of various frequencies and timbres,” enabling him and his brother “to compose sound and image simultaneously” (Brougher et al. 2005). The devices were used to create the early films Five Film Exercises (1943–94) by John and James Whitney.

In 1958, Whitney Sr. designed a mechanical analogue cam machine to create the visuals effects for Catalog (1961). The same machine was also used by James Whitney for his beautiful Lapis film (1963–66). Whitney Sr. soon embraced computer technology and the graphic potential of computer technology, developing computer programs that synchronized computer-generated graphics with music. His Whitney-Reed RDTD (Radius-Differential Theta Differential) composing programme was used to generate imagery from sound oscillations. He used his RDTD software to produce his film Permutations (1968).

Whitney Sr. made several discoveries about the connections between music and filmmaking. In 1975, he discussed the temporal dimension of art, using the term “computational periodics” while exploring the periodic nature of the visual and sound domains. 7[7. For a discussion of computational periodics, see John Whitney, “Computational Periodics,” originally published in Artist and Computer (1976).] In 1994, he used the term “complementarity” in terms of the power of computer technology.

A major art form based on a common foundation of harmony is developing that intertwines color with tone in a complementary bond. I call this associative relationship a “complementarity.” For the first time, one can design and execute visual and musical patterns in an inter-reactive form of temporal union. (Whitney Sr. 1994)

Another discovery that Whitney made was that action and harmony shaped his audio-visual work. Like other abstract filmmakers, Whitney recognized that his æsthetic and, in particular, his focus on actions had an impact on emotions: “Fluid, orderly action generates or resolves tensions much in the manner that orderly sequences of resonant tonal harmony have an impact on emotion and feeling…” (Whitney Sr. 1991, 597).

Visual Music Performance

Instruments and Systems to Perform Image Parameters

Visual music performances are realized with physical instruments that are played in a manner analogous to a musician playing a music instrument or, increasingly today, with a more system-related approach. In the latter, a variety of computer technologies and mechanical devices (if necessary), both hardware and software, are used to create a mixture of real-time controllers (both physical and virtual) that drives the live performance of images and sound. Recent contemporary artists such as Golan Levin, Fred Collopy, Robert M. Fuhrer, Roger B. Dannenberg, Sydney Fels, Kazushi Nishimoto, Kenji Mase and Frank J. Malina have developed systems to generate visuals in real time. 8[8. See Fred Collopy’s bibliography of “Lumia and Instruments for Creating Them.”] Toshio Iwai’s installation Piano as Image Media (1995) provides an interesting interface in which the visual element, light, plays the music element, a piano. As in all good music performances, however, the composition and the artistic strategy are as essential as the performance strategy, not to mention the importance of the æsthetic and technical connections sought between music and image parameters.

The second part of this essay provides an overview of some of the colour and sound correlations that have been used to create instruments to perform colour. It will examine representative examples from the very early colour-organ traditions.

The Correlation of Color to Music Tones

The theories of colour from the late 18th and early 19th centuries influenced the plethora of colour organs that were built in the late 19th and early 20th centuries. It was believed at the time that light and sound were physically similar. Sir Isaac Newton in his Opticks (1704) was “the first to observe a correspondence between the proportionate width of the seven prismatic rays and the string lengths required to produce the musical scale D, E, F, G, A, B, C” (Peacock 1988, 398). Newton devised a correspondence of relative widths of the colour spectrum to the successive notes of the music scale. His correspondences of colour to music tone were: red to C, orange to D, yellow to E, green to F, blue to G, indigo to A and violet to B.

“Instruments to Perform Color-Music”

9[9. Title from Peacock 1988, 398.]

The colour-to-music tone correlation that was apparent in Newton’s correspondences provided the inspiration for a “widespread interest in a viable color-transmission instrument that could be operated from a musical keyboard” (Ibid., 399). Several instrument builders did just that by building keyboard instruments to perform colour (with or without sound capabilities) in an analogous manner to the performance of music. Interestingly, colour-organ builders devised their own variations of Newton’s correspondences by creating their own colour-music scales as well as discovering how best to present the coloured-light performance. For example, Bainbridge Bishop noted that when he experimented with building a color organ, he found that:

a simple color did not give the sensation of a musical tone, but a color softened by gradations into neutral shades or tinted grays did so; also, that combinations of colors softened by gradations into neutral shades or tinted grays, with the edges of the main colors blending together, or nearly together, rendered the sensation of musical chords very well indeed. (Bishop 1893)

Color Music Scale — A Harpsichord for the Eyes

Louis-Bertrand Castel (1688–1757) adapted Newton’s correspondence of the relative widths of the colour spectrum to the chromatic tones of the music scale into his own colour-music scale. Castel devised a colour-music scale: twelve colours to twelve music tones. The note C was correlated to red, moving up to the note B, which correlated with indigo (Wells 1980, 104; Collopy 2000). Castel demonstrated his scale by adapting a harpsichord so that “the pressing of the keys would bring out the colours with their combinations and their chords; in one word, with all their harmony, which would correspond exactly to that of any kind of music” (Franssen 1991).

Castel built his harpsichord in 1734 and called it the Clavecin Oculaire. This colour organ consisted of:

a 6-foot square frame above a normal harpsichord; the frame contained 60 small windows each with a different colored-glass pane and a small curtain attached by pullies to one specific key, so that each time that key would be struck, that curtain would lift briefly to show a flash of corresponding color. (Moritz 1997a)

Castel was also interested in emulating the experience of participating in a musical performance, an idea that precedes the building of his organ. In 1720 he wrote: “Can anyone imagine anything in the arts that would surpass the visible rendering of sound, which would enable the eyes to partake of all the pleasures which music gives to the ears?” (Conrad 1999, 393).

Painting Music — Harmonious Colour

The American inventor Bainbridge Bishop (1837–1905) was interested in working with colour harmonies. One of his methods of creating harmonic colours was to apply the intervals and harmony of music to the visual domain. As he says in his own writing, however, he also became “dominated with the idea of painting music” (Bishop 1893). Working out a way to create harmonious colour in an analogous manner to harmonious music was one of his goals. Thus:

The natural harmonic chord of light, as illustrated by the rainbow, shows red as its fundamental or keynote; for this reason I think we should take C, the key-note of the natural scale. It will be observed that its dominant is greenish-blue, its subdominant yellow-green. The greens of nature seem to make up combinations and masses of greens inclining to these two hues. A pure crude green seems to be out of place in a landscape, and, if seen, it generally produces a harsh and discordant effect. (Bishop 1893)

Bishop set himself the task of constructing “an effective and practical mechanism which would play colors and music together.” His colour-to-pitch correspondence was based on correlating colour to the chromatic musical scale.

After several attempts and experiments, Bishop built a color organ in 1877. An interesting feature of his model was his use of glass to diffuse light. 10[10. For a description of his color organ, see Bainbridge Bishop, “A Souvenir of the Color Organ” (1893).] Bishop also provided stops and pedals in his Color Organ that “could be used by the musician, at will, to aid the expression of the sentiment of the music” (Bishop 1893).

Mobile Colour

Alexander Wallace Rimington (1854–1918) invented a more responsive device to perform colour, using electricity to power a system of colour filters and arc lamps. It was built in 1893 and was named the Colour Organ. As Peacock notes, “this name has become the generic term for all such devices designed to project colored light” (Peacock 1988).

Rimington presented a paper at St. James’s Hall on June 6, 1895 about colour music, and in 1911 he published the book Colour-Music: The Art of Mobile Colour. He believed that physical analogies existed between sound and colour. His colour-music scale was based on dividing the light spectrum into intervals of the same proportions that occur in the musical octave, therefore, “the ratio between two light waves approximated that for a corresponding interval in sound” (Ibid.). Each octave then contained the same colours. “The registral placement of colors was directly proportional to saturation, i.e., higher octaves contained more white light” (Ibid.), with saturation becoming the main source of distinguishing the octaves. Rimington’s colour organ could not simultaneously play colour and sound. He did recommend, however, that it be played with sound-producing instruments.

Rimington foresaw his colour organ as allowing standard musical works to be played and expressed in colour. He hoped that composers would in the future write their scores in dual form, composing both a music score and a score for light effects. The composer Alexander Scriabin (1872–1915) included in his score for his music composition Prometheus, Poem of Fire (1911) a score for coloured light. This was notated at the top of the score. The first performance of Prometheus with colour realization took place in New York in 1915. The coloured-lights part was played on a colour-projection device called the chromola, a device similar to Rimington’s colour organ. (Ibid.).

Lumia

Rather than focusing on theories of colour and sound correlations, Thomas Wilfred (1889–1968) focused on light alone. He believed the art of light to be a new art form that he named “lumia”, and in 1922 built an instrument called the Clavilux to realize his ideas. In addition to pure light manipulation, the Clavilux allowed for elements of time and rhythm to be articulated in live performance (Ibid.). This is apparent within the controls of the instrument, which consist of keys that are organized into three groups: form keys, colour keys and motion keys. 11[11. For discussion of the three groups on the Clavilux, see Fred Collopy, “Color, Form, and Motion” (2000).]

A neutral white beam of light of great strength is intercepted by an arrangement of lenses and built into form through the form keys. The forms, or forms, are made to move rhythmically by means of the motion keys, and either one color or several in any combination are finally introduced from the color keys. The whole instrument is played from a notation book so that any composition can be duplicated exactly, with a margin for personal interpretations by the playing artist. (Collopy 2000, 356)

Wilfred composed for the Clavilux and notated his compositions. “A Wilfred Lumia work is a composition of light, color, and form which changes slowly with time. It exhibits a very wide range of light intensity and a broad spectrum of delicate colors and shapes” (Epstein). Between 1924 and 1925 Wilfred played numerous worldwide lumia recitals. In 1930 he founded the Art Institute of Light for research into lumia and his new art form of light.

Colour Projection Instruments

Peacock notes that since 1920 many colour projection instruments and devices were invented (Peacock 1988), such as Adrian Klein’s colour projector for stage lighting (1920), Achille Ricciardo’s Teatro del Colore, the composer Alexander László’s Sonchromatoscope (1925) and Ludwig Hirschfield-Mack’s Shadow Plays (1920–25). From the 1930s, many artists experimented with interpreting music in coloured light. Peacock discusses George Hall’s Musichrome (1930s), Frederick Bentham’s Light Console, Christian Sidenius’ “Lumia, Theatre of Light” concerts and also Bulat Galeyev and the use of his colour instruments to present huge outdoor spectacles of sound and light. Interestingly, composer Alexander László commissioned filmmaker Oskar Fischinger to arrange a film projection with László’s later colour-light-music concerts.

Oskar edited together footage from several of his earlier experimental films, arranging them for five 35mm projectors, three side-by-side to form a triptych, and two overlapping these to provide additional color effects. Painted slides were also used to blur the edges of the projections. (Moritz 2000)

Fischinger, however, is not usually credited in the program. He re-created his “multiple-projector performance several times, including a piece titled Fever I II III which reportedly had a musical score composed by Erich Korngold, and a screening at the prestigious Munich State Theatre in 1927, with the title R-1, a Form-Play, using a percussion ensemble as the music (which could drown out the noise of the several projectors)” [Ibid.].

Summary

Visual music can take two strands in its approach to the craft and presentation of itself as an art medium. It can centre its focus on either the craft of composition — working with mobile visual elements over time to be realized in time as a fixed-media video or film projection — or on the performance aspect — very often an improvisational approach. Of course a fusion of the two is also possible, in which the mobile visual elements realized via mechanical or generative means are integrated into a non-fixed media that exists mainly in its performance. The latter approach often involves considerable technical skill in building a system or item that can realize a visual music in a real-time setting.

In this essay we have explored some of the pioneering efforts of filmmakers from the absolute and abstract filmmaking traditions from the 1920s onward as well as the inventors of devices to compose or perform visuals like music from the late 19th century. Their efforts provide us today with much inspiration. What is of interest is how they sought connections between music and image, how this influenced their æsthetic strategy and how they realized their music to image ideas in their work. The focus has been on bringing forth the nuances of taking a visual music composition approach to draw out the manner in which visual music is about composition and the æsthetics of composition. We have also considered the influence of colour-to-music correlations that inspired artists to build instruments to perform colour and use light as a means of composing colour harmonies analogous to music harmony. These pioneers provide us with the origins of a grammar for working with the complementarity of music and image. This grammar of visual music continues to advance today as more artist, filmmakers and composers work with the complementarity of music and image, and develop or evolve their methods for seeking out that complementarity.

Bibliography

Bishop, Bainbridge. A Souvenir of the Color Organ, With Some Suggestions in Regard to the Soul of the Rainbow and the Harmony of Light. New Russia NY, 1893. Available online at http://www.rhythmiclight.com/books/HarmonyOfLight.pdf [Last accessed 26 April 2014]

Brougher, Kerry, Jeremy Strick and Ari Wiseman (Eds.). Visual Music Synæsthesia in Art and Music Since 1900. Thames & Hudson, 2005.

Collopy, Fred. “Color, Form, and Motion: Dimensions of a Musical Art of Light.” Leonardo 33/5 (October 2000), pp. 355–360.

_____. “[Rhythmic Light Timeline].” Last updated 2007. Available online at http://www.rhythmiclight.com/archives/timeline.html [Last accessed 26 April 2014]

Conrad, Daniel. “The Dichromaccord Reinventing the Elusive Color Organ.” Leonardo 32/5 (October 1999), pp. 393–398.

Daniels, Dieter. “Art and Media in the XX. Century.” 1997. Available online at http://www.hgb-leipzig.de/index.php?a=person&b=mitarb&c=&d=&p=310& [Last accessed 26 April 2014]

Epstein, Eugene. “Light—Art—Lumia: A Site about the pioneering work of Thomas Wilfred.” [n.d.]. http://www.lumia-wilfred.org/content/intro.html

Fischinger, Oskar. “Sounding Ornaments.” First published in the Deutsche Allgemeine Zeitung, 8 July 1932, then widely syndicated in other newspapers. Available online at http://www.oskarfischinger.org/Sounding.htm [Last accessed 26 April 2014]

_____. “My Statements Are My Work.” In Art in Cinema [catalogue], pp. 38–40. San Francisco, 1947. Available online at http://www.oskarfischinger.org/MyStatements.htm [Last accessed 26 April 2014]

Franssen, Maarten. The Ocular Harpsichord of Louis-Bertrand Castel: The Science and Aesthetics of an Eighteenth-Century Cause Célèbre. Department of Philosophy, Universiteit van Amsterdam, [1991?].

Hutchison, Niels. “Colour Music in Australia: De-mystifying De Maistre.” 1997. Available online at http://www.colourmusic.info/maistre.htm [Last accessed 26 April 2014]

Lye, Len. “Beginnings.” Figures of Motion: Len Lye Selected Writings. Edited by Wystan Curnow and Roger Horrocks. Auckland: Auckland University Press / Oxford University Press, 1984, pp. 31–32.

Moritz, William. “The Importance of Being Fischinger.” Ottawa International Animated Film Festival [programme booklet]. 1976. Available online at http://www.centerforvisualmusic.org/library/ImportBF.htm [Last accessed 26 April 2014]

_____. “Non-Objective Film: The Second Generation.” In Film as Film, Formal Experiment in Film, 1910–1975 [exhibition catalogue], pp. 59–71. London: Hayward Gallery, 1979. Available online at http://www.iotacenter.org/visualmusic/articles/moritz/moritz_nonobjective [Last accessed 26 April 2014]

_____. “Restoring the Esthetics of Early Abstract Films.” In A Reader in Animation Studies. Edited by Jayne Pilling. Sydney: John Libbey, 1997. Available online at http://www.iotacenter.org/visualmusic/articles/moritz/restaest1 [Last accessed 26 April 2014]

_____. “The Dream of Color Music, and Machines That Made It Possible.” Animation World 2/1 (April 1997).

_____. “Oskar Fischinger: Artist of the Century.” In Kinetica 2 [exhibition catalogue]. Los Angeles: The iotaCenter, 2000. http://www.iotacenter.org/visualmusic/articles/moritz/fischartist [Last accessed 26 April 2014]

Ox, Jack and Cindy Keefer. “On Curating Recent Digital Abstract Visual Music.” Abstract Visual Music. New York Digital Salon catalogue (2006–08). Revised version (2008) available online at http://www.centerforvisualmusic.org/Ox_Keefer_VM.htm [Last accessed 26 April 2014]

Peacock, Kenneth. “Instruments to Perform Color-Music: Two Centuries of Technological Experimentation.” Leonardo 21/4 (February 1988), pp. 397–406.

Putnam, Samuel. The Glistening Bridge: Léopold Survage and the Spatial Problem in Painting. New York: Covici-Friede Publishers, 1929. Available online at http://rhythmiclight.com/books/TheGlisteningBridge.pdf [Last accessed 26 April 2014]

Richter, Hans. “Easel — Scroll — Film.” Magazine of Art (February 1952), pp. 78–86. Available online at http://www.rhythmiclight.com/articles/EaselScrollFilm.pdf [Last accessed 26 April 2014]

Rimington, Alexander Wallace. “A New Art: Colour-Music.” Pamphlet. 1985. Available online at http://www.lumen.nu/rekveld/wp/?page_id=185 [Last accessed 26 April 2014]

_____. Colour-Music: The Art of Mobile Colour. London: Hutchinson, 1911.

Russett, Robert and Cecile Starr. Experimental Animation: Origins of a New Art. New York: Da Capo Press, 1976.

Schils, Paul. “Color Spaces.” http://www.color-theory-phenomena.nl/08.00.html [Last accessed 26 April 2014]

Sexton, Jamie. “Colour Box, A (1935).” Screenonline: Len Lye. Available online at http://www.screenonline.org.uk/film/id/442234 [Last accessed 26 April 2014]

Turvey, Malcolm. “Dada between Heaven and Hell: Abstraction and Universal Language in the Rhythm Films of Hans Richter.” October 105 (Summer 2003), pp. 13–36.

Wells, Alan. “Music and Visual Color: A Proposed Correlation.” Leonardo 13/2 (Summer 1980), pp. 101–107.

Whitney, John, Jr. Private conversation. 6 March 2007.

Whitney, John, Sr. “To Paint on Water: The Audiovisual Duet of Complementarity.” Computer Music Journal 18/3 (Fall 1994) “Composition and Performance in the 1990s (2),” pp. 45–52.

_____. “Fifty Years of Composing Computer Music and Graphics: How Time’s New Solid-State Tractability Has Changed Audio-Visual Perspectives,” Leonardo 24/5 (November 1991), pp. 597–99.

Zilczer, Judith. “Music for the Eyes: Abstract Painting and Light Art.” In Visual Music Synæsthesia in Art and Music Since 1900. Edited by Kerry Brougher, Jeremy Strick and Ari Wiseman. Thames & Hudson, 2005.