Digitized Direct Animation

Creating materials for electroacoustic visual music using 8 mm film

by Jonathan Weinel and Stuart Cunningham 1[1. The work discussed in this article, Mezcal Animations #1–3, was created entirely by Jonathan Weinel, while Stuart Cunningham’s contribution here concerns the digitization method described in the section titled “Digitizing 8 mm Film”; hence the change in person from singular to plural.]

Direct Animation in Context

“Direct animation”, or “drawn-on animation” typically involves various techniques working directly with the film to create moving images, without the use of a camera. Artists may scratch into developed film or leader material, or apply paints, dyes, inks and other materials directly to the film. The practice emerges alongside avant-garde artistic movements of the early 20th century such as dada, futurism and surrealism, which gave rise to experimental film-making.

New Zealand-born Len Lye is credited as producing the first publically performed work of this type, Colour Box (1935). This was one of many works he produced that were commissioned by the GPO Film Unit (Russet and Starr 1976, 65). For Colour Box, Lye used Dufaycolor 2[2. For an explanation of Dufaycolor, please see Brighton Royal Pavilion, Museums & Libraries 2013.], a film dying process. Using this process he applies colours and patterns to the film, which match a lively dance soundtrack that was popular in Paris at the time, La Belle Creole by Don Baretto and his Cuban Orchestra (Horrocks 2001, 137). Lye went on to produce several other pieces such as Kaleidoscope (1935) and Musical Poster #1 (1940), for which he uses similar camera-less techniques. In Trade Tattoo (1937), he combines direct animation with discarded documentary footage, while Free Radicals (1958) was created by scratching into black 16 mm leader film.

The direct animation technique was also explored by Norman McLaren around the same time. Compositions such as Dots (1940) and Begone Dull Care (1949) demonstrate similar drawn-on techniques, with careful matching of the sound and visual forms (Russet 1976, 116–128). In the case of Dots, the soundtrack was composed by drawing the sound on to an optical sound strip (“Norman McLaren: Pen Point Percussion” 2013). This enables close correspondences between the audio and visual material.

Stan Brakhage is another artist who has produced significant works using this technique. His film Mothlight (1963) consists of moths, grass and other debris that are attached to 16 mm film. While Mothlight is silent, Brakhage considers the piece to have some musical-like aspects to its form (Ganguly 1994). In other works such as Persian Series (1999) he hand-paints directly on to the film.

Harry Smith has created many works utilising this approach, such as his series of works titled Early Abstractions (1946–57). He devised various techniques for working directly on film, including hand-painting and stencilling approaches. His films can be appreciated for their æsthetic qualities and movement, and incorporate hidden symbolic meanings (Russet 1976, 138). His films were cut with various music including Dizzy Gillespie and the Beatles.

In recent years direct animation continues to see use by experimental filmmakers. Today, the technique may also draw associations with the preservation and use of vintage equipment, and the decay of analogue mediums. Katherine Berger has discussed some works created since 2000, including her own, which incorporate direct animation techniques such as burying film (Berger 2013).

Those works of direct animation mentioned tend to be associated with the “visual music” label. Notable pioneers in the field of visual music include John Whitney, Oskar Fischinger, Jordan Belson, Mary Ellen Bute and Charles Dockum. 3[3. The online Centre for Visual Music is a useful repository for work by many of the key visual music artists mentioned.] Visual music encompasses a variety of approaches but is broadly an artistic discipline that explores correspondences between sound and visual forms. The visual forms are often abstract, using arrangements and movements of colour, shape and geometry, rather than representational forms. Works of visual music may explore correspondences between sound and visual forms in a variety of ways. Works may be created for existing pieces of music, or a new piece of music may be composed. Some may be silent, but considered still as “musical” in their arrangement of visual material.

For those pieces with a soundtrack, audio features such as amplitude, frequency or musical form may become related to visual features such as the size or position of a shape. As Roger Danneberg comments, the more interesting works are often those that do not find these correlations in the most obvious ways; hence deep structure correspondences may provoke more interest for the audiences than the ubiquitous visualisations found in desktop multi-media players, that use the audio waveform as a basis for colourful patterns (Danneberg 2005, 28).

Visual music can be seen as a constellation in an expanding galaxy of audio-visual, VJ, visualisations, light synthesizers, video music, music video, music-related video games and web-based artworks, for which the boundaries are not always clearly defined. This area also increasingly includes real-time works, such as those devised using the Jitter or Processing programming languages. The widespread use of streaming video services such as Vimeo and YouTube enable users to effortlessly publish their own creative work, or the work of others. The result of this is that many hard-to-find films are now more readily available for a wider audience. Modern cultures of remixing also mean that fixed works may be reappropriated and rescored without the consent or involvement of the original artists.

Amongst the many areas of video-related work, my piece Mezcal Animations #1–3 can be considered to fall within the categories of visual music, and audio-visual electroacoustic compositions (Video 1). I consider it as part of visual music, since the piece is concerned with abstract moving images and sound correspondence, and utilises a technique (direct animation) that is strongly associated with visual music. Since the piece is designed with an electroacoustic soundtrack, and since my background is as an electroacoustic composer, it must also be considered as an audio-visual electroacoustic composition. 4[4. For an example of electroacoustic audio-visual composition, please see Diego Garro’s Patah (2009).]

Creating the Reels

The visual material for Mezcal Animations #1–3 consists of material produced using direct animation on standard 8 mm film. The piece was constructed using digitization of the resulting 8 mm film, with only basic editing and the addition of titles carried out using software. The piece was created using two short 8 mm films that were spliced together, and consists of three movements.

The title of the piece: Mezcal Animations #1–3 is a tribute to the International Mezcal Festival held in Oaxaca each year. At the festival, you can sample a wide range of locally produced Mezcal, an alcoholic drink made from agave. The piece uses the brightly coloured materials as a form of homage, in my own way, to the intense colours and flavours of the festival. It also continues to explore some of the æsthetic considerations of my previous works, which use altered states of consciousness as a compositional principle (for further discussion see Weinel 2010 and Weinel 2011). In essence, I consider the brightly coloured, rapidly changing forms and sounds of the piece to evoke a sense of ecstatic hallucination that embodies the qualities of Mezcal; a kind-of Mezcal-themed hallucination. 5[5. It should be noted that Mezcal is not hallucinogenic, though is sometimes presumed to be due to the similarity of the word Mezcal to Mescaline, the hallucinogen contained within certain cacti. Mezcal Animations #1–3 knowingly plays on this confusion.] This approach is similar to the use of direct animation to evoke hallucinatory states used by Storm De Hirsch for her movie Peyote Queen (1965).

Choice of Film

To create the piece I used a Eumig P8 standard 8 mm projector. While many of the direct animation artists I have mentioned used 16 mm, equipment and film for 16 mm production is comparatively more scarce and expensive, so budgeting factors and availability make 8 mm film a more convenient choice. There are also some æsthetic considerations arising from the choice of 8 mm, since the smaller size makes it much more difficult to draw precise or controlled forms on the film. As Loren Means comments, this places the emphasis of the resulting work very much on what Kandinsky describes as the “will of form to matter” (Means n.d.), and works rely significantly on the characteristics of the materials one chooses to apply to the film. I did not consider this problematic since my interest in exploring the technique was in utilising the imperfections, noise and nuanced detail that can be difficult to replicate with purely digital techniques. Direct animation using larger formats such as 16 or 32 mm may be more appropriate for works where carefully controlled forms are sought, and in some cases these formats also provide the option of producing sound materials optically.

Removing Emulsion

To create direct animation films requires the artist to either scratch or draw on existing film, or work with clear leader film (film which is sold to use as a run-in and run-out on movies). Working with developed film, scratching into the black film emulsion will produce streaks of colour and light under projection. Working with clear film, the medium presents a white canvas under projection, to which colours and markings can be applied. For the purposes of Mezcal Animations #1–3, brightly coloured forms, and thus clear film was preferred. Since I was unable to obtain clear standard 8 mm leader, it was necessary to use developed film that was then bleached to remove the emulsion. This can be accomplished easily using household bleach and following appropriate safety precautions; I placed the film in a small bucket of bleach outdoors, avoiding contact between the bleach and my hands. 6[6. The author assumes no responsibility for the safety of anyone choosing to replicate the process described here. Please read the label and observe suitable safety precautions accordingly.] After soaking in bleach for a few minutes, I rinsed the film and disposed of the bleach. The result was clear film.

For Mezcal Animations #1–3, two 8 mm reels were bleached as described. I intentionally allowed parts of the reel to not be fully saturated in the bleach, thus retaining some emulsion for textural effects; these parts are visible in the film (Fig. 1). Other artists using this technique will sometimes use the existing footage on the film as part of the animation, as described previously regarding Len Lye’s Trade Tattoo. The wealth of existing 8 mm films available through online auctions, charity shops and recycling centres provide a large amount of scope for anyone interested in experimenting with this approach; everything from home movies to commercial releases and factual programmes are available to use.

The practicalities of bleaching film or working over existing footage mean that it is necessary to find suitable stock which one does not mind erasing. The scrap films I used showed very few discernable forms, so there were no ethical issues with erasing the film. Other cheaply available films will have existing footage on, which one may not feel comfortable erasing or working over. For those considering engaging in this technique, I would advise not to use reels of particular unique or historic interest as source material. Clear leader film is often the best option, however if unavailable, common mass-produced commercial titles in poor condition are likely to be the cheapest, most ethical option.

Applying Materials

With the reels cleared of emulsion, I applied a mixture of Winsor Newton inks in various colours, ink pen, nail polish and oil paint. As Means comments, it is possible to apply paints or ink which have organic and water bases together, without the colours mixing and blending. This enables the production of some interesting effects with multiple colours. Materials can also be mixed so they run into each other, or layered once the preceding layer has dried. Using acetone enables colours to be thinned and dissolved, which may also produce desirable effects. Spooling the film whilst the ink or paint is still wet can produce decalcomania effects. This causes the layer of ink to be pressed underneath a layer of film, and produces complex patterns, as were used by surrealist painters such as Max Ernst. Alternatively, smoother results can be obtained by allowing the ink to dry naturally in the air. Figures 2–4 show various examples of material produced using techniques such as these.

I added some other elements such as ground coffee, which sticks to the wet ink and adds additional texture and particles. I consciously allowed the films to dry in areas, such as outdoors, where other dust and dirt would be collected. The films therefore show a significant amount of dust particles and even bits of hair (Fig. 5). As discussed previously, other artists have gone much further in applying unusual materials to add texture to their films, such as Brakhage’s use of moths. One aspect that is important to keep in mind when creating a reel with various materials fixed to its surface is that the result must still be able to move successfully through the projector!

Direct animation produces startling and impressive effects, which are likely to appeal to those interested in (visual) noise and abstract expressionism. The process magnifies and illuminates small areas of film in quick succession. The process of magnifying such small areas enables subtle details of the materials to emerge. These may include tiny bubbling effects (Fig. 6), the interaction of different inks or paints, small dust particles or hairs, cracking effects caused by dried paint splitting when deformed by the reels (Fig. 7), or imprints from other materials imprinting on the frame (Figs. 8–9).

There may be noticeable differences between each performance of a reel, since the reel is inevitably altered each time it is touched or played. It was not uncommon for direct animation reels to feed incorrectly through the machine during performance, as the build-up of materials such as paint in sections present the smooth flow of the reel around the tape. It is also possible for the film to break or get stuck and melt through during playback, as was the case with Mezcal Animation #1 during the process of digitization. Direct animation is a process that incorporates temporal and unexpected qualities of the analogue medium. It can be frustrating when many hours are spent preparing a reel, only to have it snap during playback. One approach is to incorporate these fragile aspects of the medium into the design of the work; hence a composition could be structured into movements based on where breaks happened to occur in the film (for example).

Depending on the approach taken when applying materials, it is possible that when reviewing some sections of the film, there may be significant discontinuity between successive frames. This can produce some unusual illusory optical effects when viewing the film, as the brain seems to draw associations between successive forms (such as different dust particles on consecutive frames), causing movement to emerge from general disorder. I found these chaotic sections fatiguing to watch for longer durations however, so I designed some sequences where a simple form remains relatively static or moves across the screen. Such forms are noticeable throughout Mezcal Animation #2 (Fig. 10). Using an ink pen, I carefully draw forms on each frame. These can be inspected frame-by-frame using a viewer-editor; I used a Minette M-1 Viewer Editor. There are limitations on how much control one can have over forms devised in this way, though the variation is considered a desirable aspect of the resulting material. Static forms enable a more cohesive viewing experience.

This section has outlined the main techniques used to produce Mezcal Animations #1–3, though as discussed there are many other techniques and approaches for using direct animation. For further information, the articles “Things to Try: Scratch Film” (PIE Institute 2013) and “Abstract-Expressionist Film Painting” (Means 2013) provide a useful introduction.

Digitizing 8 mm Film

This section provides some general advice for digitizing 8 mm film, which was used prior to this project by Stuart Cunningham for digitizing Super 8 mm films, and subsequently adopted for Mezcal Animations #1–3 on standard 8 mm. 7[7. While Mezcal Animations #1–3 adopts the techniques described, the source film broke during the digitization process. As a result, further refinement of some aspects was not possible, though in general the results obtained were considered satisfactory.]

The options for digitizing 8 mm film range in quality and cost. At the cheapest and perhaps most convenient end of the scale, a cheap “point and shoot” camera with video capability or a smartphone camera can be used. In this case, using automatic settings and holding the camera steady will be sufficient, although this can be difficult and uncomfortable with longer films. This method is sufficient for producing rough drafts and can be useful for evaluating material. Using a higher quality digital video camera or Digital Single Lens Reflex (DSLR) will produce improved results. The most expensive option is to have film digitized professionally, either by paying a specialist or by purchasing specialist equipment to do so oneself. However, direct animation films may leave dirt or even cause damage to certain types of equipment, therefore some studios may be unwilling to accept the job. Devices such as the Optical Printer are the ideal solution, as they enable each frame to be re-photographed individually. However, such devices are seldom used in modern digital film production and are becoming increasingly rare, making convenient, economical access difficult to obtain.

As a cost- and resource-effective option we recommend recording footage independently, or with a willing colleague, using a DSLR camera. We shot footage using a Digital Single Lens Reflex (DSLR) camera in high definition (1080p). DSLRs have become extremely popular in recent years for shooting digital video, due to their high-quality image sensors and their interchangeable lens ability. DSLRs typically have a range of settings, a number of which can be altered to give the truest representation of the 8 mm film. We recommend using a prime lens, if available, rather than a zoom / telescopic lens: primes have fewer lens elements and typically allow larger apertures, meaning they provide higher optical quality and require less assistance to operate in low light.

Once the projector and screen have been set up, the room should be made as dark as possible, ideally pitch black, by the use of curtains, doors, etc. If the projector has several brightness settings available, use the brightest. Next, ensure that you have achieved the best possible, sharpest focus of the 8 mm footage, by adjusting the projector’s focus.

In our own work, the camera was placed close to the projector. If the projector is on a table or stand, an ideal position for the camera is just underneath it. The camera should be set on a stable tripod and aimed at the projection screen with the lens positioned as near as possible to being parallel with the screen.

The mode of the camera should be set to “manual”. This provides complete control of all of the DSLR’s settings. All auto-focusing functions should be disabled and the camera switched to manual focus. Then, using the video preview screen of the DSLR, adjust the zoom of the lens (if available) to fit the projected 8 mm area within the DSLR frame. Depending on the results sought, the 8 mm projection area can be fitted within the DSLR frame (thus capturing all material including borders), or the camera can be focused on a particular portion of the screen (effectively cropping the borders). For Mezcal Animations #1–3, the cropping approach was used to retain the full 1080p resolution and fit the film to a 16:9 aspect ratio without the need to resize digitally. Note that this approach does lose some of the animated material, and for some digitizations it may be preferable to retain the rounded or soft film edges.

The main settings of the camera to be adjusted are, shutter speed, aperture and ISO. Shutter speed should be set to 1/50 of a second (faster speeds reduce the instances where multiple frames blend together). Aperture, as a general rule, should be set to the lowest possible value available (the f value), meaning that you are using the largest aperture that the lens is capable of. In most cases, we used a 50 mm prime lens, which is capable of an aperture of f / 1.8. The ISO value can be adjusted as necessary in order to ensure that the captured image is bright enough with the chosen shutter speed and aperture. In our experience ISO settings of 100 or 200 are usually appropriate. High ISO values, especially over 1600 and 3600, will result in noticeable noise appearing in videos. We recommend recording several test videos with various ISO values to appraise the results.

We recommend shooting all recordings at full HD, 1080p, 25 frames per second (fps), if available on the camera and at the highest imaging or compression quality. There is always the ability to scale down or reduce this after editing; quality can’t be recreated retrospectively. On projectors with variable playback speed, such as the Eumig P8, it is also possible to experiment with slower playback speeds of the 8 mm source material. This can then be sped up in digital software or edited as required. Different playback speeds will also give a different impression for the viewer. Some trial and error will be required initially to make sure that the footage on the DSLR isn’t over- or underexposed and that the 8 mm screen fits suitably within the frame. Afterwards, the video footage can be edited in a preferred video-editing package. For Mezcal Animations #1–3, Adobe After Effects was used.

Creating the Sounds

Designing electroacoustic sound for direct animations presents some interesting challenges. Most of the initial materials that I designed failed to relate effectively to the intensity of movement in the images. Following Danneberg’s suggestion that visual music should not necessarily draw obvious associations with sound (Danneberg 2005, 28), I opted to use a conceptual approach which was derived from my PhD work regarding altered states of consciousness. This compositional technique, which I developed through previous works, is discussed in more detail elsewhere (Weinel 2011; Weinel 2012), however here I describe it briefly.

Essentially, a piece of music is considered as being analogous to a typical hallucinogenic episode, during which visual patterns such as spiral or tunnel forms are experienced at first (Bressloff et al. 2001), before hallucinated scenes, objects and entities emerge from these. Eventually the hallucinated entities subside and are replaced by visual patterning effects, before these also subside and normality is resumed. This form can be used to construct a composition, typically following an A–B–A' structure. In my works of this type, I use rhythmic sounds that are spatialised in circle or spiral patterns, which are considered analogous to the visual patterns of hallucination. “Hallucinated entities” may consist of various electroacoustic sounds such as manipulated voices, spectrally produced tones, or drones. In essence, circular rhythmic patterns provide a basis from which other sounds emerge.

This is the approach that I use for Mezcal Animations #1–3; hence intense rattling and drumming sounds form a basis from which other sounds such as drones appear to emerge. These rhythmic sounds are spatialised in circular and spiral forms. The piece accomplishes this using beta software produced by Rajmil Fischman (Fig. 11). From these rhythmic sounds emerge droning sounds, other gestures and tones. For the visual component of the animation, the effect is similar: from the intense rhythms of the rapidly changing visual images, some more defined forms begin to emerge, such as the static images like those described above.

For the visual component of the animation, the effect is similar: from the intense rhythms of the rapidly changing visual images, some more defined forms begin to emerge, such as the static images like those described above.

In the context of Mezcal Animations #1–3, I find that this technique, where sonic and visual elements emerge from a chaotic audio-visual sensory overload, gives the piece a slightly mystical impression. The piece could be seen in correspondence with trance states, where frenetic drumming and sensory overload serve to invoke altered states. 8[8. The definition of trance states here refers to that which is used by Rouget (1985, 11).] For Mezcal Animations #1–3, the visual and sonic intensity provides a basis from which fleeting moments imitative of hallucination emerge from the work. In the case of this piece, the form these hallucinations take is also Mezcal-themed, and a tribute to the intense flavours of Mezcal.

Concluding Remarks

In this article I have discussed the process of using direct animation for the purposes of producing a piece of electroacoustic visual music. The technique produces materials that exploit the interest of natural variations in real-world materials. For some, direct animation may seem overly messy or uncontrolled as a process. Yet the materials are rich in detail and novelty. Digitally transferring direct animations in high definition retains a good amount of this detail and adds the convenience of digital formats for further manipulation and composition of an electroacoustic soundtrack. As Loren Means comments, perhaps a downside of direct animation is that many of the frames the technique produces are interesting enough to warrant a longer period of viewing than the frame-rate allows. The viewer is given very little time to digest the nuanced detail that can be found in individual frames (Means 2013). Nonetheless, the digitization of these materials opens up many possibilities for further manipulation beyond those approaches I use for Mezcal Animations #1–3. Thus, 8 mm film may be considered among those analogue mediums such as vinyl that continue to hold a sustained interest for many of us, and combined use with digital technologies may yield many exciting compositional possibilities.

Bibliography

Berger, Katherine. “Direct-On-Found Footage Filmmaking: Mining the debris of image consumption & co-directing with nature.” Scan 10/1 (2013). Available online at http://scan.net.au/scn/journal/vol10number1/Katherine-Berger.html [Last accessed 30 April 2014]

Bressloff, Paul C., Jack D. Cowan, Martin Golubitsky, Peter J. Thomas and Matthew C. Wiener. “Geometric Visual Hallucinations, Euclidean Symmetry and the Functional Architecture of Striate Cortex.” Philosophical Transactions of the Royal Society of Biological Sciences 356 (2001), pp. 299–330.

Brighton Royal Pavilion, Museums and Libraries. “Dufaycolor.” Available online at http://www.brighton-hove-rpml.org.uk/HISTORYANDCOLLECTIONS/COLLECTIONSTHEMES/CAPTURINGCOLOUR/Pages/Dufaycolor.aspx [Last accessed 30 April 2014]

Centre for Visual Music. http://www.centerforvisualmusic.org [Last accessed 30 April 2014]

Danneberg, Roger. “Interactive Visual Music: A Personal Perspective.” Computer Music Journal 29/4 (Winter 2005) “Visual Music,” pp. 25–35.

Ganguly, Suranjan. “Stan Brakhage: The 60th Birthday Inteview.” Film Culture 78 (Summer 1994).

Garro, Diego. Patah (2009). Vimeo video “PATAH” (15:26) posted by “Diego Garro” on [?] 2011. http://vimeo.com/14112798 [Last accessed 30 April 2014]

Horrocks, Roger. Len Lye: A Biography. Auckland University Press. 2001.

Means, Loren. “Abstract-Expressionist Film Painting.” [n.d.]. http://www.meansart.com/abstract1.html [Last accessed 30 April 2014]

NFB — The National Film Board of Canada. “Pen Point Percussion: An Introduction to the hand-rawn sounds of Norman McLaren.” Film. [n.d.]. YouTube video “Norman McLaren: Pen Point Percussion” (5:58) posted by “Carlos Bela” on 28 April 2010. http://youtu.be/Q0vgZv_JWfM [Last accessed 30 April 2014]

PIE Institute. “Things to Try: Scratch Film.” Available at http://www.exploratorium.edu/pie/downloads/Scratch_Film.pdf [Last accessed 30 April 2014]

Rouget, Gilbert. Music and Trance: A Theory of the relations between music and possession. Chicago: University Of Chicago Press, 1985.

Russet, Robert and Cecile Starr. Experimental Animation: Origins of a New Art. New York: Da Capo Press Inc. 1976.

Weinel, Jonathan. “Tiny Jungle: Psychedelic Techniques in Audio-Visual Composition.” ICMC 2011: “innovation : interaction : imagination”. Proceedings of the 2011 International Computer Music Conference (Huddersfield, UK: Centre for Research in New Music (CeReNeM) at the University of Huddersfield, 31 July – 5 August 2011).

_____. “Altered States of Consciousness as an Adaptive Principle for Composing Electroacoustic Music.” Unpublished PhD thesis. Keele University, 2012. Available online at http://www.jonweinel.com/phd [Last accessed 30 April 2014]