MyPic

A Tool for computer-assisted algorithmic composition

by Daniel Swilley

It is both unfortunate and necessary that computer-assisted algorithmic composition (CAAC) is a field that one engages in primarily through programming or scripting of some kind. This coding, while allowing the flexibility for a composer to create their own algorithms, algorithmic processes and customizations, adds extra levels of abstraction between the composer and their materials (i.e. pitch collections, rhythmic values and gestural shapes become MIDI note numbers, duration in seconds and normalized envelopes formatted for the programming environment). Given the highly individualized nature of how composers go about their work, it is perhaps understandable that only a limited number of graphic or graphic user interface (GUI) based tools are available for CAAC. In addition, because of the variety of approaches taken in CAAC, these few tools are designed to be very general purpose and universally applicable, and as such have limited functionality. The commonly available tools consist primarily of a MIDI scroll display (or some other similar manner of visualization) and an envelope editor that provides visual feedback and drawing functionality for composing envelopes. The MIDI scroll display’s function is typically to view the generated output and allow the composer to make adjustments to algorithm parameters based on a reading of the visual and audio feedback. The envelope editor allows the composer to easily compose more elaborate envelopes as well as to export the envelope definition to the clipboard or to a text file. The functionality of most envelope editors however is limited: typically it is only possible to compose or edit one envelope at a time and the envelopes themselves are normalized to forced bounds of 0 to 1 on both the x- and y-axes. The development of MyPic is first and foremost an effort to increase the graphic tools available to the algorithmic composer — expanding the capabilities of the above tools as well as adding new methods of organization, feedback and representation.

The primary design goals for the development of MyPic are:

• To create an interface that allows the user to describe and orchestrate musical events through the use of envelopes (modelled after the UPIC) that could export data for use in CAAC;
• To make available various tools to draw and manipulate the envelopes that represent musical events;
• To expand on the UPIC model by having an event in MyPic consist of a bundle of data to better represent musical events;
• To have MyPic be a universal tool, able to be used in the composition of any CAAC work.

In addition to my own experiments in CAAC, along with graphic score analysis, transcription and representation, and the analysis of music, the writings and works of Iannis Xenakis serve as a primary influence for the development of MyPic. Xenakis’ work on UPIC 1[1. The Unité Polyagogique Informatique CEMAMu is a computerised and graphic-based composition tool conceived by Iannis Xenakis and developed at the Centre d’Etudes de Mathématique et Automatique Musicales (CEMAMu) in 1977.] was of particular importance in designing MyPic’s event model as well as sparking the initial desire for a graphic-driven music composition tool available for CAAC (Xenakis 2001).

MyPic

Event Model

The MyPic event model is an attempt to represent musical events through bundled data: event envelope, group name, five auxiliary envelopes and five auxiliary data fields. Central to each event in MyPic is the event envelope that defines the event in time and pitch space. The group name data point is used as a means of organizing events: channel number, instrument number or name, etc. The auxiliary envelopes and data fields are open-ended and can be used to describe any aspect of the musical event: amplitude, density, deviation, rhythmic values, rest percentage and pitch sieves, among others.

MyPic Interface

The MyPic interface (Fig. 1) consists of the canvas, toolbar, console and the inspector. The latter includes a tabbed area with windows used to enter auxiliary event envelopes and data fields, to change view and to choose export options. The canvas, where event envelopes are drawn, is modelled after the UPIC and the traditional piano scroll. Here event envelopes are expressed as a series of connected points (xy pairs, where x is time and y is pitch). The toolbar consists of a series of drop-down menus that allow the user to access MyPic’s various drawing and editing tools, as well as canvas scaling and view options. The console is where MyPic prints messages when session information is updated, for example:

event no. 3 updated. => {5.44 72.5 22.44 47.0}

In addition to the guides and rulers that give perspective to the cursor when using the canvas, the inspector lists cursor information (time, MIDI note value, pitch class, octave number and aux values) for the canvas and auxiliary envelope fields, as well as the active (or editable) event number. The tabbed area consists of an auxiliary envelope, auxiliary data, session data, view options and export options tab (the auxiliary information and export tabs are discussed in detail below). The session data tab is simply a place to save session information — whatever that may be. The view options tab is where the user can go to customize the canvas colour scheme.

Event Creation and Manipulation

In MyPic, drawing event envelopes on the canvas creates new events. To facilitate this, there are a number of different design tools available when in edit mode: new event, draw and erase nodes, as well as free hand draw and free hand erase tools. A new event is created by using the new event mode, which produces the simplest event envelope, consisting of only two points. After an event has been created, nodes / points can be added to the event envelope by using the tools available in edit mode. Also, the event envelope can be manipulated as a unit with the move and transform modes.

Auxiliary Envelopes and Data

In MyPic, the auxiliary envelopes and data tabs are where the auxiliary event information is edited and displayed. These tabs refresh when the user cycles through events to reflect the individual event auxiliary information. The auxiliary envelope tab consists of an edit mode drop-down menu and five auxiliary envelope canvases, each with functionality for copy, paste, reset and y-axis modifications (baseline shift and multiplier). The auxiliary data tab consists of fields for entering an event’s group name and display colour, as well as the five auxiliary data fields.

Exporting and Workflow

While in the export tab, the user can define export options and export event data to the clipboard or to a file. Currently, event data can be exported to variable definitions in SAL and LISP, or as plain text (in .txt format). In addition, there are various options for ordering the exported data and for excluding unused fields. The default export format is a list of lists:

{
{{EventEnv}{GroupName}{AuxEnvs}{AuxData}}
{Next Event}
{Next Event}; [and so on]
}

Here, AuxEnvs and AuxData would be represented as:

{{Aux1}{Aux2}{Aux3}{Aux4}{Aux5}}

As MyPic is only a tool for creating and exporting the above data, the user must configure their CAAC environment or algorithms to accept MyPic’s output. This is usually done by having the exported event list serve as an event or cue list for iterations of an algorithm. For example, by employing a master or conductor process, the user can trigger lower level “instrument” processes according to each event’s GroupName, perhaps making use of EventEnv to define the general trajectory for the event, and AuxEnvs and AuxData for a myriad of other algorithm parameters.

MyPic’s approach of exporting data to drive CAAC in existing algorithmic composition environments is considered optimal, as it takes advantage of existing algorithmic environments and the experience that composers have with them. This allows the composer to simply add MyPic to their arsenal of tools, rather than adding yet another environment for the composer to learn and maintain facility with to use effectively.

Application

So far the applications of MyPic have only been addressed in a general way so that its features and functionality could be expressed without the burden of overly elaborate examples. In this section, two example uses of MyPic will be presented in detail. The first, a Pierrot ensemble work, makes use of the GroupName, three AuxEnvs, and two AuxData fields in addition to EventEnvs to drive algorithms in Common Music/Grace. 2[2. Common Music and Grace (Graphical Real-Time Algorithmic Composition Environment) together form an algorithmic composition environment created by Heinrich Taube. Both are available on the SourceForge website.] The second is a granular synthesis implementation using the EventEnv, GroupName, four AuxEnvs and three AuxData fields to control parameters of Grani+. 3[3. Developed by Kurt Werner in 2010, Grani+ is an extension to Grani (granular synthesis by Fernando López-Lezcano) intended for use with Common Music. Grani+ is available on Kurt Werner’s website.]

Daniel Swilley — Wintermind (2014)

The following excerpt is from the ensemble part of a work titled Wintermind, for chamber ensemble and electroacoustic music, in which I primarily make use of sieves, Markov chains and a minimal collection of gestural shapes for the compositional materials.

The canvas screenshot in Figure 3 displays the three EventEnvs that make up the excerpt: the cello in the lowest register, displayed in blue; the woodwinds and violin in the middle register, displayed in green; and the piano in the highest register, displayed in black. The MyPic-to-algorithm parameter mapping for the excerpt is as follows:

The algorithm here makes use of a conductor process, which cycles through the event list and triggers processes for each instrument as specified by the GroupName with the start time and duration provided by the EventEnv. The approach taken for pitch, rhythm and amplitude is the same for each “instrument” process:

• Pitch — The value for each note is selected by looking up the start time of the note in the EventEnv and then selecting the closest PitchSieve value. This is if the note was not determined to be a rest by looking up the start time in the RestPercentageEnv and calculating the odds of a rest.
• Rhythm — The value for each note is the result of a selected value and a multiplier. The initial value is selected via a Markov process that is derived from analysing a collection of rhythmic values. The multiplier is obtained by looking up the start time of the note in the RhythmMultiplierEnv.
• Amplitude — The value is selected by looking up the start time of the note in the AmplitudeEnv.

The score excerpt of Wintermind shown in Figure 4 is the result of the MyPic events and algorithm discussed above. The prominent cello line is a rather transparent example of Eventing’s shape and displays the extensive use of the AmplitudeEnv and RhythmMultiplierEnv for defining the contour and rhythm of the line. In contrast, the accompanying woodwinds and violin parts demonstrate simple trajectories: pitch, a simple ascending motion; rhythm and duration, the orchestrated activity in the first measure and then a trajectory toward ever-shorter values; and amplitude, a clear crescendo. Likewise, the piano part demonstrates clear trajectories toward higher pitch, shorter durations and louder amplitudes.

Granular Synthesis

This granular synthesis example, while making use of more of the software’s parameters than in the Wintermind example, shows a more straightforward use of MyPic and its exported data. Here, most of the data from MyPic is used directly for the input parameters of Grani+. The canvas screenshot shown in Figure 5 displays the five EventEnvs that make up the example. Each event in this example results in a separate execution of Grani+ with the following MyPic-to-algorithm parameter mapping:

The algorithm here is mostly concerned with routing the appropriate data to Grani+’s various parameters. A conductor process is used to cycle through the event list and trigger the Grani+ calls when indicated. Grani+’s input parameters were addressed as follows:

• Start-time — The start time of the EventEnv.
• Duration — The duration of the EventEnv.
• Amplitude — AmplitudeMultiplier.
• File — The filename from GroupName.
• Amp-envelope — AmplitudeEnv.
• Grain-envelope — Determined by a function using weighted randomness to pick from a collection of envelopes.
• Grain-duration — GrainDurationEnv with the per-grain random wiggle value GrainDurationSpread.
• Srate — SampleRateEnv with the per-grain random wiggle value SampleRateSpreadEnv.
• Grain-density — GrainDensityEnv with the per-grain random wiggle value GrainDensitySpread.

For clarity of the demonstration, each event or Grani+ execution in this example was based on a sample of a sine tone. This approach was taken in order to allow a rather transparent comparison between the MyPic canvas and a spectrogram of the resulting audio (Fig. 6).

The spectrogram shown in Figure 6 is a -30 to -80 dB resolution of the resulting audio for the MyPic events in Figure 5. The spectrogram demonstrates a clear translation from MyPic EventEnv to the spectral content of the resulting audio. The granular gesture highlighted in grey makes the most dramatic use of sample rate change, amplitude and sample rate spread — sweeping the sample rate down and up 2-1/2 octaves accentuated by amplitude contouring, as well as going from a zero spread at the beginning to a 3-octave spread around 15 seconds before slowly narrowing again by the 30-second point. This gesture also makes use of grain density and grain duration, both starting and ending with smaller values and having an apex that coincides with the sample rate, amplitude and sample rate spread activity. The gesture highlighted in blue starts with an upward sample rate sweep of 2 octaves accentuated by a narrowing in sample rate spread (from an octave to zero), a reduction of grain density (from 20 to 5 per second), and an increase in amplitude. The gesture then maintains a sustain with slight swells of grain duration, grain density and sample rate spread. The gestures highlighted in green, red and orange are all approached in a similar manner: a sample rate spread of zero (except for a slight swell in the sustain), a slow trajectory from smaller to larger values of grain duration and density over the course of the gesture, and an amplitude that has an apex at the start of the sustain.

Summary and Future Work

Including functionality for canvas stretching and compression, importing MIDI data as events, and various features for incorporating randomness into EventEnvs, MyPic in its present state goes well beyond addressing the original objectives of design, functionality and features. While the current version of MyPic is quite stable 4[4. MyPic is available from the author’s website.], has a number of useful features and is being successfully used for CAAC, further development is intended for the future in a number of areas.

The current Event Model allows for great flexibility, yet its make up has limitations for expressing and representing all types of musical events. Future developments here include increasing the number of auxiliary envelope and data fields per event, and providing additional event types to allow for increased flexibility. So far, these new event types include a group event type that can have several voices/layers (envelopes) instead of a single EventEnv, a range event type suitable for expressing boundary envelopes for band-pass and band-reject filtering, and a random distribution event that will give options for executing various random distributions with define ranges as an EventEnv.

Other plans include the addition of an editable envelope library with import and export capabilities that will allow the user to access their preferred envelopes within any MyPic session without having to redraw it, an editable event list interface to allow the entry and editing of events as text, and increased EventEnv transformation capabilities. Lastly, a number of display improvements are intended for the future: a traditional toolbar; auto canvas relocate, resize and add time; more inspector and tabbed area layout options; as well as auxiliary envelope and data field naming.

Bibliography

Xenakis, Iannis. Formalized Music: Thought and mathematics in composition. Hillsdale NY: Pendragon Press, 2001.