eC!

Social top

English

MyPic

A Tool for computer-assisted algorithmic composition

The primary aim of MyPic, a software for computer-assisted algorithmic composition (CAAC), is to provide the composer with an interface for describing and orchestrating musical events via envelopes and auxiliary data for the purpose of driving algorithmic processes. The development of MyPic was inspired by the author’s own experiences in CAAC, which were impeded by a lack of available software to create, organize, orchestrate and manipulate the data that would drive events and algorithms. The application’s features and functionality are presented here, supported by some example applications of MyPic.

It is both unfortunate and necessary that computer-assisted algorithmic composition (CAAC) is a field that one engages in primarily through programming or scripting of some kind. This coding, while allowing the flexibility for a composer to create their own algorithms, algorithmic processes and customizations, adds extra levels of abstraction between the composer and their materials (i.e. pitch collections, rhythmic values and gestural shapes become MIDI note numbers, duration in seconds and normalized envelopes formatted for the programming environment). Given the highly individualized nature of how composers go about their work, it is perhaps understandable that only a limited number of graphic or graphic user interface (GUI) based tools are available for CAAC. In addition, because of the variety of approaches taken in CAAC, these few tools are designed to be very general purpose and universally applicable, and as such have limited functionality. The commonly available tools consist primarily of a MIDI scroll display (or some other similar manner of visualization) and an envelope editor that provides visual feedback and drawing functionality for composing envelopes. The MIDI scroll display’s function is typically to view the generated output and allow the composer to make adjustments to algorithm parameters based on a reading of the visual and audio feedback. The envelope editor allows the composer to easily compose more elaborate envelopes as well as to export the envelope definition to the clipboard or to a text file. The functionality of most envelope editors however is limited: typically it is only possible to compose or edit one envelope at a time and the envelopes themselves are normalized to forced bounds of 0 to 1 on both the x- and y-axes. The development of MyPic is first and foremost an effort to increase the graphic tools available to the algorithmic composer — expanding the capabilities of the above tools as well as adding new methods of organization, feedback and representation.

The primary design goals for the development of MyPic are:

In addition to my own experiments in CAAC, along with graphic score analysis, transcription and representation, and the analysis of music, the writings and works of Iannis Xenakis serve as a primary influence for the development of MyPic. Xenakis’ work on UPIC 1[1. The Unité Polyagogique Informatique CEMAMu is a computerised and graphic-based composition tool conceived by Iannis Xenakis and developed at the Centre d’Etudes de Mathématique et Automatique Musicales (CEMAMu) in 1977.] was of particular importance in designing MyPic’s event model as well as sparking the initial desire for a graphic-driven music composition tool available for CAAC (Xenakis 2001).

MyPic

Event Model

The MyPic event model is an attempt to represent musical events through bundled data: event envelope, group name, five auxiliary envelopes and five auxiliary data fields. Central to each event in MyPic is the event envelope that defines the event in time and pitch space. The group name data point is used as a means of organizing events: channel number, instrument number or name, etc. The auxiliary envelopes and data fields are open-ended and can be used to describe any aspect of the musical event: amplitude, density, deviation, rhythmic values, rest percentage and pitch sieves, among others.

MyPic Interface

Figure 1
Figure 1. Screenshot of the MyPic interface. [Click image to enlarge]

The MyPic interface (Fig. 1) consists of the canvas, toolbar, console and the inspector. The latter includes a tabbed area with windows used to enter auxiliary event envelopes and data fields, to change view and to choose export options. The canvas, where event envelopes are drawn, is modelled after the UPIC and the traditional piano scroll. Here event envelopes are expressed as a series of connected points (xy pairs, where x is time and y is pitch). The toolbar consists of a series of drop-down menus that allow the user to access MyPic’s various drawing and editing tools, as well as canvas scaling and view options. The console is where MyPic prints messages when session information is updated, for example:

event no. 3 updated. => {5.44 72.5 22.44 47.0}

In addition to the guides and rulers that give perspective to the cursor when using the canvas, the inspector lists cursor information (time, MIDI note value, pitch class, octave number and aux values) for the canvas and auxiliary envelope fields, as well as the active (or editable) event number. The tabbed area consists of an auxiliary envelope, auxiliary data, session data, view options and export options tab (the auxiliary information and export tabs are discussed in detail below). The session data tab is simply a place to save session information — whatever that may be. The view options tab is where the user can go to customize the canvas colour scheme.

Event Creation and Manipulation

In MyPic, drawing event envelopes on the canvas creates new events. To facilitate this, there are a number of different design tools available when in edit mode: new event, draw and erase nodes, as well as free hand draw and free hand erase tools. A new event is created by using the new event mode, which produces the simplest event envelope, consisting of only two points. After an event has been created, nodes / points can be added to the event envelope by using the tools available in edit mode. Also, the event envelope can be manipulated as a unit with the move and transform modes.

Auxiliary Envelopes and Data

Figure 2
Figure 2. Auxiliary envelope (left) and auxiliary data (right) tabs. [Click image to enlarge]

In MyPic, the auxiliary envelopes and data tabs are where the auxiliary event information is edited and displayed. These tabs refresh when the user cycles through events to reflect the individual event auxiliary information. The auxiliary envelope tab consists of an edit mode drop-down menu and five auxiliary envelope canvases, each with functionality for copy, paste, reset and y-axis modifications (baseline shift and multiplier). The auxiliary data tab consists of fields for entering an event’s group name and display colour, as well as the five auxiliary data fields.

Exporting and Workflow

While in the export tab, the user can define export options and export event data to the clipboard or to a file. Currently, event data can be exported to variable definitions in SAL and LISP, or as plain text (in .txt format). In addition, there are various options for ordering the exported data and for excluding unused fields. The default export format is a list of lists:

{
{{EventEnv}{GroupName}{AuxEnvs}{AuxData}}
{Next Event}
{Next Event}; [and so on]
}

Here, AuxEnvs and AuxData would be represented as:

{{Aux1}{Aux2}{Aux3}{Aux4}{Aux5}}

As MyPic is only a tool for creating and exporting the above data, the user must configure their CAAC environment or algorithms to accept MyPic’s output. This is usually done by having the exported event list serve as an event or cue list for iterations of an algorithm. For example, by employing a master or conductor process, the user can trigger lower level “instrument” processes according to each event’s GroupName, perhaps making use of EventEnv to define the general trajectory for the event, and AuxEnvs and AuxData for a myriad of other algorithm parameters.

MyPic’s approach of exporting data to drive CAAC in existing algorithmic composition environments is considered optimal, as it takes advantage of existing algorithmic environments and the experience that composers have with them. This allows the composer to simply add MyPic to their arsenal of tools, rather than adding yet another environment for the composer to learn and maintain facility with to use effectively.

Application

So far the applications of MyPic have only been addressed in a general way so that its features and functionality could be expressed without the burden of overly elaborate examples. In this section, two example uses of MyPic will be presented in detail. The first, a Pierrot ensemble work, makes use of the GroupName, three AuxEnvs, and two AuxData fields in addition to EventEnvs to drive algorithms in Common Music/Grace. 2[2. Common Music and Grace (Graphical Real-Time Algorithmic Composition Environment) together form an algorithmic composition environment created by Heinrich Taube. Both are available on the SourceForge website.] The second is a granular synthesis implementation using the EventEnv, GroupName, four AuxEnvs and three AuxData fields to control parameters of Grani+. 3[3. Developed by Kurt Werner in 2010, Grani+ is an extension to Grani (granular synthesis by Fernando López-Lezcano) intended for use with Common Music. Grani+ is available on Kurt Werner’s website.]

Daniel Swilley — Wintermind (2014)

Figure 3
Figure 3. MyPic Canvas snapshot of an excerptof Daniel Swilley’s Wintermind (2014) for chamber ensemble and electroacoustic music. [Click image to enlarge]

The following excerpt is from the ensemble part of a work titled Wintermind, for chamber ensemble and electroacoustic music, in which I primarily make use of sieves, Markov chains and a minimal collection of gestural shapes for the compositional materials.

The canvas screenshot in Figure 3 displays the three EventEnvs that make up the excerpt: the cello in the lowest register, displayed in blue; the woodwinds and violin in the middle register, displayed in green; and the piano in the highest register, displayed in black. The MyPic-to-algorithm parameter mapping for the excerpt is as follows:

MyPic

Algorithm

EventEnv

GestureShape/Pitch

GroupName

InstrumentName

AuxEnv1

AmplitudeEnv

AuxEnv2

RhythmMultiplierEnv

AuxEnv3

RestPercentageEnv

AuxData1

PitchSieve

AuxData2

RhythmCollection and MarkovOrder

The algorithm here makes use of a conductor process, which cycles through the event list and triggers processes for each instrument as specified by the GroupName with the start time and duration provided by the EventEnv. The approach taken for pitch, rhythm and amplitude is the same for each “instrument” process:

Figure 4
Figure 4. Score from Daniel Swilley’s Wintermind showing the results of a MyPic implementation. [Click image to enlarge]
Audio 1 (0:22). Audio excerpt from Daniel Swilley’s Wintermind showing the results of a MyPic implementation.

The score excerpt of Wintermind shown in Figure 4 is the result of the MyPic events and algorithm discussed above. The prominent cello line is a rather transparent example of Eventing’s shape and displays the extensive use of the AmplitudeEnv and RhythmMultiplierEnv for defining the contour and rhythm of the line. In contrast, the accompanying woodwinds and violin parts demonstrate simple trajectories: pitch, a simple ascending motion; rhythm and duration, the orchestrated activity in the first measure and then a trajectory toward ever-shorter values; and amplitude, a clear crescendo. Likewise, the piano part demonstrates clear trajectories toward higher pitch, shorter durations and louder amplitudes.

Granular Synthesis

Figure 5
Figure 5. Screenshot of the MyPic canvas for the creation of a granular synthesis example. [Click image to enlarge]

This granular synthesis example, while making use of more of the software’s parameters than in the Wintermind example, shows a more straightforward use of MyPic and its exported data. Here, most of the data from MyPic is used directly for the input parameters of Grani+. The canvas screenshot shown in Figure 5 displays the five EventEnvs that make up the example. Each event in this example results in a separate execution of Grani+ with the following MyPic-to-algorithm parameter mapping:

MyPic

Algorithm

EventEnv

SampleRateEnv

GroupName

SampleFileName

AuxEnv1

SampleRateSpreadEnv

AuxEnv2

AmplitudeEnv

AuxEnv3

GrainDensityEnv

AuxEnv4

GrainDurationEnv

AuxData1

AmplitudeMultiplier

AuxData2

GrainDensitySpread

AuxData3

GrainDurationSpread

The algorithm here is mostly concerned with routing the appropriate data to Grani+’s various parameters. A conductor process is used to cycle through the event list and trigger the Grani+ calls when indicated. Grani+’s input parameters were addressed as follows:

For clarity of the demonstration, each event or Grani+ execution in this example was based on a sample of a sine tone. This approach was taken in order to allow a rather transparent comparison between the MyPic canvas and a spectrogram of the resulting audio (Fig. 6).

Figure 6
Figure 6. Spectrogram of the resulting audio from the implementation shown in Fig. 5. [Click image to enlarge]

The spectrogram shown in Figure 6 is a -30 to -80 dB resolution of the resulting audio for the MyPic events in Figure 5. The spectrogram demonstrates a clear translation from MyPic EventEnv to the spectral content of the resulting audio. The granular gesture highlighted in grey makes the most dramatic use of sample rate change, amplitude and sample rate spread — sweeping the sample rate down and up 2-1/2 octaves accentuated by amplitude contouring, as well as going from a zero spread at the beginning to a 3-octave spread around 15 seconds before slowly narrowing again by the 30-second point. This gesture also makes use of grain density and grain duration, both starting and ending with smaller values and having an apex that coincides with the sample rate, amplitude and sample rate spread activity. The gesture highlighted in blue starts with an upward sample rate sweep of 2 octaves accentuated by a narrowing in sample rate spread (from an octave to zero), a reduction of grain density (from 20 to 5 per second), and an increase in amplitude. The gesture then maintains a sustain with slight swells of grain duration, grain density and sample rate spread. The gestures highlighted in green, red and orange are all approached in a similar manner: a sample rate spread of zero (except for a slight swell in the sustain), a slow trajectory from smaller to larger values of grain duration and density over the course of the gesture, and an amplitude that has an apex at the start of the sustain.

Summary and Future Work

Including functionality for canvas stretching and compression, importing MIDI data as events, and various features for incorporating randomness into EventEnvs, MyPic in its present state goes well beyond addressing the original objectives of design, functionality and features. While the current version of MyPic is quite stable 4[4. MyPic is available from the author’s website.], has a number of useful features and is being successfully used for CAAC, further development is intended for the future in a number of areas.

The current Event Model allows for great flexibility, yet its make up has limitations for expressing and representing all types of musical events. Future developments here include increasing the number of auxiliary envelope and data fields per event, and providing additional event types to allow for increased flexibility. So far, these new event types include a group event type that can have several voices/layers (envelopes) instead of a single EventEnv, a range event type suitable for expressing boundary envelopes for band-pass and band-reject filtering, and a random distribution event that will give options for executing various random distributions with define ranges as an EventEnv.

Other plans include the addition of an editable envelope library with import and export capabilities that will allow the user to access their preferred envelopes within any MyPic session without having to redraw it, an editable event list interface to allow the entry and editing of events as text, and increased EventEnv transformation capabilities. Lastly, a number of display improvements are intended for the future: a traditional toolbar; auto canvas relocate, resize and add time; more inspector and tabbed area layout options; as well as auxiliary envelope and data field naming.

Bibliography

Xenakis, Iannis. Formalized Music: Thought and mathematics in composition. Hillsdale NY: Pendragon Press, 2001.

Social bottom