Sonification as Semblance

A phenomenological investigation into music composition

by Joshua Horsley

Within my praxis, the sonification of corporeal spatial occupancy is primarily concerned with semblance: what Merleau-Ponty (1968) differentiates from “body schema” as “body image” or the perceptible form of an object, especially when distinct from its object reality. Semblance, whilst seemingly abstract, adheres to those principles characteristic of sonification, specifically the first condition of Hermann’s definition, which states: “A technique that uses data as input, and generates sound signals… may be called sonification, if and only if [the] sound reflects objective properties or relations in the input data.” Yet for numerous (and in many cases valid) reasons, much sonification practice fails to adhere to this condition; the sonified object is considered so far as its spatiality is concerned; yet to truly reflect objective properties, and for semblance, the object should be considered as it exists spatiotemporally. However, sonification that engages with the principles of semblance (thereby reflecting objective properties of the sonified object) is problematic within creative arts practice by complication of the apparent limit upon qualitative compositional decisions. Here we consider sonification praxis from a philosophical perspective, specifically: through a dialogic process of music practice and existential phenomenology, so to contribute to the fields of philosophy and auditory display, and to the wider remit of music praxis.

Semblance: The Perceptual Immanence of Corporeal Spatial Occupancy

What is the existential status of corporeal spatial occupancy? It is understood that corporeal spatial occupancy that is of constancy in its external conscious perceptibility (its outward appearance) and its spatial-temporality is toward Object Reality. Merleau-Ponty states:

In all of its appearances, the object preserves its invariable characteristics and itself remains invariable, and it is an object because all of the possible values that it could assume in terms of size and form are contained in advance in the formula of its relations with the context. (Merleau-Ponty 1945, 314)

Maintaining this, Merleau-Ponty’s reflection that “the constancy of relations is grounded in the evidentness of the thing” (Merleau-Ponty 1945, 315) is of significance to the discussion and process of sonification. One could observe that with altered context and history, Merleau-Ponty’s statements present sonification praxis with learned clarity. As a differentiation between external perceptual consciousness and spatial-temporality, Merleau-Ponty’s statement facilitates consideration beyond existential status within Object Reality: existential status of corporeal spatial occupancy within phantasy, image consciousness, and memory, within realities. In doing so, the complex phenomena of sonification is uncovered. Many experience phantasy yet it is those deemed as creative or innovative who bring theirs forth as perceptual immanence. Critique of the above statement is not laboured; it is bold and romantic in its apparent location within creative arts practice, yet it demonstrates Husserlian phantasy and provides differentiation from likewise internal apprehensions, such as image consciousness, and memory. Phantasy then, in the context of music practice, is composition realised, or heard internally, prior to any external realisation. As recognised by Husserl, phantasy is warrant of analysis: despite being an apprehension of objectivities (akin to perception) it is essentially subjective and embodied, in lieu of which, questions arise with regards to its status post-external realisation. If phantasy is brought forth as perceptual immanence, does its status remain subjective? Does it maintain the status of phantasy at all? Those questions lead toward one of substantial significance within this study: if that which is brought forth as perceptual immanence pre-exists as perceptual immanence (as corporeal spatial occupancy), yet has existed neither as perceptual immanence nor as phantasy, is it the pre-existing perceivable phenomenon? To clarify: if audio is composed through data mapping of corporeal spatial occupancy and subsequent allocation of data to audio parameters therefore has no previous existence as audio within external space and time or internal phantasy, to what extent is the realised audio of the corporeal spatial occupancy? To what extent can a sonification be of the same essential qualities as that which is sonified? To the extent of semblance!

At this stage, it is of value to distinguish the discussed entities of audio, sonification and corporeal spatial occupancy, through terminological congruence with phenomenological discourse, and in doing so, to allow some of the previous and potential questions to become redundant. Differentiating phantasy presentation and physical imagination yet recognising them as parallel cases, Husserl defines entities of physical imagination in terms of image perception. Although of vast difference in perceptual immanence and constancy, congruence is found with audio in terms of externality:

We have three objects: 1) the physical image, the physical thing made from canvas, marble, and so on; 2) the representing or depicting object; and 3) the represented or depicted object. For the latter, we prefer to say simply “image subject”; for the first object, we prefer “physical image”; for the second, “representing image” or “image object.” (Husserl 2005, 21)

Thereby, positioning Husserl’s distinctions in the audio-musical domain (and in interest of clarity):

1. Audio that is externally perceived within space is referred to as “physical image”;
2. Audio that is a sonification is referred to as “image object”;
3. Corporeal spatial occupancy is referred to as “image subject”.

However, physical imagination is then concerned with predisposition; it is an internal realisation that is a re-presentation of an external entity. Phantasy exists without having a previous object reality of its own; it is image subject sans or prior to image object and physical image. Thus, sonification understood as image subject requires externality beyond that of physical image — it does not exist as image consciousness, nor does phantasy, rather it is precisely existential semblance. The semblance encountered within a sonification is recognised as essentially phenomenological in lieu of the observation that there is absence of internality within the process. Sonification is essentially as Heidegger (1927, 58) states: “to let that which shows itself be seen from itself in the very way in which it shows itself from itself.” In many cases, what is understood as artistic creativity — phantasy brought forth — does not exist within this process. Rather, it is through the definition of the methodical construct of a sonification and the relationship it has with corporeal spatial occupancy that that which is brought forth, is.

The Problems of Semblance

Adhering to true semblance is problematic in regards to æsthetic engagement with a realised sonification of corporeal spatial occupancy. Existing all at once in its spatial occupancy, an object shares this same property within its temporal occupancy, whereby the data, no matter its numerical value upon a three-dimensional axis, exists simultaneously. True semblance dictates that however liberal the choice of duration for the realised sonification, sonification that adheres to semblance, or to Hermann’s (2008) definition, must be realised as a single static chord. Discrepancies with regards to the implications of semblance upon validity of compositional praxis were illuminated subsequent to a period of critical reflection upon my own compositional outputs. Reflection evidenced that Sedemus (2012) and Sedere, Audire (2013) were fundamentally flawed in their pursuit of an audio realisation of non-audio objective properties of semblance. Their methods utilised duration as a compositional parameter to facilitate reading of the object (for example, from left to right). Thereby, duration was enabled as a parameter and facilitated the musical quality of unfolding. However, the musical qualities were demonstrated at the expense of invalidation of the “image subject” — because image subject exists spatiotemporally and reflection of the objective properties would need to exist simultaneously.

By utilising duration to acquire musical unfolding, Sedemus and Sedere, Audire depict the image subject as unfolding with each second of audio depicting an aspect of the object in sequence (akin to Magnetic Resonance Imaging [MRI]). These works thereby reflect objects within spatial succession and not the true spatiotemporal homogeneity of the objects. The direction of reading and the use of duration as a parameter impose spatiality upon time. Philosophical engagement with sonification that demonstrates the image subject as unfolding warrants study of the work of Franz Brentano as well as Husserl’s (1991b, 29) subsequent elucidation, namely “The Diagram of Time”. Sonification that utilises the same methodical construct as Sedemus and Sedere, Audire demonstrates the running-off phenomena of the theory of time; aptly, Husserl locates his analogy within musical melody when he states: “When the new tone is sounding, the preceding tone has not disappeared without leaving a trace. If it had, we would be quite incapable of noticing the relations among the successive tones” (Ibid., 11). To elucidate: running-off phenomena within the perception of time are evident when engaging with a melody within a given key; the key is only defined when the tonic, dominant and median are heard in any sequence. The notes that are no longer present or sounding are maintained within consciousness 1[1. Interestingly, duration as consciousness of engagement is not whole, because when time’s running-off phenomena is considered in terms of Dasein’s Being — as-lived — previous notes are maintained in memory and this is evident as the sudden consciousness (or abstract conceptualisation) of any “wrong” note within a melody.], otherwise one would not perceive the key of the music. Whilst representing the continua of the running-off phenomena of (the perception of) time, such sonification demonstrates discrepancies with regards to semblance insofar as it misrepresents the continua of spatial relations within time as successive. Therein, spatial past is contained within time, whereas corporeal spatial occupancy exists simultaneously within the ever-present. It is the imposition of duration as a period of conscious engagement that begets this misrepresentation. In duration, engagement acts as an abstract conceptualisation of objects and thereby reduces them. Rather it is through a priori experience akin to Bergson’s metaphysical — “intuition” (1903) and Heidegger’s existential phenomenological — “as-lived” (1927) — that knowledge is “to the things themselves.”

When sonification adheres to semblance (whereby realisation is heard as a single static chord) compositional qualities are compromised (Video 1).

Sonification as a single static chord is problematic in terms of functionality. Concerning the wider remit of auditory display, differential perception of distinct data is nigh impossible; thus, with regards to interpretive communication, adherence to semblance renders sonification for this purpose inept. With regards to compositional potential, outcomes are attached to parameter allocation and therefore æsthetic sophistication is severely limited to the inherent sonic permeations within a chord. Qualitative compositional decisions (for example, where sound or music that exists first as heard within internal consciousness — as with Husserlian phantasy — is brought forth as perceivable externality) are discounted in favour of integrity of process towards quantifiable outcome. In this manner, sonification is an Objective creative praxis. The success of any sonification is quantifiable as representational accuracy, adherence to methodical construct, reflection of Objective properties, and as semblance. Yet, despite the apparent inconveniences of semblance — specifically in the representation of temporal occupancy in the sonification of corporeal spatial occupancy — there remains substance and quality within this praxis beyond integrity of process and truth in representational accuracy.

Sonification as Quantifiable

Speculating about the extent to which sonification is quantifiable as a reflection of Objective properties poses numerous questions. Those questions pertaining to method and process can be addressed with clarity. However, it is the realised audio, the image object, for which the supposition of quantification presents deeper issues. Qualitative appreciation of audio is predominantly subsumed to æsthetics wherein much discussion regarding differentiation of sound from music is obstructed by deviations toward positions of subjective æsthetical interpretation; perhaps understandably, given the oft passionate subjectivity associated with music. Differentiation of sound from music is of pertinence within the wider remit of sonification praxis but the qualitative orientation of sonification as semblance pertains specifically to spatial-temporality and this shifts focus from æsthetics to existential phenomenology and metaphysics.

The assertion of the presence of qualitative attributes within a practice for which quantifiable outcome is tantamount is of apparent disparity. It is of value to enter into discussion of quality and quantity to permit discourse that addresses semblic sonification in terms of reality, or realities. Heidegger’s footnoted address of what he deems to be the failings of Henri Bergson (in addition to those of Aristotle, Kant, Hegel, and Husserl by association) is concerned with the observation that Bergson’s temporal analysis is preceded by an analysis of number; thereby Bergson’s statement that time is of qualitative succession is in fact a statement that time is of quantitative succession (Heidegger 1927). Within his address, Heidegger states his intention to elucidate his objections to Bergson’s temporal philosophy within part two of Being and Time. 2[2. Heidegger never completed the second part of Being and Time.] Despite this inconvenience, Heidegger’s position, that time is of quality to the extent of being unquantifiable and irreducible to numerical measurement, resonates with the position explicated by Bergson’s (1889) “pure duration” and his own (1903) “intuition”. For Bergson, knowledge through analysis is relative as opposed to absolute; in reducing corporeal spatial occupancy to data-based representation, the object in its actuality is invalidated, and what is gained is knowledge of an abstract conceptualisation of the object, as opposed to the object as absolute. Sonification as semblance is not toward absolute knowledge; rather its purpose is specifically to contribute relative knowledge from the position of audio. It is therefore recognised that discrepancies with regards to the audio realisation of non-audio data are innate. However, the heterogeneous hierarchy of object space (quantity) and time (quality) is impermanent: spatial data realised as audio data may rather be described as spatial data in time realised as temporal data in space. Therefore, the innate discrepancies should not serve as justification for a resignation to relative knowledge in stasis. Sonification practice concerned with the audio realisation of corporeal spatial occupancy should endeavour to revise its methods to better lay claim to its conditions of definition, thereby augmenting the integrity of the practice. Such sonification is knowledge relative to the invalidation of the sonified object, as it exists spatiotemporally, whereas sonification that adheres to semblance is relative knowledge — relative to the innate discrepancies imposed by the dominating concept of the space-time continuum.

Sonification as Semblance

Therefore, without invalidating the object as it exists spatiotemporally, what is the qualitative potential of sonification as semblance? To address the preservation of the image subject as it exists spatiotemporally, credence is given to the appreciation that the non-audio data is (as with the previous imposition of hierarchy) primarily spatial and the audio data primarily temporal. When temporality is utilised as a parameter to enable qualitative compositional decisions, it is exploited as a contrivance that amounts to a sequential arrangement of properties within duration; duration is not an audio parameter, rather it is a facilitator, a container for engagement. Therefore, in the first instance, the method for the realisation of semblic sonification cannot use duration (of temporality) as a parameter; thereby, semblic sonification is composed of chord(s). To an extent, compositional potential arises upon the sonification of multiple occupancies within the same spatial environment; manipulation of spatial forms facilitates the compositional realisation of complex harmonies. However, such composition is subject to temporal limitations: as static chord there is no change beyond those inherent permeations within sonic relationships — parameter values are unchanged for the duration. If music unfolds within time — i.e. sound parameter values are unfolding, and therefore changing within a given duration facilitate musicality — it is then change that occurs within duration that permits the liberation of qualitative compositional decisions.

Thus, in returning to spatial occupancy it is locational change, or rather motion within time, that facilitates the unfolding and therefore sonification of musical qualities, and it does so whilst reflecting objective properties and adhering to semblance (Video 2).

Temporality is an existential essential for both sound and music but, notably, it is only when sonification takes account of temporality that its sonority becomes indisputably musical. It is potential for change that facilitates the demonstration of qualities and leads to the suggestion that temporality is the essence of music. My praxis posits temporality as of qualities yet the credence placed upon temporality is founded upon phenomenological analyses of spatiality, hitherto leading to disparities. Furthermore, spatial and temporal data are presented as (interpreted from) numerical quantification and, whilst duration is cited as relatively arbitrary in that it is a period of engagement with the unquantifiable quality of presence, it is simultaneously rejected as a position from which to accumulate knowledge. Duration presents further issues, given its significance upon the introduction of motion, and perhaps there is room for some debate as to whether duration is a spatial phenomenon. Upon the imposition of duration as a parameter (recognized as the instance that it enters conscious account and is no longer perceived to be as-lived), does it transcend its apparent location within temporal phenomena toward spatiality?

Bibliography

Bergson, Henri. “Introduction à la Métaphysique.” Revue de métaphysique et de morale (January 1903).

_____. Time and Free Will: An essay on the immediate data of consciousness. Trans. George Allen New York: Muirhead, 1910.

Heidegger, Martin. Sein und Zeit. Trans. John Macquarrie and Edward Robinson. New York: Harper and Row, 1962.

Hermann, Thomas. “Taxonomy and Definitions for Sonification and Auditory Display.” ICAD 2008. Proceedings of the 14th International Conference on Auditory Display (Paris: IRCAM, 24–27 June 2008). http://icad08.ircam.fr

Husserl, Edmund. “Analysis of the Consciousness of Time.” On the Phenomenology of the Consciousness of Internal Time (1893–1917). Trans. John Barnett Brough. International: Kluwer 1991.

Husserl, Edmund. “Brentano’s Theory of the Origin of Time.” On the Phenomenology of the Consciousness of Internal Time (1893–1917). Trans. John Barnett Brough. International: Kluwer 1991.

Husserl, Edmund. “Principal Parts of the Phenomenology and Theory of Knowledge.” Phantasy, Image Consciousness, and Memory (1898–1925). Trans. John Barnett Brough. Berlin: Springer 2005.

Merleau-Ponty, Maurice. Phenomenology of Perception. London: Rutledge, 1945.

_____. The Visible and the Invisible. Evanston IL: Northwestern University Press, 1968.