Focus on Institutions
A column about past, present and future ongoings in international electroacoustic and related institutions [index].
Sharing the Source
A very brief history of computing at CCRMA
In 1964 the first sounds created by computers at Stanford were being calculated by John Chowning and David Poole on an IBM 7090 using Music IV, and played through a shared disk in a PDP1 using the vector scope D/A converters. It was the Computer Music Project. In 1966 computing moved to SAIL (the Stanford Artificial Intelligence Lab) with music software running first on a PDP6 and later in a PDP10 computer (the Music10 sound compiler was Poole’s build of Music IV for the PDP10). But always on borrowed time, very late at night and on weekends, when nobody else was using the computers.
In 1975 CCRMA was created. By 1977 computing had moved to a custom hardware synthesizer, the Samson Box, controlled by a PDP10 computer and terminals scattered through the building. The Samson Box was a hardware synthesizer with real-time control and four audio output channels. In 1980 the computer was replaced by the Foonly, a PDP10 clone. This system was the core of CCRMA’s research and music creation activities for more than twelve years (it was phased out in 1990). Languages for sound synthesis, music composition and dsp research were also developed on those platforms, and a lot of music, projects and research papers came out of them. Software was shared by all, and more uniquely, sound was shared by all as the four audio outputs were connected to all workplaces in parallel.
Eventually this system became too expensive to maintain and at the very end of the 80s CCRMA moved over to its next computing platform, a network of workstations. They were NeXT machines, Unix-based computers with CD quality stereo audio outputs that worked out of the box and had very good support for sound and music applications, something unique at the time. It was very easy to design and code GUI applications and tie them to the music and audio APIs. It even had a Motorola DSP processor which allowed for real-time generation of synthesized sound.
This platform lasted for roughly ten more years, and much more music and research came out of it. The NeXT-based hardware and software platform was also used in many other computer music research centers — a very good way to share and exchange knowledge and experience… and software. We also bought some SGI workstations, many said that was the future platform for graphics and computer music.
NeXT started to fade out. The original sleek black boxes were slowly replaced by PC hardware running the same operating system and software packages. Eventually NeXT failed as a company (although the operating system itself eventually became the core of Apple’s OSX). SGI suffered a similar fate, as PC hardware, and specially graphic cards, became as powerful as expensive supercomputers had been a few years earlier.
In the meantime the Free Software Foundation’s project to create an operating system and programming environment completely based on free software (“free” as in “freedom”), spearheaded by Richard Stallman, got unexpected help from Linus Torvald’s “Linux” kernel, and GNU/Linux was born. In 1996 some graduate students at CCRMA were interested in the new operating system, and we installed Slackware in a couple of workstations that would dual boot between NEXTSTEP and Linux. It was an interesting experiment that proved Linux was becoming a viable platform for our work.
Gradually Linux was integrated into the CCRMA network, and for a while we had three operating systems sharing the same file server and authentication services (NEXTSTEP, Irix and Linux). Over time less and less machines were running NEXTSTEP, and more and more were running Linux, and the SGIs and native NeXT hardware saw less use because they quickly became slow when compared to PC hardware. I think it was around 2002 or 2003 that the lone remaining PC running NEXTSTEP in my office became really unhappy about something in the large NFS file systems in our latest home directory server upgrade, and I turned it off for good.
We also had Macintosh computers at CCRMA almost since the introduction of the Macintosh, mainly in the studios (as we do to this day). Old System 6 and then 7 did not mesh very well with the rest of the Unix CCRMA environment and was very difficult to maintain, but OSX has integrated nicely with our servers (after all, it is just NEXTSTEP and Unix at the core!).
At the time of this writing we have a mixed and well-integrated environment of Linux and OSX computers. The great majority of our computers are high powered multi-core custom built workstations running Linux, and we also have some Macs in the Studios and teaching spaces. The supporting infrastructure of servers and their services is running on free software operating systems and applications.
I’m sure many things are missing in this very short history of computing at CCRMA (for example the Symbolics Lisp workstations CCRMA had, a more detailed look at the evolution of Macs, etc., etc.).
Open Source at CCRMA
Sharing source code as a concept is nothing new, although the rise of GNU/Linux has made it more visible. Bill Schottstaedt tells me that at SAIL (Stanford Artificial Intelligence Labs), as at the MIT AI lab, it was just assumed that all the code was open to anyone. This was a long tradition in the Lisp community, probably predating the AI labs. CCRMA gave the MUS-10 compiler and related software to anyone who asked (IRCAM was an example). The Samson box code was harder to share as it was tied to a custom hardware platform, but Bill remembers assuring Pete Samson that anyone could have it. And Max Mathews always gave the code to Music IV to anyone that asked.
This tradition of open and shared source was maintained at CCRMA throughout the NeXT years. CLM/CMN and CM are examples of computer music languages that were created on the newer platform and gave continuity to all the software that had been developed on the Foonly/Sambox world. Software instruments, algorithms, everything was still there, and improved in most cases. Those languages have remained thriving and open to this day. A lot of other software was developed on the NeXTs using its very good programming, sound and music APIs, some was open source, some closed but still free as in “free beer” (and source was usually available if you asked). The common NeXT platform shared with other computer music institutions all over the world made it easy to share them.
The more standard SGI world was the cradle of Snd, a completely programmable sound editor (Bill Schottstaedt) and computer music language that was originally tied to Motif and X.
In the new world of Linux the initial installs of Slackware gave way to Red Hat as RPM packages made it easy to replicate software installations in all machines. Packages were created for all open source sound, midi and music software that we routinely used in research, music making and teaching. Regretfully software that was tied to the defunct NeXT APIs was slowly lost as they were phased out. Some of the software was developed in house (CLM/CMN and Snd by Bill Schottstaedt, CM by Rick Taube), some at other educational institutions (for example Pd, by Miller Puckette) and many more originated from programmers from all over the world interested in the confluence of sound, music, computers and open source. I started creating custom packages sometime in 1999.
The pre-packaged open source music world we installed in all CCRMA Linux workstations proved to be popular and became Planet CCRMA, a collection of open source software with an emphasis on sound creation and processing. The same software we were using at CCRMA was now available for anyone in the world to download and install (and more locally for students and others to take home and install as well).
Being completely open changed Planet CCRMA, more and more open source packages were added as users requested them from all corners of the world. And vice-versa, projects like CLM or Pd became better known outside academic circles. Everyone benefited from the freedom and wide access made possible by the Internet.
Open Source and Academia
Academia has also been historically at the forefront of the open source world, contributing millions of lines of code and computer science expertise to the ever growing list of packages and systems that is available worldwide.
The open source model is well-matched to academic institutions: the open nature of the software matches the open nature of research. Through an appropriate license such as the GPL (supported legally by copyright law), open source means that the source code of both operating systems and programs is freely available for use, modification and redistribution without additional restrictions — other than passing the same rights to other users when programs are modified and/or redistributed. Software is created and stays free by design, with “free” meaning “freedom.”
The software development and work platform becomes independent from the commercial success or failure of individual hardware or software companies (it is a quite important point, given the fate of NeXT and SGI, and many other — at a time successful — software companies that have disappeared).
Not only the software itself is free (as in “freedom”), the data format of the data created, stored and maintained by the open source programs is also free. The data is no longer tied to a given binary only application, which might in turn be tied to a particular version of an operating system, with both of them eventually becoming obsolete and disappearing, possibly leaving data behind that essentially becomes unreadable or very difficult to convert to other, current formats.
The importance of this should be self-evident for research projects in general, and academia in particular, in particular for long-term projects, the life of which depends on the data, which depends on the availability over time of the software that was used to create and maintain it.
Long-Term Survival
While open source software is not tied to particular companies that might fail, open source projects are tied to individuals or groups that might fade or disappear in time, just as commercial companies do. But the code, through the provisions of its license, survives.
If a project is interesting it will be picked up by another person or group. Even if it is not, as long as the code is archived somewhere in the Internet it will survive, and could eventually be revived if need be.
Cooperation between academic institutions is also enhanced if they use the same free tools and environments, and those tools can be further developed and customized to serve the particular needs of researchers and teachers.
Cost and Support
Besides the “freedom” definition of the word “free” (the important one!), there’s the free as in “free beer” aspect. Open source software is basically free.
There are associated costs in deploying and maintaining it over time. It’s still necessary to do that in the open source world as is the case with commercial software.
There are differences, of course. Open source is often criticized for its apparent lack of support, while the opposite is normally true. Most open source projects have very active user communities with mailing lists, actively maintained wiki pages, online community forums, etc, etc. The speed and depth of real support is sometimes amazing. In my experience much better than commercial companies, specially for difficult problems, which is when support is most valuable.
Overall cost is somewhat more difficult to evaluate and depends on the particulars of each institution. The initial deployment costs with open source software are very small, and compared with commercial software of similar capabilities open source software definitely wins here. Long-term support is also cheaper and most of the time more effective than most paid commercial support options.
Teaching and Research
Open source software has the obvious advantage of cost and availability, making it easy for everyone to have, in their personal computers, the same world and tools as they enjoy in the university work environment, with negligible cost.
For teaching, source code is invaluable when a student is advancing in realms that require original programming: for instance signal processing, HCI (human-computer interaction), MIR (musical information retrieval), new synthesis or interaction models, etc., etc.
For making music, or any computer-based art, some creators are more comfortable if the entire production chain is there in their kit. That’s the case for open source music software, where if needed, it’s literally possible to know how each audio sample is calculated. Of course, it would be a bit tedious were that the only way of working, but for those who need it, the attraction is building their abstraction layers on top of that understanding. These personal “kits” are difficult or impossible to attain in proprietary or commodity systems where layers may be hidden or resistant to customization.
If there is no attraction in building your own personal “kit” from the ground up, there are many well-established open source music and sound computing environments for music creation that add additional layers of abstraction on top of the basics (but usually you can still deal with the bits if you want or need to). To name just a few: Csound, Pd, SuperCollider, cm/clm/cmn, ChucK, etc. A wide variety of paradigms, from purely visual programming languages to object-oriented real-time text programming languages. Very old with a long history and a big community, or very new with exciting opportunities for new research and discovery. And all available to use and modify for free (in both senses of the word).
If “build your own world” is not the way you work, then there are also independent open source tools that work out of the box. Probably the best known is Ardour, a professional quality digital audio workstation package for recording and mixing. But there are also sequencers, drum machines, software synthesizers, effect plugins, audio measurement tools, etc. Mostly very high-quality software created by enthusiasts all over the world. Most of those packages, and pretty much all the environments, use a very low latency highly optimized sound server called Jack. Jack connects applications to the inputs and outputs of sound cards, and most importantly can also connect applications together. Everything can be tied together and can work together.
A big number of these tools are cross-platform. Most run both on the open source GNU/Linux environment and the closed Apple OSX system (which has Unix roots), and some run on the Windows platforms as well. So even if you use a proprietary operating system you can still participate and use the best of the open source world, when it comes to applications and media creation environments.
At CCRMA, custom software programs are often designed and tested in Linux before being deployed cross-platform for use on other operating systems (something which is becoming a common last step). Often, the initial stages of a new approach are more easily researched and understood when all aspects of the computing architecture, including variables like kernel latency, etc. are completely transparent and controllable, something which is only truly possible if all the code involved is open source.
Conclusions
CCRMA is running all of its server computing infrastructure on open source software, and most of our workstations are custom built PCs running Linux. We would not be able to have more than 50 fast workstations throughout our building with a full complement of useful software if it were not for GNU/Linux and the open source world.
There are some not-so-evident advantages to running on a commodity platform and open source software. For example, we are not tied to proprietary hardware platforms and so we were able to design our own custom workstations with no fans. There is no noise in our computer clusters and studios (Macs have to be hidden in closets, of course). That would have been very difficult or impossible in some cases, had we been tied to proprietary hardware which we can’t modify.
We do have Macs running commercial software, mainly in the Studios, but they are outnumbered by the Linux workstations. We also run academic licenses of Matlab in our Linux workstations — although users are migrating and using Octave more and more.
Even so, in our case Open Source software is essential and is the glue that holds everything together.
Thanks
Many many thanks to John Chowning, Bill Schottstaedt and Chris Chafe for their help, information, suggestions and corrections (especially about the early days of CCRMA — I only started taking care of CCRMA’s computers in 1993) that made this article possible.
Social top