Right after Christmas 2014, I started to work on a new project for the Museum of Modern Art in New York City. In March, a retrospective exhibition about the singer Björk will launch. As you might expect, the exhibition will use a lot of new, cutting-edge technologies. I am very happy with the fact that Klangerfinder was asked to participate in the production, which means that I have to handle most of the audio programming tasks.

Since this is – again – a commercial project, I cannot blog about any details about my work. However, I can assure you that it involves almost anything I have learned so far in computer science, composition and sound design ;)



This month, I am finally able to present the result of my biggest project this year:


The Android app “Sound Journey” was created on behalf of Volkswagen. My work was part of my job at Klangerfinder, the company who developed the audio engine for this app with me being the lead programmer.  Unfortunately, the app does not work in every modern car, just in the newest models of VW. However, if you own such a car, you can download the app for free and give it a try. The music changes in real-time, based on your interaction with the car. This project certainly pushed the boundary of what I thought was possible with Android phones and sound.

While preparing for a concert next week, I tried out a new pd abstraction with my laptop’s internal microphone. One can create a lot of different sounds by hitting the laptop, the touchpad or just by typing. The “aaron”-abstraction (a reference to the famous “moses” object in pd) loops the input indefinitely for the specified amount of milliseconds. The options on the right inlet are: record, overdub, play loop.

The aaron-abstraction is pretty small, so I just paste it here. Just copy it in your favorite text editor and save it as a *.pd file.

#N canvas 0 0 1596 817 10;
#X obj 666 319 *~;
#X obj 795 308 *~;
#X obj 628 241 sig~;
#X obj 812 246 sig~;
#X obj 588 487 *~;
#X obj 518 369 *~;
#X obj 485 282 inlet~;
#X obj 439 535 outlet~;
#X obj 685 41 sel 0 1 2;
#X obj 680 205 unpack 0 0;
#X msg 656 86 1 0;
#X msg 684 126 1 1;
#X msg 746 152 0 1;
#X text 839 86 record;
#X text 839 116 overdub;
#X text 836 146 play loop;
#X obj 751 383 delwrite~ d1 \$1;
#X obj 618 403 delread~ d1 \$1;
#X obj 689 7 inlet;
#X connect 0 0 16 0;
#X connect 1 0 16 0;
#X connect 2 0 5 1;
#X connect 2 0 0 1;
#X connect 3 0 1 1;
#X connect 3 0 4 0;
#X connect 4 0 7 0;
#X connect 5 0 7 0;
#X connect 6 0 5 0;
#X connect 6 0 0 0;
#X connect 8 0 10 0;
#X connect 8 1 11 0;
#X connect 8 2 12 0;
#X connect 9 0 2 0;
#X connect 9 1 3 0;
#X connect 10 0 9 0;
#X connect 11 0 9 0;
#X connect 12 0 9 0;
#X connect 17 0 1 0;
#X connect 17 0 4 1;
#X connect 18 0 8 0;
Ohhhh, this post was due a long time ago… Anyways, here comes part two of my thoughts on loudness.
First, a correction: modern classical orchestras usually tune 440 or 442, they do not go up to 445 anymore.
I personally think that the reasons for reduced dynamics in story arcs, film music and tv programmes have their roots in the developments of audio and video engineering in the 90s. Digital mixing lead to rapidly increasing loudness levels, resulting in the decrease of micro- and macro dynamics. One cannot blame the engineers for that “one louder” behavior, since they would have lost their clients if they would not have complied. Today, more producers care for dynamics instead of loudness, so the loudness war might come to an end bit by bit. The effects on other “loudness problems” as discussed in the previous post however cannot be reverted easily – we have to live with them now.
Another reason behind the urge to produce “loud” pieces of media (e.g., videos with a high scene cut rate) might be the loss of control on the context in which the media is presented. For instance, a YouTube video might be watched on many different kinds of devices, half of them having shitty displays, half of them having shitty speakers. Therefore, video producers try to create something which works on all possible devices, like audio engineers mix music which does even sound good on the cheapest playback system you can imagine. In music, this goal results in louder mixing, in video production, the need to support many devices and viewing contexts results in “loud” (i.e., intensive) videos.
Luckily, there are settings where artists still have control of the context, like concerts, exhibitions, special events and so on – basically every occasion where the art piece is connected to a specific context like a concert hall, museum or gallery.

In music, one could move away from conventional formats like mp3 and release material on different platforms and in different ways. Interactive music has a promising future, because it can be tied to interesting contexts, like games or everyday objects, and provides a longer experience before the listener (or player) gets bored. Additionally, interactive music is simply a consequence of the larger capabilities of playback devices. For instance, compare a minidisc player from the 90s to nowadays smarthpones. Why should we still use static mp3s on mini-computers with a quadcore CPU and 1 GB of RAM? Interactive music does not need to scream for attention, because it is either meant to stay in the background anyway (which is fine) or easily captures the listener’s attention because it is meant to be played with. The engineered loudness problem would also be partially solved since the interactive content cannot be completely mixed and mastered in advance. Even mastering the audio in realtime is not an option since it is not (yet?) feasible to implement the whole mastering chain on a consumer device.

If you are a passionate music listener or musician you are very likely confronted with the problem of “loudness” in contemporary music. I am not talking about high sound pressure levels at rock concerts – that is another story – but about flat (=compressed), loud recordings on CDs and the radio. The problem started in the early 90s when digital brickwall limiters were introduced and was later dubbed the “loudness war”. Based on the assumption that a record sells better if it is just a little louder than other records, sound engineers pushed the limits of how loud a recording could be. Music lost one of its most important elements: dynamics. In these days it looks like the loudness war will come to an end because of newbroadcasting regulations and “replay gain”-like countermeasures in online music stores like iTunes.

However, I am sure that the loudness war is not the problem itself but a symptom of a larger trend in (digital) media. First, we must broaden the definition of “loudness” a bit into the direction of “screaming for attention”. These are a few examples of occasions where the “attention war” takes place:

– commercials and trailers: the louder, the better. Ultra-deep drone sounds, ridiculously deep male voices, superfast cuts and stupid lines of epic blablabla are ok for one trailer. In the average German cinema, I am being bombarded with that shit for about 45 minutes.

– classical music: modern orchestras have the tendency to tune their instruments to a higher pitch (e.g. 445 Hz). Thus, the whole string section sounds more brilliant and louder.

– instrumentation in film music: in modern blockbusters, every film score sounds similar to me in the sense of instrumentation and dynamics. Every small phrase or melody is duplicated among every orchestra/synth/choir/something-section. Thus the score does not contain any surprising ups and downs in dynamics but is just a big sausage of orchestral bwwam – similar to “sausage waveforms” in over-compressed recordings.

– dramaturgy in modern films: modern blockbusters contain frantic action (and no story!) from beginning to end. Good examples are the Transformers series, Dark Knight Rising, Pacific Rim. “Loud” scenes are included instead of more story elements or quiet sections, making the story arc of the film appear like a flat line.

These the-more-the-better media all fail eventually because of the fact that loudness is measured in relative units (decibels), because humans perceive volumes relative to other volumes. To make something LOUD, it must be preceded by something quiet. The result of a constant bombardment with loudness is usually numbness, which dulls the interest for things which are really interesting/important. Consumers of media do not have short attention spans in general, but they are treated like idiots anyways. For instance, many radio stations assume an attention span of less than thirty seconds and torture their listeners with ultra-short information snippets, stupid music and “you are listening to YELL-O-RADIO”-announcements every 45 seconds.

In part 2, I will try to find some reasons for the overall loudness problem and list some possible remedies.

Related links:

Long time, no post… I am very sorry for the large gap between this new post and the last one. A list of excuses would be: christmas, new year’s eve, a stomach infection and lots of work for my sidejob at Klangerfinder inbetween.

In November, I have uploaded a video showing a theremin-like controller consisting of two cans with lightsensors at the bottom. Last week, I added a Magnetometer (a sensor which reads the direction of a magnetic field) to the setup. The original idea was to use the actual direction and strength of the magnetic field to control something. It was possible to use the sensor as a digital compass, but it was too unreliable to control musical parameters with the data. Therefore, I simply ignored the direction and strength of the magnetic field and just computed a single value based on the fluctuations in those values. This means, whenever some magnet is moving near the sensor, it will trigger a control value which roughly correlates with the intensity (i.e., speed) of the motion.

In this fairly simple example, the magnet controls the amount of vibrato in the cello tone. One can/light sensor controls the volume of the cello, the other can/light sensor controls the pitch of the drumloop.

I think the potential of the magnet-controller is in its simplicity. You just have to wave with the magnet to “excite” some musical instrument, similar to plucking a string.

The facts which will tell you how to build this yourself are already listed in the previous raspberry-pi post.

The code for this new version is also hosted on github in the lightsensors-repo. For this version, I extended the old code a bit and used the file combined.c.

In the previous post, I promised to upload a remix of a well-known silent short film this week. The good news is that creating the remixed video and the music was a thrilling experience, the bad news is that I will not upload it on this blog. The images are too grotesque/insane to be presented without an explanation and a proper context (just believe me). After all, this blog is a part of my online self-presentation (like my facebook profile and soundcloud) and I do not want people to get a wrong impression about me. If you publish material on the web, you accept the fact that people will watch the material in a totally different situation, in a totally different context. That is why I will not simply upload the video on vimeo. If you want to see the video, you can send me an email or post an (empty) comment below (you have to enter an email address to post a comment, however, it is only visible to me).

The video is out there, so there is a small chance that you might stumble upon it anyways. Since I have used my identity on the I2P-network to spread the video, I will not write about the video in detail in this post. Otherwise one would be able to connect my name with the identity on I2P which would be stupid considering anonymity.

The video was remixed using a small C program which I have written exactly for that purpose. The code analyses some music composed for the movie and rearranges the video frames according to the sound. In fact, the images follow the music, not the other way round. The result was overwhelming in a sense that the machine rearranging the images totally suprised me and that the rearranged images amplified the emotional message of the music. I will probably reuse part of the code to remix other videos which will be more suitable for this blog.

tl;dr the video is to weird for this blog, believe me. send me a message if you want to see it anyways.


Get every new post delivered to your Inbox.