l.o is our first long-form generative audiovisual series: a flock of particles brought to life by an audio algorithm inspired by modular synthesis and delay effects. As the sound expands and contracts, so do the areas of visual density, revealing ephemeral shapes and structures.
Sounds become visible. Visuals become audible.
Sounds become visible. Visuals become audible.
Inspiration
The filter sequencer at the heart of l.o’s audio algorithm is directly inspired by modular synthesis which enables musicians to connect synthesizer components in untraditional ways. Instead of controlling pitch like a typical synthesizer sequencer would, this one determines which frequencies are removed from the input chord at different moments in time. The melodies of l.o emerge simply by filtering out certain parts of the sounds without ever specifying any pitch variation.
Vintage analog equipment and its magical warmth is another source of inspiration in our musical work. This influence can be found in several parts of l.o’s audio algorithm: the design of the main echo effect which is inspired by tape loop echo units, the white noise mixed in the input signal or the distortion applied right before outputting the final sound.
This analog quality also found its way into the visual style. The low frame rate gives the project a hand animated feel reminiscent of stop motion. The filmic background noise echoes the noise present in the audio.
The visual composition is inspired by starling murmurations and the mesmerizing patterns that emerge from the birds’ fluid and synchronized movements. The particles are coordinated by various signed distance functions organizing the flock in patterns of density, size and color. They are rendered using geometry instancing, a draw call optimization technique inspired by video game development where it is commonly used for objects such as grass or trees.
Vintage analog equipment and its magical warmth is another source of inspiration in our musical work. This influence can be found in several parts of l.o’s audio algorithm: the design of the main echo effect which is inspired by tape loop echo units, the white noise mixed in the input signal or the distortion applied right before outputting the final sound.
This analog quality also found its way into the visual style. The low frame rate gives the project a hand animated feel reminiscent of stop motion. The filmic background noise echoes the noise present in the audio.
The visual composition is inspired by starling murmurations and the mesmerizing patterns that emerge from the birds’ fluid and synchronized movements. The particles are coordinated by various signed distance functions organizing the flock in patterns of density, size and color. They are rendered using geometry instancing, a draw call optimization technique inspired by video game development where it is commonly used for objects such as grass or trees.
Suzanne Ciani, a prominent modular synthesist
What is modular synthesis?
Pioneered by Robert Moog and Don Buchla in the 60s, modular synthesis is a type of sound synthesis where individual electronic modules are connected together in various configurations, allowing for a high degree of customization and experimentation in creating electronic sounds. Contrary to regular hardware synthesizers where the signal flow and individual components are chosen by the manufacturer, modular synthesis lets the musician choose each component and define how the audio and modulation signals travel through them.
Pioneered by Robert Moog and Don Buchla in the 60s, modular synthesis is a type of sound synthesis where individual electronic modules are connected together in various configurations, allowing for a high degree of customization and experimentation in creating electronic sounds. Contrary to regular hardware synthesizers where the signal flow and individual components are chosen by the manufacturer, modular synthesis lets the musician choose each component and define how the audio and modulation signals travel through them.
Starling murmurations from The Art of Flying by Jan Van IJken
audio
l.o's audio signal path is made entirely of custom components built on top of the Web Audio API.
.png)
l.o's audio signal path, dashed lines indicate parallel processing
After the filtering stage, the sound is modified by a series of effects exploring variations around a single algorithm type: feedback delay lines. A feedback delay line is a system for creating echoes by delaying an input signal, applying a volume change to the delayed signal, and then adding it back into the original signal.
The main echo effect in l.o mimics the behavior of a tape loop echo, a technique developed throughout the 40s & 50s by modifying reel-to-reel tape decks. It was later commercially packaged by Roland into the Space Echo series. The speed at which the loop plays often varies, resulting in subtle pitch bending that creates a chorusing effect and sense of humanness. The use of magnetic tape results in a compressed and saturated tone which creates sonically pleasing harmonics, the source of that analog “warmth”.
The main echo effect in l.o mimics the behavior of a tape loop echo, a technique developed throughout the 40s & 50s by modifying reel-to-reel tape decks. It was later commercially packaged by Roland into the Space Echo series. The speed at which the loop plays often varies, resulting in subtle pitch bending that creates a chorusing effect and sense of humanness. The use of magnetic tape results in a compressed and saturated tone which creates sonically pleasing harmonics, the source of that analog “warmth”.
The second echo effect is an implementation of the Karplus-Strong algorithm which creates new "notes" out of the original sound source.
The flanger, a popular tool in psychedelic music, is the third feedback delay line effect implemented. Changes in delay time create a slowly sweeping filter which pokes holes in the spectrum and gives the impression of pitch variation.
Roland’s RE-201 Space Echo, a legendary tape echo unit
What is the Karplus-Strong algorithm?
Originally published in 1983 by Kevin Karplus & Alex Strong, and then further refined by Julius Smith, it was designed to mimic the acoustic properties of a plucked string. At a time when digital instruments were just beginning to appear in studios (the mass produced Yamaha DX7 was not available until 1983), computationally elegant and sonically rich digital synthesis techniques were quite rare. The decay and tuning of the string are controlled by modifying the feedback gain and delay time of the echo. For those familiar, this effect is akin to Ableton’s Resonator plug-in.
Originally published in 1983 by Kevin Karplus & Alex Strong, and then further refined by Julius Smith, it was designed to mimic the acoustic properties of a plucked string. At a time when digital instruments were just beginning to appear in studios (the mass produced Yamaha DX7 was not available until 1983), computationally elegant and sonically rich digital synthesis techniques were quite rare. The decay and tuning of the string are controlled by modifying the feedback gain and delay time of the echo. For those familiar, this effect is akin to Ableton’s Resonator plug-in.
On the visual side, the first prototype of l.o was built using the p5 library. However, it quickly became clear that the project would need to draw tens of thousands of particles at the desired frame rate. This is challenging to do in p5 which doesn't provide direct access to low-level graphics controls. Enter the library three.js and its InstancedMesh abstraction.
Instancing is a graphics feature that allows the rendering of a very large number of similar meshes in a single draw call, thus saving a ton of CPU-to-GPU communication. Thanks to this technique, each output of l.o comes down to three draw calls: one for the noisy background, one for the grid of particles, and one for the initial “Play” button.
Audiovisual design
AV design
Can the audiovisual medium be more than visuals in the service of sound, or sound in the service of visuals? How can the synthesis of the two media create a richer whole?
Creating complex forms by repeating and transforming a simple input:
a process at the heart of generative art and ambient music
a process at the heart of generative art and ambient music
Visually, the repetition of a line, dot, or ring across a grid structure forms a flock of flickering particles. Over time, as the sound expands and contracts, the frequency of repetition in both X and Y axis fluctuates, and reveals intricate ephemeral patterns and structures made of basic building blocks.
.jpg)
A still frame from l.o: repeating a line with variations in density, size and color
Within the sonic component, everything you hear comes from running an input sound through various effects and modulations. Filters and echoes manipulate a sustained chord until subtle melodies and rhythmic structures appear, resulting in a hypnotic and evolving soundscape unfolding from an elementary source.
AUDIO CLIPS PLACEHOLDER
What is the audiovisual medium?
Previous Art Blocks projects have asked these questions. In Flux by Owen Moore, sound reacts to visual data. In montreal friend scale by Amon Tobin, the visuals are a literal representation of a sonic concept.
In l.o, our intention was to create an AV dialogue where sound and visuals are mutually dependent on each other, as if one couldn’t exist without the other. The diagram below details the 9 points of connection between the two, distinguishing between static elements randomly chosen for each token upon initialization, and dynamic elements modulating over time.
l.o's audiovisual design diagram
The connection between filter frequency and particle grid density is perhaps the one that glues together the audiovisual experience the most. As the filter lets through lower frequencies, the sound's body can be heard more. This in turn increases the visual fullness by rendering more particles in tighter spaces. Sound and visuals become one by responding to the same beat determined by the filter sequence.
The interactive playground below helps get a sense of this interplay between spectral and visual density. Move the slider and note how sound & visuals respond. As the high pass filter allows lower frequencies to come through, the grid of dots becomes denser. This is the core AV mechanic in l.o.
The interactive playground below helps get a sense of this interplay between spectral and visual density. Move the slider and note how sound & visuals respond. As the high pass filter allows lower frequencies to come through, the grid of dots becomes denser. This is the core AV mechanic in l.o.
Audio will start when moving the slider
Creative decisions were intentionally made to reinforce the audiovisual connection as much as possible. For instance, the primitive shape (line, dot or ring) was taken into account when designing the oscillator and amplitude modulation paired with it:
- Line - by modifying the amplitude with a square wave, we create a sharper tone with clearly defined edges in time and frequency. To the point where clicks can be heard. This echoes the straight edges of a line.
- Dot - a sine wave oscillator is modified by a sine amplitude modulation, creating the softest and least spectrally rich tone overall. This mirrors the roundness and simplicity of the shape.
- Ring - the amplitude modulation shape was chosen to be a combination of two sine waves running at different frequencies, one slow and one fast. This stems from thinking of a ring as a smaller circle within a larger one.
Another example is the relationship between color and harmony. For each palette, the notes of the initial chord were chosen to match the color mood. There is a total of 15 palettes organized across 4 mood categories. Each category corresponds to a set of chord notes, and each palette within that category is assigned a unique inversion of the chord.
Building l.o fulfilled a longtime quiet dream of ours: creating a generative audiovisual project by combining our experience from the music studio with our programming skills. It is an honor to collaborate with Art Blocks to present this work to the generative art community.
We couldn't wrap things up without sharing a playlist of music we love. We also included one of our productions from Still. Hope you enjoy!
With love & gratitude,
David & Johan
We couldn't wrap things up without sharing a playlist of music we love. We also included one of our productions from Still. Hope you enjoy!
With love & gratitude,
David & Johan