Modulating the modulators

A control function with a particular shape can serve a role similar to a traditional musical motive. Even when it is modified in duration, rhythm, or shape, its relation to the original remains evident and it serves to unify a larger composition. A motive or shape might be recognizable at different structural/temporal levels, in which case the form may take on a "fractal" or "self-similar" character.

In other chapters we've taken some first and intermediate steps to progressively increase the complexity of examples in which a control function modulates another modulator of the same shape, such that a single shaped is used at different formal levels in a somewhat self-similar manner.

Here's a more full blown example of a single control function used to modulate a sound at many formal levels, with modulators modulating other modulators, in this case including the parameters of pitch, note rate, volume, and panning (location).


The carrier sound is a triangle wave oscillator, in the tri~ object. The volume of that oscillator is continually modulated in a way that actually separates it into individual notes; it is windowed by a repeating triangular shape going from 0 to 1 and back to 0--the first half of the triangle function stored in the wavetable in the buffer~. The rate of those notes is itself modulated by a triangle function, varying from 1 to 15 notes per second every 25 seconds (the rate is varied up and down + and - 7 from a central rate of 8 Hz, by a triangle oscillator with a rate of 0.04 Hz).

The volume of the sound is further modulated by another triangular LFO that creates a swell and dip of + and - 15 dB in the overall volume every ten seconds, to give a periodic crescendo and diminuendo spanning 30 decibels, which is about as much as most instrumentalists do in practice, even though their instruments are often technically capable of a wider range of intensities.

The pitch of the sound is modulated in almost exactly the same way as was demonstrated in another article. The pitch glides in a triangular shape around a central pitch that is itself slowly changing in a triangular shape over a span of every 50 seconds. The rate of the glissandi varies from 1 to 15 Hz, varying triangularly in a 20-second cycle. The depth of the glissandi varies from + and - 0 to 12 semitones, controlled by a 15-second cycle (perceptually a 7.5-second cycle).

The perceived location of the sound pans back and forth between left and right controlled by a triangular function at a rate that varies from 1/16 Hz to 16 Hz -- quite gradually to quite quickly -- with the rate itself determined by a triangular cycle that repeats every 30 seconds, using the most common panning technique, known as "intensity panning". This takes advantage of the fact that one of the main indicators of the location of a sound's source is inter-aural intensity difference (IID), the balance of the sound's intensity in our two ears. The more the intensity of sound in one ear exceeds the intensity in the other ear, the more we are inclined to think the sound comes from that direction. Thus, varying the sound's intensity from 0 to 1 for one ear (or one speaker) as we vary the intensity from 1 to 0 in the other ear (or the other speaker) gives the impression of the sound being at different locations between the two ears (speakers). So a triangle wave with an amplitude of + and - 0.5, centered around 0.5 is used to vary the gain of the right audio channel, and 1 minus that value is used to determine the gan of the left audio channel. As one channel fades from 0 to 1, the other channel fades from 1 to 0, and vice versa.

Our sense of the distance of sound sources is complicated, but in general it's roughly proportional to the amplitude of the sound. So the same sound at half the amplitude would -- all other things being the same -- tend to sound half as close to us (that is, twice as distant). The perceived overall intensity of the sound will depend on the sum of the two audio channels. Perceived intensity is proportional to the square of the amplitude, and the perceived overall intensity is thus proportional to the sum of the squares of the amplitudes of the two channels. So if we want to keep the sound seeming to be the same distance from the listener as we pan from left to right, we need to keep the sum of the squares of their amplitudes the same. So, as a final step before output, we take the square root of the desired intensity for each channel, and use that as the gain control value for the channel. The picture below shows the gain values for the two channels as they are initially calculated by the triangle function (on the left) and then shows the actual gain values that will be used -- the square roots (on the right). The first is the desired intensity of the two channels, and the second is the actual amplitude for the two channels that's required to deliver that intensity as the virtual sound location moves between left and right.


In order to make the rate of panning span the desired range from 1/16 Hz to 16 Hz, we used the triangle function as the exponent of the base 2, using the pow~ object. As the triangle function (the exponent) varies from 0 to 4 to -4 to 0, the result will vary from 1 to 16 to 1/16 to 1. When the rate is less than about 1 Hz, the duration of each panning cycle is greater than 1 second, and we can follow the panning as simulated movement; when the rate is greater than 1 Hz, the complete left-right cycle of panning takes places in less than a second, up to as little as 1/16 of a second (62.5 ms), so we perceive it more as a sort of "location tremolo" sound effect.

So in this example program the triangle wave function was used in nine different ways:
1) as the carrier waveform
2) as a window (amplitude envelope) to make individual "note" events
3) to modulate the rate and duration of the notes
4) to create 10-second volume swells
5) to vary the central pitch of the oscillator
6) to make pitch glissandi around that central pitch
7) to vary the depth of those glissandi
8) to vary the rate of those glissandi
9) to vary the panning of the sound

Second steps in modulating the modulator

Here are two programs that show further development of the programs described in first steps in modulating the modulators. There we saw how to use an LFO to modulate the pitch of a carrier, and how to use another LFO at an even slower rate to modulate the amplitude of the modulator.

In this first example we modulate both the rate and the depth (the frequency and the amplitude) of the LFO that is modulating the pitch of the carrier oscillator.


Once again we use all triangle functions, and we use a central pitch of 60 (middle C). The depth of pitch modulation -- plus or minus a certain number of semitones -- changes continuously, determined by the instantaneous value of a very-low-frequency oscillator with a peak amplitude of 12. Therefore, the depth will be as great as + or - 12 semitones, or as little as 0. The rate of the modulation varies from 1 Hz to 15 Hz, controlled by very-low-frequency oscillator that has a peak amplitude of 7 and an offset of 8 (so it varies up to + or - 7 Hz around its central rate of 8 Hz). Because these two control functions have different periodicities, the effect is continually changing, repeating exactly every 60 seconds.

In the next example we add one more modulator to continually change the central pitch, varying it up to + and - 30 semitones around a center of 66.


The central pitch will slowly, over the course of 50 seconds, rise from 66 to 96, fall to 36, then rise again to 66. The actual moment-to-moment pitch will oscillate up to 12 semitones around that, so the true pitch at any given instant could be as low as 24 or as high as 108, roughly the range of a piano. Since all three of these control functions have different periodicities -- 20, 30, and 50 seconds -- the entire cycle only repeats exactly every 5 minutes.

It's worth noting that as we combine different long cyclic phenomena with different periodicities -- in this case 20, 30, and 50 seconds -- the result of their combination varies continuously over a longer period that's equal to the product of the prime factors of the periods -- in this case 2 times 2 times 3 times 5 times 5 = 300 seconds. The effect is one of something that remains the same -- after all, it's the same three cycles repeating precisely over and over -- yet always seems slightly different because the juxtaposition and relationship of the cycles is always changing. This phenomenon is an essential component of much "minimal" or "phase" music.

First steps in modulating the modulator

Classic waveforms can be used to shape music synthesis, and that idea can be extended to shape musical composition with simple, recognizable, repeating shapes. Indeed, many figures that we think of as traditional pitch patterns in pre-electronic music have a direct correlation with those classic waveforms. The picture below exemplifies simple melodic patterns in traditional notation that could be achieved with discrete sampling of classic waveforms at low frequency, used to control the pitch of a carrier sound.


These melodic figures and their corresponding control functions can be described as:
1) trill, pulse wave
2) fingered tremolo, pulse wave with increased amplitude of modulation
3) glissando, linear ramp
4) scale, discrete sampling of a linear ramp
5) vibrato, triangle or sinusoid with low amplitude of modulation
6) up-down arpeggio, discrete sampling of a triangle function with high amplitude of modulation
7) melodic sequence, sawtooth (or any other shape) that is itself modulated by a ramp function
8) motivic melodic figure, discrete sampling of an arbitrary shape as it changes over time
9) up-down arpeggio (variation), discrete sampling of a sinusoidal function
You can probably imagine many other similar melodic shapes that are similarly simple yet effective.

These examples show clearly how a melody can be thought of as pitch modulation by a control function, and the shape can be simple, as in most of these examples, or more complex, as in example 8 above.

Sometimes more interesting effects can be achieved by using using these shapes operating at different formal levels at the same time, or with one shape modulating another as in example 7 above.

The triangle function, while decidedly not the most interesting shape imaginable, is particularly recognizable, and therefore is good for exemplifying these principles clearly. So we'll use it in a variety of examples for shaping sound synthesis and composition, focusing particularly on modulating one control function with a lower-frequency version of itself, which is to say, shaping the sound at a different formal levels, by means of self-similar use of a single shape.

The example below shows the most basic use of the triangle waveform as both a carrier oscillator and as a low-frequency control function for the pitch of that oscillator. The carrier oscillator generates a triangular waveform with the tri~ object which, instead of producing an ideal triangle function, protects against producing partials that will exceed the Nyquist frequency. The pitch of that oscillator is modulated by a low-frequency oscillator -- a cycle~ object reading from a wavetable that has been filled with one cycle of a triangle function. (When the patch is first opened, the small part of the program on the right fills the buffer~ with the values needed to make the stored triangle function. It also sets the scope~ to show one second of sound per display; the scope~ refreshes its display every 344 buffers of 128 samples.)


The modulating oscillator has a rate of 3 Hz, so the pitch of the carrier oscillator completes 3 cycles of the triangular shape per second. Since the amplitude of the cycle~ object is 1, the pitch fluctuates + and - 1 semitone around the central pitch of 60 (middle C). We'll call the rate of modulation Fm (pronounced "F sub m", meaning the frequency of the modulator), which is 3 Hz in this case, and we'll call the depth of modulation Am (pronounced "A sub m", meaning the amplitude of the modulator), which is constant at 1 in this case.

We can vary the pitch modulation over a longer period of time by modulating Fm and/or Am with an even slower oscillator. For example, in the program below we use one very-low-frequency oscillator to modulate the amplitude of a low-frequency oscillator that is modulating the pitch of the carrier oscillator.

We have set Fm to a constant of 6 Hz, but Am is modulated by another oscillator with a rate of 1/30 Hz and an amplitude of 12. So every 30 seconds the depth of the "vibrato" changes, according to the triangle wave function, from 0 semitones to 12, to 0 to -12 and back to 0. You probably won't recognize the difference between a vibrato depth of + and - 12 semitones and its inverse, + or - -12 semitones, so in effect the vibrato seems to complete a full cycle of expansion and contraction once every 15 seconds.

So the pitch modulation, with a rate of 6 Hz, is itself modulated in amplitude repeatedly every 15 seconds. This is a simple case of a a modulator modulating a modulator.

Sine wave as control function

Sine and cosine are trigonometric functions that come from graphing the y or x value, respectively, of a point as it traverses the circumference of a unit circle in a constantly changing radial angle from 0 to 2π radians. The cosine is actually exactly the same as the sine with a phase offset of π/2 radians, which is to say starting 1/4 of a cycle into the sine function. To talk about any such function, regardless of phase offset, we can use the noun sinusoid and the adjective sinusoidal.

It happens that the sinusoid is also the graph of simple harmonic motion, such as the natural oscillation of a pendulum or the simple back-and-forth vibration of the tine of a tuning fork or an alternating electrical current. Simple harmonic motion is oscillation at a single frequency, so the sinusoidal wave is the most basic "building block" or elemental unit of all sound.

The cycle~ object in Max acts as a wavetable oscillator for generating periodic signals, and by default it uses the cosine function.

(Internally it is actually reading from a 512-point lookup table, and interpolating between those points as necessary to generate a smooth signal at any frequency.) Its peak amplitude is 1; it oscillates in the range from 1 to -1. Every time you turn on audio signal processing in Max, all cycle~ objects begin in cosine phase--i.e., starting at 1. However, you can supply a phase offset in the right inlet, so to make a cycle~ start with sine phase, you must supply a phase offset of 0.75 to start 3/4 of a cycle into the cosine function.

The sinusoidal oscillator can be used at an audio frequency as a carrier oscillator, or as a control function at a low frequency (or indeed any frequency) to modulate other signals. Since the sinusoidal oscillator was a basic generator in almost all early electronic synthesizers, we have become very familiar with the sound of using one oscillator for sinusoidal modulation of another oscillator's frequency and amplitude to create vibrato and tremolo effects. A singer or flutist will generally use a combination of vibrato and tremolo -- modulation of both frequency and amplitude of the tone -- for expressive effect at a rate somewhere between 5 to 8 Hz. Electronically and digitally, of course, we can modulate a tone at any rate, from extremely slow (such as 1/20 Hz) to audio rates (such as 2000 Hz). So we can use these different rates for sinusoidal control functions at the phrase level, the note level, or the microsonic timbral level.

This program demonstrates the use of the sine function for low-frequency modulation of a tone. One sine wave is the carrier oscillator that we actually hear, and the other three sine oscillators are the modulators.

When MSP is turned on, that fact is reported by the adstatus object, and that report is used to set the phase offset of the cycle~ objects to sine phase. This ensures that they all start with the correct, identical phase offset. The frequency is modulated + and - 25 Hz around a center frequency of 440 Hz. That's + and - about a semitone, so it fluctuates between approximately Bb and Ab around A 440. The amplitude is modulated + and - 0.4 around a center amplitude of 0.5. That's a total range from 0.9 to 0.1, which is a factor of 9, which is about 19 dB. Both of those ranges are larger than most singers or instrumentalists would ordinarily use for expressive vibrato, but they're made deliberately large here so they'll be obvious. The tremolo (amplitude modulation) rate is 6 Hz, which is about normal for idiomatic instrumental vibrato. However, the vibrato (frequency modulation) rate is only 1 Hz, which means it glides fairly slowly -- slowly enough that we can track its pitch -- and we tend to hear its extremes, Bb and Ab, as the main pitches. Interestingly, if you adjust the vibrato rate to be 6 Hz like the tremolo, the vibrato will be so fast that -- at this still fairly narrow pitch interval -- we tend to hear its center frequency, A 440, as the main pitch.

A third, very slow modulating sinusoid is used to shape the overall amplitude over a longer period of time. Its frequency is 1/8 Hz, which means that it completes one full cycle every 8 seconds. So, every 4 seconds it goes from 0 to peak amplitude (+ or -) and back to 0. Therefore, we hear a complete crescendo-diminuendo every 4 seconds. Even though the waveform is inverted when this modulator is negative, we don't perceive that fact when we listen to the tone in isolation, so amplification by a negative factor (the second half of the cycle of the modulating sinusoid) sounds the same to us as amplification by a positive factor (the first half of the cycle).

You can experiment with some different rates of modulation, including very slow and very fast (even audio rate) speeds.