Modulation 101:
Make you sounds come to life with MIDI modulation
By Neil Leonard

Electronic Musician, April 1998
Copyright 1998, Neil Leonard

People often think of electronic instruments as cold and impersonal. After all, I can identify many saxophone players after hearing only a few notes, yet it is hard to say the same for musicians working with synthesizers and samplers. Saxophonists encode their signature in individual notes by varying the placement of their tongue, teeth, and lower lip, altering wind velocity, and by using their larynx. As a result, the pitch, timbre, and volume of each individual note evolve in complex and subtle ways. Beginning wind instrumentalists labor to control such variations. Even accomplished wind instrumentalists burn the midnight oil to maintain a semblance of uniformity between notes. By contrast, a beginning synthesist can effortlessly create notes with absolutely no variation in pitch, timbre, and volume. Master synthesist such as Jan Hammer, Wendy Carlos, Bernie Worrell and Joe Zawinul recognized these idiosyncrasies of their instrument and found ways to work with the time variant nuances that help make their electronic music so memorable.

The way to introduce nuance into electronic sounds is through the use of modulation, both in performance and while designing patches. Though the modulation architecture of today's sleek digital synthesizers is often hidden deep within menuing systems, it was unavoidable in early modular analog synthesizers. When you look at a modular system in use, the first thing that you notice is a tangled mess of patch cords covering the front panel. This is in fact where the term "patch" came from. These instruments made no sound until the performer took a patch cord and routed a control voltage, such as the keyboard's control voltage output, to an oscillator's control voltage input. In the case of the Moog modular system, the keyboard and oscillator were designed to work on a one volt per octave scale. Oscillators had additional control voltage inputs that could be used to connect the control signals from a low frequency oscillator, envelope generator, or other sources.

The modulation principals at work in today's synthesizers are based on those developed for modular synthesizers. Whether your sound chip is designed to play samples or uses the latest synthesis technology to model the physical behavior of acoustic instruments, dynamic changes in individual notes can be created by using a tactile interface or by programming such variations using LFO's and envelope generators. While the principals of modulation are the same, the implementation of modulation features and associated terminology is far from uniform in today's synthesizers. To get a feel for the lay of the land, let's look at the three essential parts of a modulation architecture: control sources, patching methods, and sound parameters that are often modulated.

Get Control

There are three common types of external control sources that are used to perform musical nuances. First are the physical controllers that are found on electronic keyboard instruments. These include velocity sensors and key pressure, pitch bend wheel, modulation wheel, data sliders, program change buttons, and the keyboard itself. The arrays of knobs and sliders that were characteristic of earlier synthesizers are now making a comeback and can be found on many mid-range synthesizers, particularly instruments that use physical modeling algorithms or emulate analog synthesizers (see Fig. 1).

Next are the sockets provided by many devices for connecting additional controllers such as expression pedals, sustain pedals, and breath controllers. Some effects units are made to work with custom foot controllers that have multiple pedals and a matrix of buttons. One effects manufacturer recently made a foot controller that has a programmable toe switch at the tip of the pedal. This could enable a performer who is using both hands, such as a guitarist, to toggle a chorus effect on and off while controlling its depth with the pedal at the same time.

The third way to create control signals is by using external MIDI controllers. These include slider boxes, wind, drum or guitar controllers, pitch-to-MIDI converters, three dimensional sensors, and specialized kits that sense things like proximity, pressure, and temperature changes (see Fig. 2). A number of new devices in this category have appeared recently, and the real-time, tactile feedback they give the user make them a very valuable addition to the synthesist's arsenal.

Most MIDI messages can also serve as a control source. The MIDI Note On message typically modulates the pitch of the oscillator in increments of one half step. The Pitch Bend message typically modulates the pitch of the oscillator up or down a whole step. The MIDI Control Change message is used to specify continuous changes for up to 121 controllers. Some of these controllers have default assignments, such as vibrato, volume, pan, and sustain. However, when programming many synthesizers and effects units, you can typically assign any controller message to any parameter. In fact, in the most open systems any MIDI message can modulate any sound generation parameter.

From the Inside Out

Internal modulation sources are created by internal circuitry and must be programmed. The two most common internal control sources are the low frequency oscillator and the envelope generator. The LFO creates single cycle waveforms including some collection of the following: sine, triangle, square, sawtooth/ramp up, inverted sawtooth/ramp down, pulse, and inverted pulse (100%-pulse width). You might also find a quasi-random waveform available on some machines. The waveform shape, rate, phase, and pulse width can be programmable, though not each of these parameters are editable as a rule (see Fig. 3). LFOs are most commonly used for vibrato and tremolo.

Envelope generators come in all shapes and sizes, but the classic model from the days of modular systems is the ADSR envelope generator, which stands for attack, decay, sustain, release. The function of an envelope generator might assigned to one fixed parameter, but does not have to be. In some synthesizers there may be as many as five envelope generators that can be assigned to any parameter.

Control sources, whether internal or external, need to be routed to a desired sound generation parameter. One vintage analog system, the Electronic Music System VCS3, (often referred to as the Putney and used on Pink Floyd's Dark Side of the Moon), used a patch-board matrix to establish these connections. In this case the control and audio signals could be routed anywhere in the system by inserting metallic pegs in the matrix. Through systems like this one the term "Matrix Modulation" was coined. This matrix style design is still used by some manufacturers who implement the matrix in software.

By contrast, some synthesizers, including inexpensive General MIDI synthesizers, use fixed modulating routings. You can edit the rate of an LFO on one page and specify its level when applied to pitch on another, but the link between that particular LFO and the oscillator is hard wired. What you gain in terms of ease of use is lost in terms of flexibility in programming and variety of available sounds.

In between these two extremes there are a wide variety of designs for modulation routing. Patch chords are in again, at least in software. On Lexicon's popular PCM 80 effects processor, "patches" are specified in software by entering a source (selectable from any internal or external controller) and a destination (any effects parameter). Other manufacturers have implemented different schemes but provide the same "any controller to any parameter" result. If you like rolling up your sleeves and digging deep into programming, this approach is ideal. Even if you don't want to program, you might find that the presets on these devices sound superior. This can often be the result of good programming techniques that allow the sound or effect to change over time.

The sound generation parameters that are commonly modulated are the oscillator's pitch, the amplifier's output level, and the filter's cutoff frequency. However, any sound generation parameter can be modulated. When using a physical modeling synthesizer to emulate a brass instrument for example, a breath controller can be used to vary the model's parameters for wind velocity, the position of the lip and teeth, or to create idiosyncratic harmonic slurs.

Some effects units can morph between two entirely different effects algorithms by moving one slider. Modulators can even modulate parameters for other modulators. When you increase the modulation wheel setting while playing a flute patch, the vibrato depth might increase. In this case the modulation wheel is scaling the depth of the LFO being used to create the vibrato. This is sometimes referred to as "second order modulation." Another common example is the use of a control source, such as key velocity, to scale the depth of an envelope generator. This can be quite effective when creating piano sounds.

Some synthesizers have a low velocity envelope and a high velocity envelope and interpolate in-between envelopes based on key velocity. Again, second order modulation. There are synthesizers that enable the synthesist to use first, second, third, and fourth order modulation, and advanced synthesists like Wendy Carlos adamantly prefer these systems.

Tipping the Scales

Though I've touched on many of the fundamental issues regarding modulation, there are other techniques at your disposal. Many instruments, for example, can process control inputs before they are assigned. One common implementation of this technique is the ability to scale controller values. Using this method, the synthesist might designate what values a slider will send when it is at its minimum and maximum position. This offers some possibilities that may not be immediate obviously. Moving a slider from its minimum to maximum position causes the synthesizer to interpolate in-between values. This simple linear relationship is the default on many instruments.

But what if you could change the response of the slider so that, for example, the values moved from low to high, then back down to low as the slider moved along its path? Or how about the ability to program a pitch wheel so that a bend up shifted the pitch a major third and a bend down shifted it a minor seventh? This configuration, which approximates the intervals that a guitarist can perform by bending strings and using a whammy bar, would be a non-linear relationship. Some digital instruments provide ten or more scaling pairs for creating a wide variety of non-linear contours, and others provide numerous mathematical functions for scaling and combining controller values. You might not want to get this involved in editing patches, but you will hear these techniques at work on the presets of some of today's most popular synthesizers and effects units.

Audio Control

Analog synthesizers are often considered to be preferred by today's dance musicians for their warm sound and tactile interfaces. Some modular analog systems have another distinguishing quality. Audio signals can serve as control signals and vice versa. While this might seem esoteric and impractical, consider this. The average LFO on today's digital synthesizer doesn't typically produce a signal higher than 20-50 hertz, which is comparable to the rate of a fast vibrato.

But an oscillator can produce a sine wave at any frequency that can modulate the another oscillator, also at any frequency. If you listen to the output signal from the second oscillator, the waveform will be much more complex and richer in harmonics than the sine wave. This is called frequency modulation (FM), the same technology used in FM radio and for synthesis on multimedia sound cards. The first mass marketed synthesizer that used FM was the Yamaha DX-7, which is reportedly the biggest selling synthesizer in history. Synthesists using modular systems have been using FM for years, and this type of control source is now often found in desktop synthesis systems that are modeled after modular systems.

Sequencer Control

Though we have discussed the use of modulation in performance and synthesis, sequencers can also record and generate MIDI controllers. It is now possible to control a wide variety of mix parameters via MIDI, and many musicians sequence data for pitch bend, volume, panning, and effects levels. Often, they will dedicate a separate track for each of these parameters. This data might have been played in using a keyboard controller, foot pedals, or even a MIDI slider box. Once recorded, this modulation data can be cut, pasted, scaled or limited in a graphic editor.

Many modern sequencers allow the musician to create custom consoles to generate control data. On the PC, for example, these consoles or "panels" can be used to control the synthesizer on a sound card (see Fig. 4). In some cases, these software sliders and dials can also be controlled by external devices, such as MIDI slider boxes. You can often find custom-built consoles and panels on the software manufacturer's Web site that can be used for both new and older instruments.

ALL IN GOOD TIME As a saxophonist and synthesist I have found that there are at least as many ways to create time varying nuance with a synthesizer as there are with a saxophone. A good understanding of the principals of modulation goes a long way towards achieving successful results. Despite how much synthesizers, samplers, and effects units have changed, much of the expressive power of the instrument remains in its modulation architecture. As a great philosopher once said, "it's not what you've got, it's how you move it!"

Neil Leonard III recently received the award for the most valuable contribution to the Music Technology Division curriculum at Berklee College of Music.