Audio Feature Based Synthesis Part 1: The Building Blocks

This is the first part of a series of tutorials involving musical synthesisers, machine learning, and the Web Audio API. Basic prior understanding of sound synthesis is required - otherwise this would be a book! I recommend that you read the section on Subtractive Synthesis in this great article. Over the course of a few posts, we'll build up to an awesome, feature-rich, musical synthesizer.

In this tutorial, we'll build a basic subtractive synthesizer using Javascript, and the Web Audio API. One very high level summary of subtractive synthesis is that any sound that you hear is composed of lots and lots of discernible, small components, and subtractive synthesis is about making interesting sounds by starting with a very rich sound, and systematically removing (or "subtracting") some components, until you're left with a particularly aesthetically pleasing sound. This tutorial will deal with building the minimum viable subtractive synthesiser. We'll use the two most basic building blocks of modular synthesizers - an "oscillator" and a "filter" - to piece together the simplest synthesizer that can be reasonably described as "subtractive". In part 2, we'll build some basic infrastructure in order to actually play the synthesizer, as this will be useful in later tutorials.

Sound "components"

It's useful to have a little bit of understanding of exactly how sounds break down into components - what those components are, and what their properties are. Any sound is composed of some set of sine waves each with their own frequency, loudness, and phase, summed together. The particularly loud sine waves contribute more to the sound than the quieter ones, and the timbre of a sound depends on whether lower or higher sine waves are loudest, and what the relationships are between each of the waves. Usually, the 'frequency' part of the sine wave is put on an axis, with a chart of some kind representing the loudness and phase parts of the sound. We call this representation of a sound the "frequency domain", because the primary axis is on frequency. Here's an example of a frequency domain graph of a signal courtesy of Guzik Technical Enterprises:

A plot showing showing the plot of a signal across an x-axis representing frequency measured in hertz

Synthesizer architecture

Oscillator

In subtractive synthesis, we start with a rich signal. In this case, we'll use an "oscillator" to generate that signal. Oscillators are devices (or computer programs) designed to generate signals, usually based on certain rules. Oscillators can generate a variety of different kinds of signals, each with their own special characteristics. We'll use a "sawtooth" wave for this tutorial, but you can read more about the properties of each of the basic waveforms on wikipedia. We're going to use sawtooth waveforms because they are particularly harmonically rich - that means that sounds produced by sawtooth oscillators have particularly many sine waves in them, without sounding too overwhelming like noise.

Like the vast majority of synthesizer building blocks (we'll call them nodes going forward for reasons that will become apparent), the oscillator takes certain parameters. In typical subtractive synthesizers, the oscillator is the node that defines the pitch of the output sound. It does this by looking at its "frequency" parameter. Another typical oscillator parameter is the waveform type. In this case, we'll choose "sawtooth", but we'll explore other waveform types later on in this tutorial series.

Within synthesizers, nodes usually either create sound from scratch (as is the case for an Oscillator), or recieve sound from other components and modify it, which brings us to the...

Filter

The filter is a synthesizer component that accepts input sound, processes it to remove some components of the sound based on parameters, and outputs the modified version of the sound. Filters are an enormous topic for another post, but for the purposes of this post, you shouldn't worry about it because we'll be us. If you want to read more, about filters, check out this great guide on the topic. We'll use a filter called the "low pass filter". As the name suggests, this filter takes input audio, and allows the parts of the frequency that are lower than a specified frequency to pass through, and attenuates (quietens) the high parts of the signal. The effect that a low pass filter has on a sound is to make the sound less "bright", and more "bassy". You'll hear it soon!

How does this work in code?

We'll use the Web Audio API to build our synthesizer! Don't worry if you're not an all-star web developer - for our tutorial, the Javascript syntax we'll use will be pretty similar to other C family languages you might have used in the past, like C, C++, Java, Ruby, Python, or similar. You should note that the Web Audio API is available in most modern browsers, but not by default in node.js.

We start by creating an "audio context". You can think of this as a box to put all of our synthesizer nodes in. Audio Contexts are created like this:

const context = new AudioContext();

Great! We've got our context. Next up, we'll create an oscillator:

const oscillator = context.createOscillator();

The oscillator has some parameters, including Frequency and type to let you set the pitch and waveform of your oscillator, but the defaults are good, so lets not meddle with them.

Nearly there already in only 2 lines of code! Only one more node to create - the filter.

const filter = context.createBiquad();

Biquad is a fancy name for the implementation of the filter in web audio, but this line creates a standard audio filter node with all the defaults.

Finally, we connect our nodes together, and plug them into the output.

oscillator.connect(filter);
filter.connect(context.destination);

The context.destination variable is the audio output of the web audio context. You can think of it like a plug socket for your computer's speakers.

Finally, we'll need to start the synthesizer. We'll do it in response to a button click, so that your web page synthesizer doesn't do autoplay audio!

document.querySelector("button#start").addEventListener("click", () => {
oscillator.start();
});

Here's an example synthesizer running in CodePen. It also includes a stop button, so that you can stop the beeping if you want!

document.querySelector("button#stop").addEventListener("click", () => {
oscillator.stop();
});

See the Pen The Simplest Synth by Hugh Rawlinson ( @hughrawlinson) on CodePen.

Summary, and what's next?

CONGRATS ✨⚡️💻🎛 YOU BUILT A SYNTHESIZER! You can use your newfound skills to annoy people when they least expect it. Pro tip: setting the oscillator's frequency to around 12000 hz really annoys young people, and older people can't hear a thing!

oscillator.frequency.value = 12000;

Next time, we'll learn how to send notes to your synthesizer, so that you can play it with your keyboard (of the musical, mechanical, or inbuilt varieties!)