The A-Zed of Audio
One of the wildest experiences I ever had was eating in the dark. And when I say dark, I mean dark. The restaurant is called O.Noir, and I guess you could call it a concept restaurant; not only do they feed you, but they give you the experience of a blind person – after ordering your food in a dimly lit bar area, you're guided into a completely lightless room and seated at your table by one of the restaurant's blind waiters. After struggling for minutes to see anything, you eventually give up. You're in darkness so deep that having your eyes open or closed makes absolutely no difference.
And then the food arrives. I should also add that for the more adventurous souls, the restaurant offers "surprise" dishes – that is, well, exactly what it sounds like. I distinctly remember the first bite of something with a slippery texture like a cooked mushroom, the smell of tomatoes and garlic. As I realized, it was pasta.
Hearing – much like taste in this sight-deprived environment – is amplified incredibly. Really, all your senses are amplified as they fill the void your vision has left behind.
The way we hear isn't so different from the way we taste. Let's take the pasta as an example – or rather, the sauce coating it. After a few disorienting bites I realized what I was eating, and it was obvious that what I was eating was in tomato sauce. Now, I don't know the ingredients of that tomato sauce but I can assure you there's more to it than tomatoes. Garlic, onions, oil, some other vegetables maybe, basil, salt, who knows what else. But to me, it was tomato sauce. But we're talking about sound. How did we get here?
What we call sound is a disturbance. Like any disturbance it has consequences, and in the case of sound, these consequences are called overtones. In order to hear anything at all we need to be surrounded by a medium which can carry imprints of the disturbance between the source (a guitar, a car horn, a screaming baby) and the destination (our ears). Usually that medium by which we're surrounded is air, less often it's water. The reason there’s no sound in outer space is because there’s no medium for the source to disturb.
For a more terrestrial example, let's consider a vibrating guitar string. The string has a fixed length, and for convenience let’s say its length is 1 metre. As soon as the guitarist plucks the string there’s a rapid vibration that runs across the entire string; this happens in about 1/100th of a second. Just like when you drop a rock in a pool, the vibration runs right down to the ends and then bounces back toward the middle. With that collision in the middle we get a vibration happening over half the length of the string, which amounts to twice the frequency. The way mathematicians and physicists and other serious folks write this is "2f", where f is the original frequency, and we multiply it by 2. Easy as π. Another tiny fraction of a second after the 2f vibration starts going, other sub-vibrations begin over 1/3 the length of the string, 1/4 the length, 1/5, etc. In the end, what seemed like a simple string vibrating actually turns out to be more complex than getting a universal health care bill passed in the United States.
What's really amazing about this (the vibrations, that is) is it happens in a fraction of a second – a fraction so tiny we can’t possibly hear each additional vibration as a distinct event. And to be clear, what we're hearing is the original vibration – called the fundamental – plus all those sub-vibrations which are happening at higher and higher frequencies. Those sub-vibrations are called overtones.
When we hear any sound in the real world, what we're hearing is a concoction of frequencies: a fundamental plus overtones. And unless we're listening with a carefully trained ear, we don't hear these as separate, just like we don't taste the distinct elements of a tomato sauce unless we have a carefully trained palette.
And if you vary the ratio of ingredients in a tomato sauce, at some point you no longer have tomato sauce. Let's suppose you increase the vegetables by 10 times and decrease the tomatoes by the same amount, plus add some broth – now you're working with some sort of soup. Much in the same way, if you vary the ratio of your overtones you start to completely alter the content of your sound.
Every sound has a timbre, just like every food has a flavour. We mentioned that every sound is made of a fundamental plus its overtones, and we can look at this in a very orderly (and idealized) form called the harmonic series. The harmonic series describes the way overtones occur, and goes 1f (=the fundamental), 2f, 3f, 4f, 5f, 6f, 7f, etc. Depending on the way a sound-maker is built, certain overtones will be more pronounced than others. Also, some of the overtones may be a bit out of tune from the idealized harmonic series. A beautifully crafted instrument – much like fine wine or other food on which rich people are sustained and which slightly-less-rich people quickly Instagram to show their friends how rich they want to appear – has well balanced overtones that fall very close to the idealized harmonic series. All of this composes the timbre – that is, the flavour – of the sound.
We have no trouble at all distinguishing the sound of a guitar from the sound of a person’s voice, and we rarely have difficulty distinguishing tomato sauce from vegetable soup; we experience all the ingredients that compose the whole, and the specific combination of which overtones are strong and which overtones are weak gives a sound its distinction.
Once we understand this, we can get into some really weird areas that involve hearing things that aren't there. As it turns out, we’re extremely acquainted with hearing the harmonic series and we easily recognize the entire pattern even if parts of it are missing. This happens especially when we listen to music that contains low frequencies (think of the deep bass sound on a house kick drum) through speakers incapable of producing low frequencies (think of the speakers built into your computer). What happens is that we hear only the overtones and we actually fill in the fundamental. It's like tasting tomato sauce that’s missing the tomatoes – we taste the other ingredients, know what tomato sauce should taste like, and then ignore the fact that something important is missing. We imagine we're tasting a complete tomato sauce!
Though, if this actually happened with food we would say the chef is a lunatic. Yet with sound we're far more easily tricked: we experience the equivalent of what I've just described above, and we hear the fundamental frequency even though it isn’t there. The scientific explanation of how we perform this filling-in is a mystery, yet like many mysteries it’s very real. We know it's very real because it explains our experience with telephony, as people once called it. The fundamental frequency of a male voice is around 150 Hz, and a female voice around 300 Hz, yet phones only work with frequencies in the range of about 300 Hz - 3400 Hz. How, then, do we distinguish between male and female voices (because really, in the words of John Lennon, it isn't hard to do)? We can distinguish because we hear the frequencies come through the phone, and certain frequencies suggest themselves to be the overtones of a missing fundamental. And then we fill in the frequencies that are missing. It’s instant and it’s magic.
This also isn't restricted to phones. Waves Audio – an Israeli company that makes audio processing software – has a plugin called MaxxBass that actually exploits this business of missing fundamentals. As their website proclaims, "MaxxBass uses psycho-acoustics to calculate precise harmonics that are related to the fundamental tones of sound. When these harmonics are combined, it creates the effect of lower, deeper frequencies."
What this marketing mumbo-jumbo means is that if you want a sub-bass with a fundamental tone of 52 Hz but you know that 97% of your listeners won't have speakers that can produce those gut-wobbling frequencies, you can apply a little bit of MaxxBass and it'll calculate the overtones that would be present in the harmonic series for a 52 Hz sub-bass tone, and it'll introduce those in a range that an average loudspeaker can handle. The other nice advantage to this sort of audio trickery is that you can listen to what seems like bass-heavy music with much less risk of noise complaints from your old neighbours.
At the end of the day, this business of missing fundamentals takes us into a really interesting area because we’re designing sound that isn't what it claims to be. We study how we hear, and then figure out ways to trick ourselves into hearing things that aren't there. But isn't that also the case with food? What are "natural and artificial flavours"? What's aspartame? What's MSG? They're all examples of ways in which we've designed ways to trick ourselves into experiencing certain things that aren't there.
So what? The answer to this lies in the dark. Literally. When we think that sound is only about the ears, or food only about the tongue, we end up being very capable of tricking ourselves. But as I learned sitting at O.Noir, chewing, smelling, feeling, tasting, and hearing some delicious and mysterious food, our senses work together when we're actually paying attention to them. You might be able to trick the ear into hearing bass that isn't there, but you can’t trick a body into feeling bass that isn't vibrating it. You might sell some extra speakers in the name of efficiency, but the real magic happens when the senses are working together. I had to eat a meal in the dark to figure that one out.
Jordan Mandel is a Digital Media Lab Instructor at the UW Stratford Campus, and writes for this blog regularly. His hobbies include untangling headphone cables, Yahtzee, and treadmill walking. More of his work can be found at jordanmandel.com/blog, which is home to the award-winning satire rag, The Outa Times.