Spatial Effects

In linear sound design, such as in film which is where I mostly operate, there’s a constant requirement for the audio to match what’s on screen and sound the same to every listener. Unlike in games, where spatialisation of sounds is handled in the game’s audio engine, film requires sounds to be spatialised before they’re played back.

What is spatialisation? Basically, it’s placing and moving sound in the stereo space. If a car goes across the screen from left to right, you’d expect the sound of that car would go from left to right. If the camera’s point of view has a fire burning on the ground, you’d expect to hear the fire below and in front of you. Realism is the goal.

So, how do we do that artificially? At it’s most basic, there’s 3 ways, or rather 3 axes, I work on: panning, height, and depth.

Panning you’re likely familiar with already, it’s simply moving a sound side to side in the stereo field. The car example above is, at its most basic, panning.

Height is moving a sound vertically, from above you to below you. The fire example uses this, and is typically handled as a crossfade between slight highpassing, for higher sounds, and lowpassing for lower sounds. Clap your hands above your head, then in front of your head, then below your head. The one above likely sounded bright, the middle likely sounded full, and the one below likely sounded slightly dull. Same concept!

Depth is basically distance from the listener. The further away something is, the narrower it is in the stereo field, the quieter it is, and the more objects between it and the listener affect the sound. This is more complex than the last two, and often works in tandem with the last two, but can be simulated with a narrowing of the stereo space by summing the two channels a variable amount, and a comb filter being spread further and further based on distance.

There’s also a few other considerations beyond those 3. If the sound is in an enclosed space, or against a hard surface, there will be reflections against the walls, floor, and ceiling as applicable. This is generally called reverberation, or just reverb. Direct reflections against surfaces are referred to as early reflections, whereas the diffused sound following that, essentially multiple delayed versions of that sound, is what is typically known as ‘reverb’.

Doppler is also important for any sound that moves closer or further away from the listener. When coming closer, the perceived frequency of the sound will increase, and when it moves away it will decrease. The angle at which this occurs affects how much the frequency seems to change. If a sound is very far away and comes very close, the perceived frequency delta will be much greater than that of a sound coming from nearby to nearer.

Finally, resonance of the space or objects nearby affect how things sound to the listener. Using a modal resonator, one can easily place harmonic nodes and select material of the space based on its size, shape, and materials shown on-screen, often in conjunction with reverb.

The tools I use for this are varied. I have a custom Bitwig Grid preset I made that does all of this with just a few knobs, and can be modulated based on what’s happening on-screen. There’s also plugins like Ambeo Orbit, which isn’t particularly high quality but can create similar effects. I also use a series of hardware modules, including Make Noise Erbe-Verb, Worng Soundstage, Mutable Instruments Warps with Parasites for the doppler panner mode, Mutable Instruments Rings and Elements as modal filterbanks/synthesizers, and Calsynth Monsoon, based on Mutable Instruments’ Clouds, for spatial stuff and actual synthesis. Play around and see what works for you!