For the last couple years, I have been working with 360 degree video and immersive audio. Basically, my GoPro Fusion has two wide angle lenses that when stitched together give you a 360 degree video. Accordingly, the audio is also 360 degrees, typically referred to as Ambisonic.
One of the features of this type of audio is “headtracking.” As the viewing perspective changes, the audio follows (tracks) the video. You can imagine gaming with a headset on, turning to look at something on the side, and the audio of that event increases in volume. Nifty stuff.
On a recent trip to Tofino, on Vancouver Island, I shot a 360 degree video of a walk along one of the beautiful beaches there. I finally got around to writing a song to attach to the video.
The song was engineered entirely as a 1st order ambisonic recording. So three mic positions — W (omni), X (front to back), Y (left to right) and Z (top to bottom).
Here is a link to the video with ambisonic sound. Notice how the relative volume of the mix changes as you pan around the image. I’m not sold on
I’m not sold on head tracking for music. As different instruments come into focus, it messes with with mix. So, while its amusing, not sure its so great for a song.
To that end, I took the ambisonic mix and generated a binaural mix, stereo but engineered to be more like how our ears actually pick up sound and hour brain processes it. Here is a link to the same video with a binaural mix. In this case, the sound does not move as the image is panned.
Now, that sounds really nice. To me, it sounds more open than a typical stereo mix.
Going forward, I will be engineering the audio I work on using only immersive formats. I am working on one right now that is being engineered and mixed in Dolby Atmos 7.1.2. This is largely a result of the robust support Nuendo 11 has for Atmos in the latest release. So far I am liking it very much. Once I get the song together, I’ll post it here, along with a more complete explanation.
Until then, I’ll leave you with this. It is an orchestral performance in 360 degree video with immersive sound. The audio was recorded using multiple Zylia ZM-1S digital microphones. The video is a promotional made by Zylia to demonstrate their microphones.