EarthSongs

Description

EarthSongs was a prototype Natural History experience for the Magic Leap One mixed reality platform. It was directed and produced by Mitch Turnbull, who leads the field in 360 and immersive documentary alongside the Bristol based XR studio All-Seeing-Eye. It was initially funded as part of the South West Creative Technology Network Immersion Prototypes.

EarthSongs took audiences on "a beguiling, playful and creative interactive audio-visual exploration of natural soundscapes through play, experimentation and creation while highlighting the beauty and importance of sounds in nature to our wellbeing and cultural heritage".

By making use of early stage Augmented Reality and spatial mapping, scenes and textures of natural habits across the world overlayed themselves onto the geometry of your physical environment while binaural atmospheres grounded you in them. Abstract visual motifs inspired by the plummage and decorations of birds and animals can be moved and interacted with, while audio reactive animations are driven by their isolated calls.

I was involved as the audio director, guiding the team on best practices for audio in XR and would go on to create the binaural atmosphere design and prepare all the audio assets for game-engine playback. This involved a mixture of audio recovery techniques (spectral enhancement and editing) from ornathogical databases as these sources are often digitally degraded by the heavy mp3 codec compression and recording practices. In conventional film the fixed mix of the soundtrack would afford good masking of any background noise on a sound source, but in game based application where a source can be auditioned and interacted with at close range, this is not so easy. Furthermore game applications generally apply their own heavily lossy compression (commonly Vorbis) which would result in a double compression loss in audio quality. In practice, the Magic Leap One's built in speakers were extremely toppy and lacked rich bass tone. On headphones the soundscape is reasonable but through the built it it comes across as shrill. Unfortunately- it is often the latter in which the experience is showcased to afford an easier audience on-boarding.

EarthSongs went on to critical success on the 2020 XR festival circuit, as a Finalist of Virtual Noise at Kaleidoscope/SXSW, and the official selection at Wildscreen Festival, Climate Week NYC, Simple Things, XRLO/Tate Modern, i-Docs, and EarthX Film Festival.

Video

EarthSongs - a mixed reality prototype for the Magic Leap One device from bramble media on Vimeo.