In a land that never was and a time that could never be, a boy stared into the eyes of a fantastical creature. Around them, a magical forest, in front of them, a magical journey.
is a sound-based real-time CGI VR fairytale for people young and old--inviting audiences into a magical space of imagination, where rustling leaves of an autumn forest are also the footsteps of something familiar, yet strange.
It is written by Shelley Silas, directed by award-winning VR artist , with cutting-edge 3D sound produced by sound designer , the and ΒιΆΉΤΌΕΔ R&Dβs audio team.
The work will premiere in April at the , which focuses on cutting edge artworks that explore new uses of media, highlighting innovation.
The dynamic for The Turning Forest showcases the ΒιΆΉΤΌΕΔβs vision for new listening experiences over headphones and how this can be used in . Weβll write more about the technical aspects of the sound and the development into a virtual reality experience later. But since this project started its life as an audio piece, first hereβs the story of how the extraordinary soundtrack came to be made.
The studio recording sessions occurred last year in Salford
The soundtrack for The Turning Forest was commissioned under the EPSRC Programme Grant S3A: and the ΒιΆΉΤΌΕΔ as part of the .
Frank Melchior, ΒιΆΉΤΌΕΔ R&Dβs Lead Technologist for audio says:
"Our work is all about enabling the best story telling in the world. Sound provides one of the most powerful and direct routes to our audiences. Our team is laying the technical foundations for amazing sound that will immerse people in new listening experience. Whether itβs for telling our stories using virtual reality headsets, car radios, your mobile phone or your living room β you cannot close your ears."
The starting point for The Turning Forest was a great story. The S3A project presented writer Shelley Silas with the challenge of creating a short drama specifically designed to demonstrate the possibilities of .
Shelley says: βI normally start writing with characters and then story and give very little attention to the technical side, as that comes last, and I don't really have much to do with it. The challenge for The Turning Forest was to begin with the specific technical and audio requirements and finding characters and story that would work well organically within those parameters without forcing them to work. Rather than the brief being limiting, I found it extremely liberating and was surprised the whole way through the process. It was a challenge I wish I'd had many years ago. It has given me a new way to write and exciting possibilities.β
The team captured 3D field recordings for the production
Working with ΒιΆΉΤΌΕΔ R&D and S3A researchers, sound designer then embraced the challenge of creating 3D soundscapes and creatures, and unusually dealing with height and distance for sound.
Eloise says, βSuddenly the way our creature moved became important. The boy running needed to be realistic. The creature needed to leap over our heads. The audience needed to hear the heaviness of the body, the vast size of the creature, the depth of birds in the forest, the different layers of ice and water in our icy landscape. Musical notes needed to escape from between our creatureβs teeth and float away. The sound needed to encompass the listener.
The way we recorded was also a different approach. The process was object based, so it was about capturing individual sounds and then placing them in a 3D space until the world contained 100s of sound objects. Overall, it was about storytelling, and creating a believable, magical world.β
Composer Jon Nichols created the music for the piece, components of which will become interactive in the VR version.
The original spatial sound mix was created in ΒιΆΉΤΌΕΔ R&Dβs listening room on a multichannel speaker array. The story takes place around you, immersing you in the sounds of the forest. The soundtrack was created using , which means the content can be replayed on different sound reproduction systems without a complete remix. The VR version to be shown at Tribeca will use dynamic binaural sound, reproduced on headphones.
The ΒιΆΉΤΌΕΔ R&D listening room where The Turning Forest was first mixed on a 3D loudspeaker rig
ΒιΆΉΤΌΕΔ R&Dβs binaural expert Chris Pike explains:
βBinaural techniques allow us to create a spatial sound experience for headphone listeners. Weβve been working on this topic for several years and have built up a good understanding of the technology and production techniques. Recently we produced special binaural mixes for an .
But with VR systems the spatial sound experience has a new dimension, we can update the binaural sound processing in real-time as the listener changes orientation, just as is done with the video content. This creates a soundscape around that listener that you can freely rotate within and really enhances the feeling of immersion in VR. This is what we call dynamic binaural sound.
With the software tools that weβve built weβre able to produce a rich 3D soundscape and present it on headphones in dynamic binaural sound. In this case a virtual reality headset provides us with the listener tracking as well as displaying the imagery.β
In many VR projects sound is not the first consideration, and compromises have to be made. The challenge we set Oscar Raby was to start with this unique sound track and to visualise it β creating a magical fairy tale world. You will see and hear the results in April.
- -
- More on Virtual Reality and 360 Video:
- from day
- ΒιΆΉΤΌΕΔ R&D - 360 Video & Virtual Reality
-
Immersive and Interactive Content section
IIC section is a group of around 25 researchers, investigating ways of capturing and creating new kinds of audio-visual content, with a particular focus on immersion and interactivity.