Â鶹ԼÅÄ

The Â鶹ԼÅÄ Philharmonic Pick A Part web application

Pick A Part is the latest in a series of orchestrated trial productions that started with audio drama The Vostok-K Incident in 2018 and recently saw the release of an orchestrated episode of 1927's Decameron Nights. You can read more about how we worked with 1927 to make Decameron Nights.

Â鶹ԼÅÄ Taster - Try Pick A Part for yourself

Adapting to lockdown

We've been discussing audio orchestration with the Â鶹ԼÅÄ Philharmonic for a while now - it's something that we've been interested in the potential of, but it's been hard to actually put together. One of the challenges is that orchestral music is normally recorded with all of the musicians playing together at the same time, whilst audio orchestration required object-based audio - that's having access to all of the individual parts that make up the piece.

That all changed when the country went into lockdown earlier this year, and musicians at the Â鶹ԼÅÄ Philharmonic had to adapt to making music from their homes. Suddenly, we had unprecedented access to recordings of individual musicians playing their parts to be edited together later. So we jumped on the opportunity to see if our idea of applying device orchestration to orchestral music would work.

Simon Highfield, a technical producer at the Â鶹ԼÅÄ Philharmonic, sent over the recordings from the musicians so we could start experimenting. And we were immediately very excited by how well it worked. Not only was this an immersive listening experience just by connecting one or two phones, but it was also novel to be able to explore a piece of music in such detail.  

Simon Webb, director of the Â鶹ԼÅÄ Philharmonic:

There are so many potential applications for Pick-a-Part; this is the Â鶹ԼÅÄ at its very best, collaborating, innovating and sharing. Whether in an educational setting exploring the inner workings of a string quartet, exploring different audio mixes of a recording, or playing along with professional musicians in your own home, this is a brilliant piece of kit.

How we made Pick A Part

Pick A Part is the latest production that we've made with our new production tool, Audio Orchestrator. With Audio Orchestrator, you can import your audio files, set up rules that determine how they're assigned to connected devices, and export a prototype application.

To make Pick A Part, we modified the template application that is produced by Audio Orchestrator to add some new features, including:

• showing a picture of the selected instrument on the connected devices;
• automatically assigning an instrument once a device has been connected; and
• displaying a representation of the instruments assigned to each connected device.

We're using trial productions like this to experiment with new features and see what we might want to feed back into the Audio Orchestrator software.

If you'd like to find out more about Audio Orchestrator, you can request access through Â鶹ԼÅÄ MakerBox, read our blog post, or have a look at the .

The Audio Orchestrator production tool being used to add rules that determine how the audio objects are allocated to connected devices.

Keeping instruments in sync

I mentioned above that one barrier to using audio orchestration with classical music was that we needed access to the individual instruments. The second challenge that we needed to address was that music requires very accurate synchronisation between devices. The synchronisation framework that we use is accurate to within around 10–20 milliseconds, but there's sometimes additional delay added by devices, which can reach up to 200 milliseconds. That's OK for the types of sounds in the drama productions that we'd previously released, but not good enough for music.

To get around this, we developed a manual calibration stage so that listeners can easily correct the time-alignment for their own devices. Danial Haddadi, an , ran an experiment to determine how well listeners can align two devices, and what method is easiest for doing this. Based on the experimental results, he implemented the calibration tool that we used in Pick A Part. The same speech content is played from the main device and an aux device, and the listener is asked to adjust the time delay on the aux device until the two devices are in sync.

Pick A Part is the first orchestrated music experience that we've released, so we're interested to find out about how well this works and how easy it is to use the calibration tool.

Calibration mode on a connected device - use the slider to match the audio playback on two devices

What's next?

We've got a few more trial productions made with Audio Orchestrator in the pipeline, including a spooky Hallowe'en drama with the Â鶹ԼÅÄ Writersroom. Keep an eye on Â鶹ԼÅÄ Taster if you'd like to hear more. Hundreds of creators have requested access to Audio Orchestrator, and we're really looking forward to see the range of ideas that they develop. This will help us to evaluate the production tool, decide on new features to implement and assess the potential for audio device orchestration. We're also involved in research collaborations at the universities of and looking at technical and creative aspects of device orchestration.

-

Â鶹ԼÅÄ MakerBox - Audio Orchestrator

Â鶹ԼÅÄ R&D - Vostok K Incident - Immersive Spatial Sound Using Personal Audio Devices

Â鶹ԼÅÄ R&D - Vostok-K Incident: Immersive Audio Drama on Personal Devices

Â鶹ԼÅÄ R&D - Evaluation of an immersive audio experience

Â鶹ԼÅÄ R&D - Exploring audio device orchestration with audio professionals

Â鶹ԼÅÄ R&D - Framework for web delivery of immersive audio experiences using device orchestration

Â鶹ԼÅÄ R&D - The Mermaid's Tears

Â鶹ԼÅÄ R&D - Talking with Machines

Â鶹ԼÅÄ R&D - Responsive Radio