One of the great things about the ΒιΆΉΤΌΕΔ's Connected Studio is spotting talented and like minded people who you may be able to collaborate with in the future. Mudlark are one such company who had come on Connectedstudio a few times and had pitched ideas similar to some of our research work.
After a few meetings, we revealed some of our thinking with Perceptive Media and a project was formed.
Here is collaboratively working with ΒιΆΉΤΌΕΔ R&D on an exciting new project from an external company point of view...
Take it away, Greg's Mudlark...
Ian Forrester of ΒιΆΉΤΌΕΔ R&D approached us with an idea he had about taking the experiment in new editorial formats prototype into a next stage. Earlier in the year, we'd proposed a 'perceptive' experience for the CΒιΆΉΤΌΕΔ homepage, taking in data feeds to change how it was presented and incorporating playful elements (eg blowing away clouds if it was a rainy day near you), so ΒιΆΉΤΌΕΔ R&Dβs Perceptive Media was a natural fit.
, ΒιΆΉΤΌΕΔ R&D's original Perceptive Media experiment involved a two-character radioplay - one of whom was acted by a text-to-voice synthesiser - which uses data to adapt the content to the user's context. eg using location data to fill in dialogue gaps such as weather or place names ("you can do anything, you can go to the Imperial War Museum" - if you're in Salford/Manchester").
The main issue ΒιΆΉΤΌΕΔ R&D faced when they have tested the Perceptive Media radioplay is that the audiences could not separate the web-based content from the shiny screen - the audience would be waiting for something to happen on screen, rather than listening to the audio content. Not ideal for a radio play listening audience. The content needed to be free from the screen, to see how audiences really reacted to Perceptive Media.
The Brief
The possibilities of networked things and Perceptive Media are vast, opening up content to be βremixedβ live or contextualised for an audiencesβ environment and context. Ian's brief to us was to create a domestic radio-style object that would playback the radioplay and pull in data to adapt it to the user's context.
This would enable audiences to act naturally around the radioplay content, and for ΒιΆΉΤΌΕΔ R&D to understand more about Perceptive Media and audience behaviour. We decided that the object itself would provide further data feeds - using analogue electronics and sensors - in addition to the web-based data already being collected.
The Domestic Environment
Influenced by networked objects that fit seamlessly into the domestic environment and serve a natural function - such as Voy's Ugle, Russell's Bikemap, the Good Night Lamp and SkrekkΓΈgle's own radio object, Plugg - we wanted to ensure that the object was a part of the home, not an intervention.
The Perceptive Radio is a behaviourally-driven design. To do this, we spent time role-playing, listening to the radio, and documenting how we behaved around them - as well as what else was going on around the home. There were three key scenarios that most frequently played out:
- Pottering and moving about in one room (eg making a cup of tea, walking to and from cupboards)
- Pottering and moving in and out of many rooms (eg housework, gardening)
- Settled down to properly listen to the radio (eg armchair with a cup of tea and pet)
The first two scenarios point at partial attention - with radio as an accompaniment - whilst the third is a clear, focused attention scenario.
Designing the Radio
The design challenge was to demonstrate how a networked object could deliver tailored media experiences that are sympathetic to domestic environments, without being disruptive or jarring. Marrying the contexts with environmental behaviours was the next stage in domestic design:
- What would a user want a radio to do when they are moving around a room frequently?
- What would a user want a radio to do when someone calls them up?
- What would a user want a radio to do when they are settling down to relax?
In order to avoid any unnecessary diversions or wrong routes, we settled on four key design rules:
- Behaviour of the audience as control inputThe Perceptive Radio is to be designed as an object that is sympathetic to the domestic environment - something that sits comfortably and naturally in a home - and the behaviour of the listener affects the output. Passive, not active, inputs.
- No physical interactionsWhilst the box itself has many affordances that could be used as sensor inputs - such as orientation, angle, touch - it was key that the object itself would be secondary. A function, rather than a tool. So, beyond obvious controls (on/off, play, master volume), the user's behaviour and environment would be the only control inputs.
- Meaningful inputsAs the user is the control input, the controls have to be meaningful. It is important that those controls are behaviours that already exist, and not attempt to create a new invisible control system. We focused on ambient sound and natural movement rather than forced gestures.
- Believable responses/effectsFor the Perceptive Radio to fit comfortably into a domestic environment, the output (effects) from the interactions must be believable. There are many variables that could be mapped to interactions - such as pitch or dialogue pace - that would be jarring to the listening experience, rather than being sympathetic to the audience. All effects must be believable and beneficial to the user.
Four Design Scenarios
- .In this scenario, the user is moving around the kitchen, boiling the kettle, collecting mugs and milk whilst they listen to the radio. As they move away from the radio, the volume changes so that the user can hear a consistent volume level at all times.
-
.In this scenario, the user is moving around the kitchen and filling up the dishwasher whilst they listen to the radio. As the dishwasher kicks in and there is a large spike in ambient noise, the radio alters the depth of the audio playback to foreground the most important element (eg the actors voices).
-
.In this scenario, the user is settling down to a cosy evening listening to the radio. They stoke their fire and turn off bright lights in the room. The change in light levels in the room causes the radio to adjust the EQ levels, cutting most of the treble, to make a more relaxing listening experience.
-
.In this scenario, the user is settled down to a cosy evening listening to the radio before being interrupted by phone call. The large spike in noise coupled with the user's movement away from the radio, causes the radio to pause playback of the audio.
The Technical Approach
As an object, the Perceptive Radio is a bit of a trick on the user β it looks dumb, but is pretty powerful. The broad remit for the technical and object-design side was that it needed to be βsmall enough to fit in a box, fast enough to use the Web Audio APIβ whilst processing data feeds from analogue electronics. Working with Adrian McEwen from MCQN LTD in Liverpool to successfully put the internet into a thing, our first hope for running Breaking Out was a smartphone working with a Raspberry Pi and Arduino.
Unfortunately, the Perceptive Media radioplay does quite a lot of heavy lifting β including Web Audio API and performing real-time object-based processing (putting a load of reverb on the text-to-voice synth) β which requires a good quality browser such as Chrome and a decent strength computer. This kind of processing power is sadly not available in small things and β after testing Breaking Out on Android/iOS mobile devices, Netbooks, Chromebooks and an array of Raspberry Pi β we decided to build a compact Mini-ITX system (a 7β³ PC) to power the Perceptive Radio.
Using a Beaglebone board for the analogue electronics hub, we had a slightly larger than desired technical solution with enough power to process Breaking Out and analogue inputs comfortably.
The Object
The Radio object itself needed to accommodate a 7β³ PC system, sensitive analogue electronics sensors and have capacity to add further sensors/functions later down the line and crucially look natural in a home. We brought Patrick Fenner on board β an βopen source engineerβ who makes 2D things 3D β to help βmake the boxβ.
Patrickβs experience and speciality in rapid prototyping with lasercut materials took the direction towards a more Digital Radio design, using finger jointed edges, birch plywood and an acrylic fascia.The addition of chrome suitcase corners and a carry handle are there for extra structure support and to enable Ian to carry the box comfortably on his travels to demonstrate it.
....Thanks to Greg Povey of Mudlark for a great explainer post...
I would like to thank... Greg Povey, Richard Birkin, James Rice of Mudlark. Adrian McEwen of MCQN. Patrick Fenner of Deferred Procrastination. Supportive collages Tony Churnside, Jasmine Cox and Adrian Woolard.
Fork the code on Github: the /