In November 2017, we released The Inspection Chamber, a conversational audio drama available now on Amazon Alexa. Our aim was to explore long-form entertainment experiences for voice user interfaces and smart speakers.
The piece is a science-fiction comedy where the user is a character in the story and has an effect on the narrative. After the experience was released, we had lots of positive feedback from the industry, but we wanted to find out what audiences felt about conversational audio drama to see how we can improve our ongoing explorations in this field.
We've made the detailed report from our user testing available; read on for a look inside our process and a summary of our findings.
Designing the user study
We have some broad analytics about use of The Inspection Chamber, but we wanted to gather detailed information about how people feel about the experience and their time spent with it - we needed to conduct a qualitative user study. We set ourselves a broad research question: do users respond favourably to long-form fictional interactions with a voice experience? We then broke this down into four key themes of interest:
- Interaction format & patterns: do people like the interactions in the piece?
- Narrative and storytelling: do people like the story and plot?
- Attention span: how do people feel about paying attention to a 20-minute audio experience?
- Behaviour change: does the experience force people to change their behaviour? How do they feel about that?
Joanne designed an interview guide to tease out people's feelings on these topics, then we tested it on each other a few times to check that it flowed smoothly and covered everything we wanted. We were ready to interview some users!
The interviews
We recruited 24 participants, who already owned an Amazon Alexa and were interested in audio drama. We recruited evenly from four age groups: 20-30, 31-40, 41-50 and 50-60, across an even mix of genders. The participants used The Inspection Chamber at home β€” we wanted them to listen in their natural environment β€” and we then followed up with a 45-minute phone interview. The interviews were transcribed and anonymised, then we pulled out relevant quotes from each interview for further analysis.
Analysing the data
After conducting the interviews, we had a lot of quotes to sift through. Andrew and Joanna did an heroic job of analysis, grouping quotes up into the four themes we'd identified earlier and further sorting them them by whether they said something positive, negative or neutral about that theme.
Here are a few headlines from the detailed results of that study:
- Participants weren't fans of science fiction on the whole, so didn't respond well to the story. They did however, enjoy interacting with it, and most could see themselves enjoying a story in a genre they did like.
- Participants wanted much greater influence over the plot, and to feel like they were shaping the story. We used quite a subtle interaction model for The Inspection Chamber, based more on immersive theatre (changing details, the texture of the story) than causing large changes in the direction of the narrative, and on on the whole people wanted to see their actions have a larger effect than this.
- Participants had an appetite for more personal touches: use of their name, music from their playlists, their location or time of day.
- Participants enjoyed the novelty of the format, and wanted to see more things like this made.
- Some participants used the piece with their family, and enjoyed the group activity.
Recommendations
Based on the study, we've got a few recommendations and things to try in our next voice-interactive audio productions:
- Create a personal experience using the users profile.
- Create a personal experience using external data sources and someone's context.
- Make people feel like they have a bigger influence on outcomes.
- Create more obvious changes based on user interactions.
- Use on-boarding - at the beginning and set the scene, provide expectations.
- State how long the drama is and how far through the participants are.
- Explore the potential of a group activity / family or social activity.
- Explore the potential of a short episodic format.
We've released the results of the study in the hope that they'll be useful to other designers of entertainment voice experiences. Let us know if you find them helpful in your own work!
-
Internet Research and Future Services section
The Internet Research and Future Services section is an interdisciplinary team of researchers, technologists, designers, and data scientists who carry out original research to solve problems for the Βι¶ΉΤΌΕΔ. Our work focuses on the intersection of audience needs and public service values, with digital media and machine learning. We develop research insights, prototypes and systems using experimental approaches and emerging technologies.