Creating A Personalised Documentary: What The Audience Told Us

Published: 23 January 2020

Nick Hanson
Development Producer

In 2019 麻豆约拍 Research & Development commissioned the personalised object-based documentary, Instagramification. We've previously written about the making of the pilot, but now, following extensive user testing, we can share some of the wider lessons specifically around the editorial considerations and the audience's response. The intention is to help us better understand the challenges and opportunities around the use of personal data to customise factual content to individuals.

To recap, the documentary was made by using our StoryFormer tool. Different versions of the same story are served up based on the viewer's personal preferences. These include which UK nation they live in, whether they like celebrity culture, sport or technology, and whether they would prefer to be 'informed' or 'entertained'.

Our research questions were:

Through StoryFormer, can we create object-based experiences that allow for greater personalisation of the same story?
Can we create coherent content experiences that fit together in a meaningful way?
Does it lead to efficiencies such as 'sweating the assets' or are we just creating more content for fewer eyes (as with interactive video)?

Some of these questions were partly answered during the development and production process, and we addressed them in our earlier blog post. However, to answer these in full required further testing with audiences.

麻豆约拍 R&D - Behind the Privacy Notice: Audience Data in Personalised Experiences

We did this in three stages. Firstly we looked at the responses to the survey questions on 麻豆约拍 Taster. The questions we asked for our research demonstrated a positive response to the format:

Moreover, 67% of respondents to the survey believed the 麻豆约拍 should be making "more stuff like this". It has also been the top-rated pilot on Taster since launch with an average rating of 3.6 out of five, with 36% of users giving it a five-star rating.

However, Taster can admittedly be a self-selecting user group who are actively open to new formats. Also, although these results appear to show that the audience responded positively to the pilot, they don't tell us precisely what they liked or if they have fully grasped the true nature of the personalisation.

So we conducted a second study, this time using the online survey tool UserZoom. We asked 60 participants with a 50/50 male/female split, all between 18-34, from mixed socio-economic backgrounds and all familiar with Instagram, to record themselves watching the documentary and comment out loud throughout. We followed this up with a series of survey questions:

Generally speaking, the UserZoom results were also positive. However, in terms of allowing us to understand the value and effectiveness of the personalisation, the survey was inconclusive. For example, we asked users if they preferred a personalised documentary to a 'traditional' non-personalised one, but of course, our users had nothing to compare like-for-like (hence no clear opinion). We will revisit this point in the third and final study.

The comments made by users were, for the most part, extremely positive:

"The choices I made were absolutely reflected in the content, and that's what kept me watching."
"100% reflected what I would want or expect to see."
"This has taken interaction to a whole new level."

However, Instagramification did not win everyone over. Some had valid criticisms:

"I think the questions need to go into more depth so it can be properly tailored."
"I didn't feel it was personalised. I said I wasn't interested in celebrity culture, and there was quite a lot of it in the documentary."
"Focusing on select topics within the documentary is very dangerous鈥eople will not learn outside their world."

The final comment alludes to the potential for personalised content to create echo chambers or reinforce stereotypes, an incredibly important and valid point raised by several users. Our study is not exploring this in-depth this time, and we will certainly revisit this, but for now, we're focussing on how we tailor content based on our research questions stated above. The comments on depth and level of personalisation, however, are directly relevant to our study.

When watching people explain what they liked or didn't like about Instagramification, it became clear they didn't always realise how or when the personalisation was happening for them. For example, if the user stated an interest in celebrity culture, they are often happy to see many celebrities featured in the documentary. If they said they were not interested in celebrity culture (as per the comment above), they were often disappointed and felt it wasn't tailored to them.

In fact, all versions of the documentary feature celebrities to some degree, the difference is in the length and depth of those stories and the tone in which they are covered. In short, audiences often made the wrong assumptions about what was tailored to their preferences, even when they felt very positive about what they were seeing.

On the whole, we were satisfied that we had created a documentary with multiple coherent pathways, all of which made narrative sense. We were also happy that we had done this without making hours and hours of content that would not be seen by most viewers (as is the case with interactive video). But we still wanted to better understand how we use the data from our audiences to create experiences that are meaningful to them as individuals.

Our third and final study involved working with design agency . We recruited 10 participants for one-on-one 60-minute qualitative research sessions. Each participant watched the video, selecting their personal preferences and followed up with a conversation about what they had just seen. They then watched again but chose the exact opposite of their original preferences. This was followed up by another discussion. This method allowed us to make comparisons between what the user thought about their personalised version and an alternative version.

The results again saw a generally positive response to the documentary with an appetite for personalised content, though significantly, most emphasised they would want transparency and control over how to do this. The tone and language of the two presenters had a big impact, and the structure and flow of the story were generally positive. Significantly, most (though not all) participants felt the first version they viewed (the one that used their data) was preferable to the second version they saw. This demonstrates that through object-based media, we can create different versions of the same story where each individual receives a version more relevant to them.

However, we're still not 100% there. As with the previous study, initially, participants didn't work out exactly how or which parts of the documentary had been tailored to them. Significantly, even when it was tailored to them, it wasn't always done in a way they felt had a positive impact. For example, those who stated they liked sport received an extended version of a story on footballer Cristiano Ronaldo. However, when questioned, some said that just because they liked 'sport' doesn't mean they like 'football'. Or just because they like 'football' doesn't mean they like Ronaldo. Or just because they like Ronaldo as a footballer doesn't mean they want to hear more about his family life as was the case in Instagramification. This demonstrates perhaps the most significant editorial challenge for object-based storytelling 鈥� how can we translate the data into meaningful content?

Summary

What does this all mean? It's clear we need to get better at knowing what data to collect. We also need to get better at understanding how that data translates into meaningful content experiences.

Looking back at our research questions, success in creating scalable and repeatable personalised factual content is finding the 'sweet spot' between some potentially conflicting things: retaining the story structure, providing meaningful personalisation and getting value from the assets.

As we stated previously, personalised storytelling doesn't need to be about creating radically different versions for each viewer. Instagramification branches off into different routes through the story at the point of making one big decision, but then works through a series of 'micro-personalisations', where lots of small changes combine across the course of the story for a more substantial effect.

The pilot was largely successful in demonstrating how object-based media can create a different version of the same story around the needs of diverse audiences, but it also surfaced the challenges of doing so. There is still a long way to go, but Instagramification represents a big step in the right direction.

麻豆约拍

Accessibility links

Nick Hanson

Rebuild Page

Useful links

Theme toggler

麻豆约拍