Â鶹ԼÅÄ

So this year the Â鶹ԼÅÄ has done something exciting for the 2018 FIFA World Cup: for the first time all the games we broadcast on Â鶹ԼÅÄ One have been available in virtual reality.

For the Â鶹ԼÅÄ it’s important to innovate and explore new experiences for audiences now and in the future. We saw this as the perfect time and opportunity to test what our audience expects from a virtual reality sport app. There were a number of areas we were keen to learn more about including audience expectations, device preferences, the amount of time people spend using VR, how popular the platform is and what the demographic of a typical VR user is.

To take a look into this we have partnered with FIFA and its official VR suppliers in and to create stunning immersive environments our users can dive into and watch the action. Partnering with these companies allowed us to learn a lot whilst providing a great new platform for our users and so far nearly 350,000 people have taken advantage of this opportunity.

In this post we are aiming to show the work that has gone into launching this app and what we have learnt along the way.

How we delivered the apps: by Rebecca Heapy — Â鶹ԼÅÄ Sport Apps Project Manager

Six months to customise an out of the box solution — easy right? Then as the weeks tick over you realise there is much more to it than a collection of posters and a couple of end-to-end tests. You’re now working with more suppliers based in multiple geographical locations. Internally the number of teams / departments involved has also grown as you’re…

• setting up CDN networks
• settling contracts and agreements
• ensuring security standards meet our levels
• prepping editorial team on the new content interfaces

All this on top of supporting and developing the , one the biggest Sport apps in the market.

So how can you ensure successful delivery when so many elements are out of your control? Without sounding cliché, communication was key.

Initially, direct contact with the teams building the interfaces was limited and irregular. The first obstacle we had to overcome was establishing governance and routines with all parties, fostering transparency and collaboration. Forging a strong direct relationship with the development team was key and helped us gain a better understanding of our role in this project and get closer to the product itself.

When equipped with more knowledge we were able to efficiently customise the environments, promotional elements and (probably the biggest of all our tasks) test the app. It was paramount that before releasing, before even announcing the apps, that we were 100% confident in the application and the services that feed in to it so that it meets the standards our audience expect.

How we made sure they worked: by Vinny Sebastian —  Â鶹ԼÅÄ Sport Apps Senior Test Analyst

Whilst we have a lot of experience testing the Â鶹ԼÅÄ Sport app, we do not have much experience testing in VR. Testing a VR app from a third party, not only involves looking at the app but also ensuring the mobile and console hardware (cardboards, headsets etc..) each display the app correctly to the user and checking the stream and on demand video quality is acceptable and plays. During this experience many lessons were learnt from our testing as the VR app was delivered.

We had to base our testing on specifications, unlike working hand-in-hand with the development team like we are used to. After analysing the spec, we needed some clarification before we were happy to start our testing. We were also given a document with acceptance tests which were based on user journeys that we used for our testing.

The end-to-end test sessions lasted for approximately two hours and covered testing the stream, branding, video on demand, data and camera angles in the stadium.

In our first end-to-end session we had eight people independently testing the app across a selection of devices. People were going through the test scenarios at their own pace, some were filling out spreadsheets while others were shouting out the issues they found, which resulted in a disorganised round of testing.

As a result, we decided that we would go through each user journey together and with one person scribing the issues that were found.

From then on the test sessions ran more smoothly, however, we took part in a further six tests rather than two tests due to issue found along the way. This extra vigilance ensured we delivered a product of the highest quality.

Learnings

The game’s changed!

Two years ago for the 2016 Rio Olympics we first dipped our toe into immersive / VR technology. In the two years since, not only the headsets but the software / environments have improved dramatically. This is also reflected in the usage with over four times more people downloading the World Cup app compared to the Olympic app. The quality of the lounge, which is central to the app and how users navigate, is fantastic across all platforms and something our users have picked up on.

The camera angles available are great, but there could be more…

As the tournament progresses we have been conducting user testing and monitoring feedback. The users of the app love the multiple camera angles but some would like even more control. We currently have cameras set up in the stand and behind both goals but users would like to see this expanded along with the ability to have more control over the live feed.

Same app different users

The app has been a great success in terms of downloads and one area we have seen significant growth is in smartphone users. However these users have the lowest average session time. When looking at users who invested in headsets like the Oculus Go, PlayStation VR and Samsung Gear VR they spend far more time watching than smartphone users.

When thinking about VR in the future we need to take this into account e.g. should a smartphone VR experience be based around a second screen or highlights and headset users around live games?

Same app different users

People don’t want to be left out, ever

Whilst working with our partners we supported all devices available to us so we could learn as much as possible. Unfortunately not every VR device or headset was supported as this was very much a trail, and the findings taken from this will feed into the decisions we make around device support in the future.

What next…

At the moment the experiment continues until the World Cup finishes on July 15th. After that we will take the learnings from this already successful experiment and factor those into whatever might be next. But we know, there is certainly an appetite for VR and Sport to work together.

If you haven't had the change to make the most of the World Cup experience yet, you can download the app from the below locations: