Automating a Springwatch bird quiz using machine learning techniques

Our Intelligent Video Production Tools team has been working with the Springwatch, Autumnwatch and Winterwatch programme, using machine learning techniques to help the producers find interesting video clips from wildlife camera feeds.

One result of this work is a database of video clips from both Autumnwatch and Winterwatch 2020; this contains the video and a host of metadata, including the outputs from the machine learning. This database is hosted on the cloud, meaning that we can write computer programs to access it and find content.

However, this content is a valuable resource for creating prototype interactive experiences for us in the Object-Based Media Toolkit team. For example, an clip in the database might also contain data about when it was collected, what type of bird the algorithms think it is, and the confidence with which that prediction was made. We can use this information to explore ways of building Object-Based Media (OBM) experiences automatically.

We are working with the team on creating prototype news-based experiences and have developed a library of python code that could automatically create simple OBM experiences based on story tags. After looking through our Autumnwatch database, it became clear that the most usable content came from bird feeders and that the most useful items were short clips where the algorithm identified a single bird.

By taking a set of clips from several different species, we could quickly create an OBM experience where the viewer could choose which species to watch on the feeder. There were a few problems where the birds are not identified correctly, or more than one was in shot, but it demonstrated the value of automation very convincingly.

The next step was to select eight clips from each of the four species, 32 in total, that turned up most often at the feeders and automatically turn it into a quiz. Instead of selecting which species to view, you are shown a clip, and you choose the name of the species to which you think the bird belongs. If you get the answer right, the clip is repeated, whilst if you get it wrong, you see a clip of the bird you selected. We also added the ability to track your score.

We shared the quiz with the programme production team, who were enthusiastic about the concept and agreed we could publish a quiz based on their Autumnwatch content. We also discussed several ideas for quizzes that would work with Springwatch content where the main focus of interest for birds is the nests rather than feeders.

To make things more challenging, we鈥檝e taken the automatically generated quiz and imported it into our StoryFormer production tool, along with the video clips. We then added clips of less common birds to provide eight different species to identify and a more varied set of multiple-choice answers. We have also improved the presentation, editing the clips to remove long sections with an empty feeder and added background audio to tie the clips together. We鈥檝e also included the option to share your score on social media at the end.

We are looking forward to the next Springwatch and seeing what the cameras can capture. We hope to produce more quizzes during the series and discover more about the challenges of automated content creation. We also want to explore other types of interactive content that can be built from the different camera feeds.

麻豆约拍

Accessibility links

Michael Armstrong (BSc Eng)

Rebuild Page

Useful links

Theme toggler

麻豆约拍