By Oscar Schafer, R&D Engineer
A great tit and a blue tit detected in the New Forest
Over the past few years, there has been an ongoing partnership between The Watches and Βι¶ΉΤΌΕΔ Research & Development (R&D), focusing on “Intelligent Production Tools” and getting even more value from the wildlife cameras. In an effort to bring viewers the very best of British wildlife, an increasingly important aspect of making a nature show is perhaps surprisingly… technology.
Abundance of sources
From the River Ness to the New Forest, we have seen the largest network of cameras yet contributing to the online streams and the live shows this past year. However, with an increasing number of cameras, there’s an added pressure on the camera operators and the creative team not to miss those “golden nuggets” that make the Watches so special. This is where artificial intelligence (AI) and machine learning come into the picture.
Starlings detected on the pier in Aberystwyth
Using a combination of object detection and classification algorithms, tailored specifically to British wildlife, software is able to assist the production team in capturing animals on camera. We witnessed it in action on Autumnwatch (below) when the software automatically classified different birds visiting the feeder, but the software is also able to detect both mammals and fish. The exact algorithm and techniques deployed to monitor a specific camera stream often depend on the location. This is still an area of active research for the Βι¶ΉΤΌΕΔ R&D team. In some cases smaller or further away objects can trick the automated system, causing it to mistakenly identify animals.
Nocturnal
With several of the most loved creatures on the Watches being nocturnal, one area where this AI technology has been deployed during Winterwatch is in monitoring of nighttime activity. Much of the action caught on camera is fleeting and easily missed, with crew monitoring up to 20 camera sources at once. Working with the infrared camera feeds, the AI system is able to detect and classify animals shown on the streams. When detecting an animal in shot, the system starts recording - delivering a set of clips available to the production team for editorial review. These clips can then be used on digital productions or in the live TV shows.
Something else that the use of technology generates is a rich set of metadata. While in operation, the analysis software delivers a generous source of information, which in turn can be used for further analysis or as part of experiments run by the production team.
A fox detected at night in the New Forest
What next
Of course, a big part of what can make a video clip interesting is when it features something unusual or rare in nature - whether that be a bird’s nest being predated in the New Forest, or an otter appearing on the banks of the River Ness. Despite the fast emergence of technology, the best domain understanding still lies with the human experts. Work is underway to improve the metadata generated, adding more information to further assist the teams working on the show.
From an audience perspective, the use of intelligent production tools such as the AI system in place during Winterwatch also opens the door to new and richer experiences.
Read more about the technology behind the AI system on the R&D blog.