麻豆约拍

A large number of teams within the 麻豆约拍 work on various aspects of of our online services. Responsibilities span the entire stack, quite literally everything from low-level networking all the way through to audience engagement. In this post, we’ll give you a brief overview of what some of our teams have been working on in 2019. Each contributing team has written an introduction to explain their role and responsibilities, along with some highlights of their work this year. If anything in particular catches your interest, please leave a comment and we’ll get some follow up deep-dive blog posts written to give more detail.

Programmes Team

The programmes team is based in , London, and manages , and , as well as tooling for ourselves and the editorial staff who publish to these platforms.

• We finished off our migration from the 麻豆约拍’s on-premises infrastructure to , including a complete rewrite of the codebase. This has resulted in render times being slashed down to 115ms on average.
• To increase the reliability of /programmes we've implemented a stale-while-revalidate cache using that compresses and stores all of our queries for at least 6 hours. This means that our application will be able to survive extended downtime from our data providers without a major interruption in service.
• We moved from a CI/CD environment we shared with a couple of other teams to our own environment fully under our control in AWS. Build times and our automation tests generally are faster without having to queue to be run.
• We added structured data to enable search engines to understand the complicated nature of /programmes content better, thus helping our end users find the right content faster through their favourite search engine.

Online Technology Group (OTG)

The Online Technology Group is responsible for the provision and management of the 麻豆约拍’s core online services such as networking, , , audience-facing , , , routing and facilities to content origins, our on-premise hosting platforms, and media analytics.

Website traffic management

• Launched our new, in-house website traffic management (TLS termination, layer 7 routing, caching, origin HA) service in two UK (our largest audience) as part of our worldwide rollout, alongside the existing mainland Europe and Ireland PoPs.
• Gradually enabling on our website traffic managers, as platform technology allows.
• Built “bq-logs”, a large-scale, low cost, auto-scaling, reliable, extensible system to receive, enrich and import our traffic management access logs (or any structured or semi-structured data) into Google BigQuery. This system requires no day-to-day management, currently processes around 1TB of logs every day and enables access to all 麻豆约拍 online teams to the data it contains, maintaining access control over personal data.
• Built a very simple, fast, low cost, auto-scaling and endpoint which feeds reports into bq-logs (above) for , and similar reports.
• Began deploying a consistent baseline of HTTP security headers, such as CSP, NEL, , ,
• Deployed records on bbc.co.uk, bbci.co.uk and bbc.com.
• Enabled on the bbc.co.uk and bbc.com apexes.

Media Distribution

• Along with our colleagues in R&D, Media Services and iPlayer, successfully delivered the FA Cup Finals and all of Wimbledon 2019 in glorious live streaming UHD, to compatible smart TVs.
• BIDI (our in-house media CDN) reached a milestone of serving 1 Terabit of video on-demand traffic per second in 2019. BIDI is now running from 78 servers embedded within 15 data centres, across 4 s.
• New BIDI cache hardware model designed and validated, which provides 119% more throughput, 316% increased programme storage capacity and is 88% more power efficient at peak throughput. All whilst providing an enhanced quality of service to the audience and at a lower cost per unit to the 麻豆约拍.
• Radix, our in-house media content origin platform has been refreshed. The new platform provides a better quality of service and increased capacity to meet the demands of iPlayer's on-demand catalogue for the next three years.
• Software improvements helped reduce drive write rate on cache servers by 75%, improving latency and increasing drive longevity.

Media Services

Media Services is the team responsible for audio and video workflows across the 麻豆约拍 Online. We and manage access to media, deliver video to linear playout systems, and develop the media players for Web and native iOS and Android. We've been making some subtle but important changes under the hood. Here are the highlights:

• We improved our workflow so that we can deliver audio described programmes to iPlayer before they are broadcast on . This meant that we could deliver the entire second series of Killing Eve with audio description, so visually impaired viewers could enjoy the whole series before broadcast like anybody else. One of our developers, Marcel, wrote a blog post with all the details here.
• We integrated a new transcoding service from Bitmovin into our workflow that dramatically speeds up the delivery of content to iPlayer. Last year, for example, fans of Strictly Come Dancing had to wait hours after broadcast for the show to appear on some platforms: it took over 2 hours to transcode each episode. This year, we reduced that time to just 10 minutes by splitting the programme into small chunks and processing them in parallel. This is just one of several transcoding services we use to optimise our media transcode function for different scenarios.
• "Content Enrichment" is a term that's being used a lot in planning meetings. If we can better understand our own content, we'll be able to provide better recommendations to the audience. One way we can do this is by creating transcripts of the audio and video we produce. Media Services has now launched a managed transcription service using 麻豆约拍 R&D's tool “麻豆约拍 Kaldi” that will make it easier for other 麻豆约拍 teams to transcribe their content at a fraction of the cost of other speech-to-text providers on the market. So far we’ve transcribed 75,000 hours of archive audio!
• The Playback team have been busy. The web team rebuilt the ‘continuous play’ experience so that programmes and clips will automatically play in sequence. Alongside that, they commenced a major programme to rebuild the web media player from scratch in order to take advantage of new technologies available in browsers, such as , and . This also enable us to leave behind older or legacy technologies like , and suchlike (noting that we will still support older browsers for as long as they are needed even as we move to the new player sometime next year). The new player will also sport the new design which we’re excited to roll out.
• The Mobile playback team have been equally busy; we onboarded several new apps over the last year and have been working with them to enable full use of the players. Again, much work ‘under the hood’ is ongoing to maintain an uninterrupted delivery of 麻豆约拍 content to those apps with a robust and reliable service.

iBL

iPlayer Business Layer is “The Beating Heart of iPlayer”. If content is the lifeblood of iPlayer, iBL is the team responsible for controlling its flow through the product. We achieve that by gathering the metadata, curation, business logic and algorithms in one place: we are iPlayer as an API.
Some of the highlights of what we worked on in 2019:

• Curator DSL - In order to help our editorial teams with their work, we have developed a for selecting a subset of episodes on iPlayer, which can be refreshed automatically rather than needing manual updating of groups. This is using existing commissioning information such as genre, but also analyses and labels episodes based on descriptions and other data - see also the content enrichment work Media Services is working on above.
• Personalised Bundles - On our journey for iPlayer to be Total TV, we have introduced more personalised “Bundles” of content on the iPlayer homepage, such as the “If you Liked” bundle, which suggests shows based on your recent watching history and communicates that reason to you.
• Content to Content Recommendations - We have taken some of the concepts 麻豆约拍 R&D have developed for their Recommender System into production and running some of iPlayer’s recommendations using this system. Stay tuned for more in this space in the future!
• Live Events - 2019 saw the return of after a fallow year, and the 麻豆约拍 is committed to show much more live coverage across the board, including sports. This year, we have revamped our live event workflows after some user testing: From dynamically updating the titles and images on the playback screen, and on the homepage, based on what’s actually happening on stage or tennis court right now to grouping content into sensible slices, we’ve made many improvements to iPlayer, and won 馃弳.
• Full Series - Following recent Ofcom regulation changes we are expending the number of series available on iPlayer significantly, and are improving the ways in which iPlayer handles this. We allow browsing by series now, use better imagery and promotional materials for a whole programme rather than individual episodes, and we keep track of users’ watching progress and current episode.
• Experiments - We have run dozens of driven experiments in the live product this year, many of which resulted in changes to the audience experience. From re-ordering categories for an U13 (under 13) audience to a more useful way we generate recommendations for new users who don’t have a watching history, we hope that these under-the-hood improvements have made some people’s lives easier.
• Scotland - Scotland got its own TV channel, and obviously this is featured on iPlayer.
• Programme Publication Workflow - With the increasing size of the iPlayer catalogue, we’re undergoing a migration to an event-driven workflow to update available episodes to ensure everybody gets the latest Episode of Ru Paul’s Drag Race UK as quickly as possible.
• Better client workflows - Our APIs uses Persisted Queries, which means that if any iPlayer client would like to update what they query from the API, they need to register their new query first, we need to review it, and then add it to our API. This used to be a manual process, which we have now automated, allowing clients to update their queries in minutes rather than hours or days.
• Operational Visibility - All the new services to deliver all the new features above have introduced some changes to our operational workflows. We have worked with the 麻豆约拍’s operations teams to allow them to handle most actionable events without needing to call out an engineer. Across the whole department and organisation we have also started to share a lot more background on incidents and how we can work together to learn from them.

iPlayer Web

The iPlayer Web team are responsible for building and maintaining the face of iPlayer on the web, across thousands of different types of device.

• Analytics & Data - With audiences at the heart of everything we do, we need to ensure we can capture data around user behaviour, so that we can fully understand how users use the product. We have therefore increased our focus on understanding user data and insight, with more advanced implementations of analytics.
• Childrens’ experience - iPlayer has lots of programmes but not all are suitable for our younger audiences! We have started creating a safeguarded and child-friendly experience in iPlayer Web, for those users who are signed in with an Under 13’s 麻豆约拍 Account. We have run several experiments on this experience, for example, simplifying the user interface to see if young people find it easier to discover content. There are many more improvements to come, but to see our progress so far please check out how to register for a child account
• End-of-playback experience - We have run over 10 experiments this year, around the end-of-playback experience. We’ve been trialling various changes to the user interface, to assist users in discovering new programmes that they’d like to watch.
• Live Events - As mentioned by iBL, we designed and launched a brand new live events experience on iPlayer, making it easier to discover the various live streams available for users to watch at various live events such as Glastonbury and Wimbledon.
• React Migration - We finished moving all of our custom client-side code to - the last things to move were the main iPlayer navigation and footer. This makes for a more consistent codebase, allowing quicker development and experimentation across the product.
• End-to-end test infrastructure - We started making improvements to our end-to-end acceptance tests and infrastructure, with an aim of making tests quicker, more stable and run more frequently. By upgrading our testing framework and running tests in parallel, we saw time to run a full regression of the playback experience more than half, making it even quicker to ship new features to the audience!

Technology Strategy & Architecture

Technology Strategy & Architecture (TS&A) is a collaborative group which brings together expertise from across 麻豆约拍 Design & Engineering. It’s the team that looks ahead and assesses what technology the 麻豆约拍 will need in five years' time, and maps out a plan of how to get there; and ensures that we have the right priorities in place so we can digitally transform the 麻豆约拍.

• The ran a successful conference and fair in October with a focus on responsible use of . 300+ internal and external guests took part and were offered a range of discussions on developments, creativity, regulation and ethics and invited to explore their hands-on technology fair. A summary of the conference, and an exhibition walk-through can be found .
• facilitated a pan-麻豆约拍 initiative to create the 麻豆约拍’s first Machine Learning Ethics Guidelines - specifically for the 麻豆约拍 Recommender Engine project but with the expectation that they’ll be rolled out further to other machine learning initiatives.
• Our substantial work on developing personalised recommender systems has started to take effect with the first in-house recommenders being deployed. You can read more here.
• Following publication last year of a on the 麻豆约拍 Capability Framework, developed by TS&A, the EBU asked to share the framework to support the creation of an -owned model which could be adopted by its members. This July, the EBU technical committee formally approved the work to create an EBU media industry capability model. Read our follow-up blogpost .
• The 麻豆约拍 Machine Learning Fireside Chats are now available as a podcast on . The ML Fireside Chats are friendly bi-monthly events to discuss and share knowledge in the ML space. To find out more, join our . To join the conversation follow us on Twitter:

RMS (Sounds)

RMS is the service that powers the 麻豆约拍 Sounds experience across Mobile and Web. Our API provides a view into the content, and guides the experience that clients receive when using Sounds. Some of the work we’ve done (or are working on) this year includes:

• Local Radio Stations in the Listen Live Rail - To help make Sounds more personalised and relevant to the audience and increase user engagement, we started displaying a Local Radio suggestion to a user. If a user has a postcode set and has enabled personalisation, we will find the nearest Local Radio Station to that user’s postcode and display it at the end of the Listen Live Rail, along with the National Radio Stations.
• Shared, distributed cache to support Personalised Content - We’re investigating caching at different layers, and sharing the cache with other instances of the same service. In order to support more personalised content, we have started a long term investigation into new caching strategies. RMS serves hundreds of thousands of requests per day, and historically, most of those were for static or infrequently changing content that every user could consume. With the move towards more personalised content, we will have to change our caching strategy, as each user will receive a different experience. The variance of pages that could be cached grows massively with each piece of personalised content, and it is not practical to cache this in the way we’ve been doing it before.

Research & Development

R&D activity at the 麻豆约拍 has been going on for almost 100 years. A specific department was established when the 麻豆约拍 Research Department was formed in 1930. Doing research and development activity is Clause 65 of the 麻豆约拍 Charter. In 2011 the 麻豆约拍 Executive Committee’s paper interpreted Clause 65 to mean the following for the corporation’s research and development activity:

• Drive Innovation in support of 麻豆约拍 strategy and public purposes
• Horizon scan, acting as early warning radar in the face of technical change
• Provide an expert team to solve problems or maximise opportunities
• Add tangible value to the wider media industry
• Maintain 麻豆约拍 partnerships, and with them, primary position in the wider ecosystem

Our 2019 highlights include:

• 麻豆约拍 Click used our Object-Based Media tool Storyformer to make their .
• We introduced our personal data store prototype 麻豆约拍 Box designed to give audiences control over their personal information.
• Casualty audiences benefited from our new trial on Taster which made it easier for people with a hearing impairment to enjoy 麻豆约拍 dramas the feature allowed audiences to set the audio mix to suit their needs.
• R&D tested the use of 4G/5G broadcast technology to deliver live radio services to members of the public in . The island was chosen because it has limited fixed broadband, little or no mobile coverage and poor digital radio coverage.
• Connected Studio, part of R&D launched Makerbox the destination for developers, producers and storytellers to use cutting edge 麻豆约拍 R&D and third party originated tools to help them create new ways of telling stories to audiences.
• We continued our High Dynamic Range trials and increased the jeopardy by tackling the 2019 .
• The 麻豆约拍 Proms was used to test Notes our new web app that sends you extra information about what you’re seeing and hearing to your device, as it happens.
• We launched our Diversity & Inclusion policy on what speaker opportunities we will be involved with to help to improve diversity in the tech industry.

News and World Service (Web)

麻豆约拍 News and World Service online bring 麻豆约拍 news content to the web both for our domestic UK audience and for our international audiences, around the world.

This year our main focus has been on moving large parts of our website to the cloud and to our new technology stack (Including the ~40 world service language sites). As well as delivering coverage of the December general election, and improving our existing web products.

Specifics:

• Released a new that is “performance-first” to ensure that our users around the world are getting a first-class experience that caters for different needs
• Continued to migrate from our legacy monolithic applications to a more approach (mostly made up of and microservices on AWS).
• Added support for personalised content on the News front page with top stories tailored to each user's preferred nation.
• Developed a new -based “preflight” service that is responsible for the resiliency and availability of our web platforms.
• Upgrading our existing Elections infrastructure and creating new components for general election coverage in the UK, India, Nigeria, Thailand, Indonesia, and Sri Lanka amongst others.
• Using ECS clusters to render images with R and publish directly to Twitter and other platforms.
• Used natural language selection templates to create semi-automated editorial content in English and Welsh for the general election.
• Moving our iframe services to .
• Working with our data science teams to deliver an API to provide content recommendations based on users recent article reading history.

Weather website

The 麻豆约拍 Weather website serves an average of 8 million users per week, offering 14-days of hourly forecast information, warnings, interactive maps, and the latest editorial analysis and videos from the 麻豆约拍 Weather Centre.

In 2018, we relaunched a significant portion of the 麻豆约拍 Weather website on the cloud, expanding our use of to transform and render incoming weather data. Several features, however, remained on the 麻豆约拍’s outgoing on-premise platform. Our primary aim for 2019, therefore, was to complete our migration to the cloud, and in the process deliver a visually consistent and performant experience across the entire site.

Highlights include:

• Automated UK Weather Warnings and Flood Warnings pages.
• A refreshed Monthly Outlook.
• Migration of our article, video and photo galleries to the cloud, here’s an example article.
• Incorporating the 麻豆约拍’s new typeface, Reith, in line with our new TV graphics.
• Showcasing Weather Watchers photos (user-generated content) on our UK forecast pages
• New and improved Tide Tables information

In early 2020, we’ll release a new look for the remaining section of the site: our specialised forecasts covering Coast and Sea.

Search

The Search team runs the back-end services that power most of the search features on 麻豆约拍 Online including the website, mobile, TV and voice applications. We’ve been busy completing our migration from the on-premise “Forge” platform to AWS, enabling improvements to our search features going forward.

• Migrated search engines, services and data feeds to AWS
• Created a content enrichment pipeline and associated editorial tooling on AWS
• Started migration of search results pages to AWS
• Invested in Relevancy Engineering skills

Navigation

The Navigation service provides the 麻豆约拍 blocks, navigation links and footer on the website, as well as delivering the cookies/GDPR banners, analytics integration, the 麻豆约拍 Reith font, etc.

• Supported migration to the new 麻豆约拍 website analytics system
• Created a new EU cookie banner to allow our audience to opt in or out of the various cookies we use in 麻豆约拍 Online
• Supported insertion of banner in 麻豆约拍 CoreServices/Webcore products
• Improved click-per-view tracking support, enabling a feedback loop for improvements to Search etc
• Created a replacement service

Application Security (AppSec)

• Conducted a plethora of security architecture surgeries and in collaboration with 麻豆约拍 Online teams as they develop or change their services
• Re-launched the Security Champions forum in January 2019 for our online teams
• Introduced an initial training pathway for 麻豆约拍 Security Champions where new members have to take secure coding training, threat modelling training and data protection training in order to qualify as a formal 麻豆约拍 Security Champion
• Ran a pilot of a computer based training secure coding training tool and migrated to that from the previous classroom based course. This was successful so the platform was opened up to the entire 麻豆约拍 Security Champion community as a part of their training pathway
• Launched a newly developed threat modelling course and have trained 120 麻豆约拍 Security Champions across the year
• Ran a testing event against 麻豆约拍 iPlayer and 麻豆约拍 Sounds in order to identify vulnerabilities in their codebase and to help productionise our red teaming capability
• Hosted an at 麻豆约拍
• Launched a proof of concept deployment of , a vulnerability consolidation tool for 麻豆约拍 Online teams and InfoSec
• Launched a proof of concept deployment of , fronted by , for the 麻豆约拍 Security Champions to test their knowledge and skills in a safe environment.

So there you have it, a snapshot of what some of the teams in 麻豆约拍 Online have been working on in 2019. To the best of my knowledge, this is the first time we’ve published a roundup and whilst not all our teams were able to contribute this year, I hope we’ll continue this each year and improve as we go.

As I mentioned earlier, if there’s something which particularly interests you or a question you’d like to ask, please leave a comment and we’ll do our best to cover them in follow-up blog posts.

Thank you to all the 麻豆约拍 Online teams who contributed. All that’s left to say is that we hope you had a great 2019 and wish you all the best for 2020!