Subtitle Quality Projects at IBC2013
Posted by Michael Armstrong on , last updated
is the biggest annual event for the publication and promotion of the work that we do at Βι¶ΉΤΌΕΔ R&D.
Back in December last year I ran a series of user tests for a piece of research on the relationship between the impact of errors and delay on the perceived quality of subtitles. Just before we ran the tests we made the decision to publish the results externally, and because the results would be of interest to many in the broadcast industry, I decided I would aim to present the results at IBC. My proposal for a paper was accepted back in March and at the same time we started to plan our participation in a Βι¶ΉΤΌΕΔ R&D stand. The full paper was finally accepted for publication in June.
My paper, "" was one of three papers in the session "". This was the first time a whole session at the IBC conference has been devoted entirely to the subject of access services, and showed a growing interest in the subject, particularly subtitles.
The work was comissioned by Nigel Megitt of Βι¶ΉΤΌΕΔ Technology, Distribution and Archives, Solution Design team who wanted guidance on the relative impact of changes in accuracy and delay on the audience's perception of the quality of subtitles. These results were to inform future investment decisions, helping ensure good value for money for the impact on the audience's experience of TV subtitles.
The paper describes the way that AV quality estimation techniques can be adapted to measure the quality of subtitles. It goes on to present the results from a test of live subtitles which investigated the relative impact of delay and accuracy on subtitle quality.
The results show that the impact of accuracy and delay depend on whether the person is watching with the TV sound on or off. When watching with the sound off, the main impact on perceived quality is from the level of errors, but there is also a significant impact from the subtitle delay. However, when watching with the sound on there is little impact on perceived change in quality with changing error rates but changing the delay on the subtitles has a strong impact on quality.
So for the large part of our subtitling audience who are hard-of hearing who watch TV with both sound and subtitles it is the delay which has the major impact. As a result of this work colleagues are investigating ways in which we may be able to reduce the delay on broadcast TV channels.
This work also prompted ideas about how subtitle quality could be improved for our online TV content, and we showed this in the two demos on our stand.
Photo: Matt Shotton looks on as Matt Brooks makes improvements to his demo.
The first, by , was a demonstration of how subtitles which have been produced on a live programme, running 4-8 seconds behind the speech, can be automatically realigned and reformatted to create two-line block subtitles which appear on the first word. It is a proof-of-concept designed to show how we may be able to improve subtitle quality for video delivered over IP some time after broadcast. You can read more about it in his white paper, "".
The second demo was an illustration of the results has been getting from experiments aiming to place subtitles in ways which are better for the viewer. The demonstration visualised the gaze-position data from a set of experiments comparing people's eye movements for subtitles placed close to the object of interest with ones placed in the normal position. These rather colourful visualisations were both informative and eye-catching.
Our stand attracted a great deal of interest. Our demo was included in the first "" session which, alongside the paper session created significant interest in our stand.
We were on our feet talking to visitors for almost the entire conference.
Our visit to IBC was incredibly successful. It was very hard work but well worth the effort. There was a huge level of interest and we gained many new contacts and several offers of help in our work from others in the industry.