CSI User Experience Conference 2012 Part 2 – Live subtitles & measuring quality

CSI User Experience Conference 2012: TV Accessibility

CSI User Experience Conference 2012: TV Accessibility

For background info on this conference read Part 1.

Ofcom gave a presentation highlighting their access priorities for the next year which are:

1) To improve the quality of subtitling. This is particularly of interest for live TV subtitling (usually done via respeaking) and the subtitling of TV programmes completed very close to its transmission date thereby not allowing time for prepared subtitles to be created for the viewer to watch and so the subtitles are created by the same process as live TV.

2) Improve accessibility to EPG’s (Electronic Programme Guide) and functionality for the visually impaired.

For the purpose of this blog I will be focusing on points made about subtitling. During a roundtable discussion last month hearing impaired viewers gave the following feedback on live subtitles:

Watching TV with subtitles is much less relaxing than those of us who don’t need to use subtitles. You need to concentrate while reading subtitles, that means less time to look at the picture itself. It’s particularly tough if you’re watching with live subtitling you are having to relate the dialogue you are seeing against the picture that no longer relates to that subtitling. It means that it can be really quite disjointed as a viewing experience. And of course inevitably there are errors and editing which can make story lines difficult to follow. And from time to time, there are more substantial failures of subtitling. Very occasionally, you get misleading information which viewers are not made aware of if they are using subtitles. They maybe be told the statistic is 15% when its actually 50%. Easy enough to happen but it means that part of the viewing population is being misled.

As a result of such feedback Ofcom set the following objectives:

    Ensure they have an up to date understanding on how live subtitling is produced via respeaking and voice recognition software. (You can read some information about this on my #subtitlefail! page. And if there is more up to date info needed please let me know.)
    To understand how broadcasters manage the production of the programmes that are pre-recorded but delivered close to the transmission date to see what they are doing to ensure that they can provide the best quality subtitles for these programmes, not just for first transmission but also for repeats on linear TV within the first 7 days of first transmissions.
    To understand the technical problems that can sometimes occur throughout the transmission chain and not just at the end of the chain on a viewers TV that cause subtitling errors.
    To look a what measures of subtitling quality there are and if this can be broadened:

    We decided that we wanted to have a look at what measures of subtitling quality there were at present, whether there was cope to broaden and develop these, and whether there might be merit in asking broadcasters to report against a broadened measure in order to provide transparency about the quality of the subtitling they were producing. One of the reasons we thought this might be helpful, is that we have got a long experience of publishing statistics on the quantity of subtitling, that mean that you can go on to a website and see what a channel has been doing against this target over the last year or 6 months. That’s helped to drive quite substantial amount of over compliance. If you do visit our website you will find that most broadcasters are doing rather more tan they have to it may be that transparency and fresh air could also be employed to help broadcasters focus on how to improve the quality of subtitling.

    On this note I’d point you to an interview with Alex Varley CEO of Media Access Australia who I think has done some excellent work in this area. Also worth a watch is his presentation titled: Does measuring subtitle quality really make a difference? I find it interesting that tveeder was set up to allow Australian viewers to provide feedback on the quality of live subtitling of the news. By streaming the captions online anyone can review the subtitles verbatim and assess the error rate. Can something similar be done for other channels? However I am not sure how feedback on quality is gathered and collected using this tool.

Other feedback that Ofcom received on subtitling:

    Delays are really irksome to viewers because of that difficulty of marrying up the dialogue and picture. It’s even more irksome to them, than errors.
    Scrolling subtitling are much more difficult to read and comprehend quickly as it requires you to spend a lot more time reading and see much less of the picture.
    A loss of subtitling part way through a programme that a viewer is watching that with family results in said viewer suddenly being out of the picture is really frustrating. (This is a classic example of isolation and reminds me of the CCAC’s campaign video Don’t Leave Me Out)
    When a viewer knows that a programme has been pre- recorded but they have have not made it available for broadcast in time for that to be delivered with pre-recorded subtitles. When popular programming comes to broadcaster so late that they have to be subtitled live it often leads to a consequential degradation of the viewing experience for the hearing impaired person.

In Part 3 I will discuss the technology surrounding live subtitling and include comments made in a panel discussion around this subject.