Category: Ratings

What BARB’s error reveals about the bizarre world of TV ratings

UPDATE: Jack Knight has written a fantastic comment on this post, giving a lot more background to BARB and sampled ratings in general. I highly recommend reading the comments after the post. If you’re interested in the history of audience ratings, I highly recommend reading ‘Rating The Audience: The Business of Media’.

 

BARB – the organisation that measures ratings for UK TV channels – has admitted that there were errors in its tracking system, and as a result some Channel 4 and ITV shows have ended up with false ratings. Broadcast Magazine’s article says that one of the programmes given false ratings was ITV’s X Factor – their highest rating show, and one of the biggest advertising targets in broadcast television:

“The entertainment show originally recorded an overnight audience of 8.96m (33.6%) on ITV1 and ITV1 HD in figures released last Monday, but this has now grown to 9.84m (36.91%) under the revised data. Meanwhile, C4 shows including 999: What’s Your Emergency and Grand Designs have also experienced audience uplifts. However, others have fallen, such as the 22 September episode of The Comedy World Cup, which dropped from 1.8m (7.9%) to 631k (2.76%) on the back of the gaffe.”

How can mistakes like this happen in a multi-billion pound industry reliant on accurate audience metrics? Looking for an answer to that question opens up lots more questions about why such a huge and influential industry relies on relatively crude measuring techniques that haven’t changed much in decades.

TV ratings are measured using mechanical devices that record the presence of viewers in the room when the TV is on, usually by the viewers pressing a button to register that they’ve entered the room. So it really registers presence, rather than attention – the viewer could be reading a newspaper, doing the ironing or using their iphone, but for the sake of the ratings they count as an avid viewer.

Ratings technologies have been refined over time, but the basic concept hasn’t changed since it was invented by Arthur C Nielsen to measure radio audiences in the 1930s. BARB is the UK version of TV ratings, using a panel of 5,100 homes to represent the UK TV viewing public.  So each percentage point in the examples above stand for a measurement sample of just 51 homes. The amount of people in these homes is around 11,300, so each percentage point stands for a maximum of 113 people pressing their buttons when they walk into the living room. It’s often a lot less, as the percentages above are share of the total viewing audience (BARB calls this the ‘universe’) at that time – many BARB panellists might be out of their homes, or might not have the TV on at that time.

If we take the numbers of viewers in the sample above, we can work out the size of the TV viewing universe watching when these errors occured. For example, the 8.9m audience originally reported for X Factor was 33.6% of total viewers that night, so one percentage of that audience is 8.9m/33.6% – 264,880 viewers. This means that the BARB’s estimate for the total UK TV viewing audience on a Saturday night is around 26.4m people, which is 39% of the UK population of 62m people. So we could transfer this to roughly work out that the number of BARB Panellists registering themselves as viewers that night is 39% of 11,300 – 4,407 people.

Still with me? Lets now take the share of X Factor’s reported viewing to work out how many BARB panellists registered themselves as watching that programme. The original share reported was 33.6% of total viewers. We know the total BARB panellists watching TV was 4,407, so the number watching X Factor according to the original report was 33.6% of 4,407 – 1,481 people. So BARB measures 1,481 people watching a TV programme, and extrapolates that number to report an audience rating of 8.96m viewers. No matter how scientific and representational the survey, is remarkable to think that multi-billion pound creative decisions are made on such a small sample size.

Now lets look at the error size. BARB under-represented X Factor’s ratings by 3.31 percentage points, which was a difference of 880,000 viewers in the reported ratings. Again, if we take the total panellists viewing X Factor that night as 1,481 people, 3.31% is 49 people.

An error in measuring 49 people pressing a button when they walk into a room means that one of the UK’s largest media businesses under-represented the performance of their most important programme by 880,000 viewers. Is it just me, or is that completely insane?

Is Youtube a TV Channel?

In an interview with Broadcast Magazine last week, Simon Cowell suggested that Youtube could soon be considered competition for traditional TV Channels:

“There’ll be a point in the not-too-distant future when we’ll be able to watch TV and YouTube will be Channel 6. When we reach that point, they’re going to be serious competition.”

The comment comes just after the X Factor channel on Youtube joined Syco’s Britain’s Got Talent to become one of the few UK channels to top 1 billion views, closely followed by the BBC. Taking into account Youtube’s 2011 redesign to focus navigation around channels and the launch of its Original Content channels in the US and Europe, is Cowell right? Is Youtube becoming more and more like a to a ‘traditional’ TV Channel?

What is a TV Channel?
To answer the question, we need to remind ourselves what we mean by a TV Channel. Nowadays, channels are navigational elements – brands that convey to the audience a set of values about their programming schedule. But originally, channels and schedules were a solution to a specific problem – when you could broadcast content over a network all day (rather than the limited duration of a theatre or opera programme), what kind of structure would help audiences know when your content was available?

The first people to have this problem were the early ‘telephone newspapers’ created in the late 19th Century. They thought that telephones would be used to broadcast, not just for person-to-person conversation, and invited users to subscribe to content available on special one-way telephones installed in their homes.

The Telefon Hirmondo in Budapest was one of the most successful telephone newspapers, with around 15,000 subscribers at its peak. They solved the problem of how to organise content for their 12hr daily broadcasts by creating ‘issues’ – what we would now call ‘schedules’, carving the day up into chunks of hours or part hours. Here’s a sample of a Telefon Hirmondo ‘issue’:

2:30 PM 3:00 PM Parliamentary and local news.
3:00 PM 3:15 PM Latest exchange reports.
3:15 PM 4:00 PM Weather, parliamentary, legal, theatrical, fashion and sporting news.
4:00 PM 4:30 PM Latest exchange reports and general news.
4:30 PM 6:30 PM Regimental bands.
7:00 PM 8:15 PM Opera.

As the broadcast technologies of radio and then television emerged, they adopted the structure of these ‘issues’, and over a century later, we’re still organising broadcast content in pretty much the same way. The complex art of organising content for optimal viewing – the art of ‘scheduling’ – became one of the critical skills in broadcasting, defining the success of one channel over another, and therefore the price of advertising on that channel. A traditional TV channel is, in essence, its schedule.

What is a ‘channel’ on Youtube?
Youtube has traditionally been seen as a platform, not a channel. Rather than an editorialised schedule of content, it’s an open, searchable platform, allowing users to upload as much or as little content as they want, and for audiences to view content on those same terms. As the platform grew over the last 10 years, video views emerged as the most common metric of attention, with success seen purely in terms of the highest number of views. Reaching 1bn views is a significant milestone, one that only 50 channels have currently achieved.

But since the redesign, Youtube have pushed for subscribers to be the core metric, and for creators to focus on channels rather than individual videos. But Youtube videos are shared and circulated in lots of different ways, and the patterns of attention around videos are way more complex than broadcast viewing. Channel subscriptions are not yet the most popular way to find content on Youtube, with most viewing sessions starting with organic search – Youtube is the second biggest search engine on the internet. Youtube’s channel strategy is an attempt to change this, and to try to encourage more loyalty and ‘channel-like’ behaviour in its audience. The aim is to get longer viewing sessions, and to raise the channel brands on Youtube above individual videos, making it more suitable for the kind of viewing patterns expected on smart TVs in the living room. But this transition to a channel strategy is still in its early days.

What makes a successful Youtube channel?
As Youtube makes the transition from videos to channels, its worth comparing the list of most viewed videos to the list of the top 50 most subscribed channels on Youtube. The top of the most viewed list is dominated by mainstream content brands and talent-branded VEVO channels. Here’s the top 10 channels by video views:

Universalmusicgroup 6,911,136,702
Machinima 3,668,308,520
JustinBieberVEVO 2,860,897,761
RihannaVEVO 2,756,231,298
expertvillage 2,568,074,626
LadyGagaVEVO 2,270,263,760
AtlanticVideos 2,050,668,530
EminemVEVO 2,030,339,362
RayWilliamJohnson 1,984,191,369
IGNentertainment 1,935,914,288

Whereas the most subscribed is dominated by Youtube-native talent channels – Rihanna is the only VEVO channel to make the top 30 most subscribed. Here’s the top 10 most subscribed channels:

RayWilliamJohnson 5,900,000
nigahiga 5,700,000
smosh 5,400,000
Machinima 4,800,000
collegehumour 2,900,000
realannoyingorange 2,500,000
BlueXephos 2,400,000
thelonelyisland 2,400,000
RihannaVEVO 2,360,000

This illustrates the different strategies being using to make successful Youtube content. For some, the brand or talent is established enough outside of Youtube to drive views through organic search alone – this explains the number of VEVO channels in the top 50 video views list. Channels without other sources of traffic have to work harder to get attention to their content – this is why the top 10 most subscribed channels are Youtube-native comedy and games channels.

What’s interesting is when you look at the amount of work people put in to get subscribers. If we divide the number of subscribers by the number of videos uploaded to the channel, we get an unscientific, but interesting statistic – the average number of subscribers added per video. Here’s the channels that have done the least work to get their subscribers:

LMFAOVEVO 58,621
nigahiga 45,238
RihannaVEVO 32,329
KatyPerryVEVO 30,508
thelonelyisland 28,916
Universalmusicgroup 26,324
JustinBieberVEVO 25,085
JenniferLopezVEVO 24,344
PitbullVEVO 23,404
smosh 21,429
ChrisBrownVEVO 20,021

And here’s the channels that have worked the hardest, sometimes only adding a handful of subscribers per video upload:

rajshri 2
Associated Press 4
CBS 6
expertvillage 8
muyap 15
IGNentertainment 24
BBC 42
clevverTV 59
Spinninrec 153
Machinima 224
barelypolitical 261

Its interesting how many ‘traditional’ broadcasters are in that second list – AP, CBS and the BBC. And just out of this list, The Ellen Show comes in at number 12, TheXFactorUK at number 14, and BritainsGotTalent09 at number 16.

There are so many competing strategies on Youtube right now that comparisons like this are not hugely revealing, but there do seem to be three kinds of channels emerging:

Talent-led channels – broadly music based, views driven by organic search, very few uploads
Broadcast-led channels – linked to existing TV shows/channels, lots of uploads, (mainly clips), views largely driven by organic search, but few subscribers
Youtube-native channels – lots of subscribers, lots of uploads, most traffic driven by links within the Youtube platform

So, will Youtube become a TV Channel?
It’s early days in Youtube’s channel strategy, but at the moment, its hard to see the different strategies that talent, broadcast brands and native Youtube creators are using merging into something as coherent and consistent as a traditional TV channel brand. When Simon Cowell looks at Youtube and recognises it as a ‘TV Channel’, he’s seeing it from the perspective of someone who has been immersed in broadcast TV for years, and is more familiar with those patterns of attention than some of the new patterns emerging from native Youtube talent. To a man with a hammer, everything looks like a nail, so a man with some of the biggest broadcast TV brands will look at Youtube, which sometimes look like traditional TV, and assume that it will eventually become like the things he has spent his life building.

I think its more likely that TV channels will become a bit more like Youtube. If Smart TVs and other VOD boxes take off, we’ll start to see some new user journeys around content on our TV – organic search, subscriptions to channels/shows we love, social and algorithmic recommendations, etc. This will change the way that schedulers think about TV channels as much as the rise of multichannel satellite and cable did in the 1990s. Channel 4′s 4/7 channel on Wikipedia – scheduled partly in response to online buzz about Channel 4 shows – is an early indicator of this trend. Simon Cowell might be looking at the right thing, but from the wrong perspective – its not about Youtube becoming more like TV Channels, but about TV Channels becoming more like Youtube.