Bias in BBC Question Time
Is there bias in BBC Question Time panel selection?
Update: This analysis was referenced in The Economist.
The BBC has been fighting back against accusations of bias in the selection of political guests. Such accusations are not new, but do they have merit?
In an attempt to defend the impartiality of the show the BBC Press Team recently published a graphic reflecting the political alignment of panellists.
Unfortunately the graphic is a bit scarce on the specifics of what the sources and the beeb have declined to elaborate on it or provide the underlying data.
Without the underlying data, it’s hard to know exactly what the graphic actually represents.
Someone tried to obtain information about guests on the show from the BBC via Freedom of Information Act in 2018 — as someone else had one previously in 2010 — only to be rebuffed by the organisation, which cited an exemption in the regulations they felt didn’t require them to provide information as it fell under the category of ‘journalism, art or literature’.
This was disputed by the requester as not being in the spirit of the provision for exemption in the act, but the BBC has stuck by it’s decision and refuses to release the information or to elborate on why they are unwilling to provide it.
List of Past BBC Question Time Panelists - a Freedom of Information request to British Broadcasting…
I am aware that The Freedom of Information Act does not apply to the BBC in the way it does to almost all other public…
Question Time guest appearances - a Freedom of Information request to British Broadcasting…
Please can you supply me with data on Question Time Guests. I would like to know the names of guests and how many times…
In September 2018, someone tried to start an effort to crowdsource gathering this data at bbcqtdata.com, with a corresponding Twitter handle and hashtag.
I reached out to help them achieve their goal, but was informed they were not seeking further technical assistance as they had that in hand.
They subsequently closed the project without publishing any data and deleted the Twitter account for the project.
So I decided to spend some time looking at it on Sunday evening.
I started by writing some software to parse the crowdsourced episode guide from Wikipedia and then using the Wikipedia API to extract structured data, such as what party a guest was a member of.
This went well so I spent another evening watching the show and cleaning it all up — both fixing issues with Wikipedia data that I saw (by updating pages) and on improving the script I’d written.
I then ran the script to get the numbers and put the data into Google Sheets to create a graphic.
The episode guide data goes back all the way to the shows inception since 1979. If we time slice it the same way (from 2000–2018) and only include data for the same number of parties we get an uncannily similar result to the chart released by the BBC Press Team.
It combines the Scottish and Welsh Labour and Conservative parties, does likewise for the Greens and incorporates appearances of members of the lesser known Labour and Cooperative party (which has high profile members such as Ed Balls and Kezia Dugdale) in the tally of Labour politicians.
There are several ways to slice the data and the results change around by a percentage point when you do, but the data correlates highly. Charting historical data going back to 1979 produces a similar overall picture.
What about the ~36% who are not politicians? Could political guest data be a red herring, masking bias in the selection of contrarian talking heads?
These have been looked at before too and the result turns out to be just as humdrum with no obvious political bias. You can crunch the numbers on the data, if so inclined, but it appears disappointingly benign and drama-free.
All sorts of claims can be asserted when assessing episodes in isolation or by slicing data until it fits a pre-determined conclusion, but they may be misleading and the claims not hold up when looked at in context.
Appearance data for guests and the above graphic can be found in CSV format and the software used to export the data has been published on GitHub.