Misunderstanding statistics the BBC way

We’re screwed.

The battle between the continual advance of technology and mankind’s equally continual ability to misunderstand data continues.

But the latter edges ahead in the domination table.

We fail to appreciate the complexity of questions we ask.

We fail to comprehend the responses.

Yet we continue to construct flawed questionnaires.

And misrepresent the findings.

This story on the BBC website emphasises just how doomed to a world of misunderstanding we are.

The headline is:
Many Asians do not feel British.

Straight away my brain wants to know whether these Asians were born in Britain or not (i.e. it is possible to come from another country and live in the UK without giving up one’s nationality).

So that’s a black mark to the BBC straight away – for a badly constructed tabloid headline.

As we read the story we are told ‘38% of UK residents of south Asian origin felt only slightly or not at all British.’

So now the story has narrowed, has transformed to become ‘UK residents of south Asian origin’ and not the wider meaning of the description: ‘Asians’?

Surely this a different meaning entirely?

A second black mark to the BBC then – this time for misrepresenting their poll.

But as I ponder this misrepresentation I can’t help wondering how many ‘UK residents of English origin’ would term themselves British?

Or Welsh origin?

Or Scottish origin?

Or Irish?

But the BBC survey doesn’t give us any comparative data.

So that’s another (third) black mark to the BBC – failing to provide comparative data.

Beginning to feel wary of the BBC’s reporting of what is, after all, their own survey (and against my better judgement) I click on the link and download the survey results.

The survey questioned 500 people of (south) Asian background (but the survey authors don’t tell us what the specific criteria is that determines being ‘of (south) Asian background’).

Or ‘of Asian background’ in general.

That’s another (fourth) black mark to the BBC – lack of transparency of the sampling criteria.

Anyway.

The survey.

Don’t click on the link in the news item.

Don’t do a statistical analysis of the BBC’s own survey data.

No really.

Don’t.

Because you’ll find that the BBC’s own survey data is wrong.

I could give you examples (I was nerdy enough to break down the percentage of responses and number of respondents to Table 7 then sum the returns to try to get back to the quoted sample of 500. I bet you a large Pimms that if you did that you wouldn’t get back to the 500 that you should!).

No, seriously.

Don’t.

Because it proves the survey is wrong.

And while you’re not recompiling the survey responses please don’t ask yourself what about the other British subjects of other Asian nationalities (e.g. there are more British subjects of Chinese extraction than of Indian, Pakistani and Bangladeshi combined.  And what about our cousins from Malaysia?).

So then a quick flick forward to Table 8:
Q.8 Generally speaking, to what extent do you think that white British people treat you as British?

Hmm€¦

But the BBC news item reported the wording to this question as:
‘… nearly half believed white people do not treat them as British.’

So whereas the question asked ‘white British’, the BBC have reported it as ‘white’.

So some distortion there.

Another black mark to the BBC.

How many is that?

I can’t remember.

I’m starting to lose the will to live.

Crack on.

Getting tired of the BBC’s manipulation I fast forward to Table 11:

Table 11
Q.11 Do you think, as an Asian, that you have more opportunities, the same number of opportunities, or fewer opportunities in the UK compared to the opportunities you would have in South Asia?

What does ‘opportunities’ mean?

Opportunities to do what?

Own a BMW?

Get divorced?

Stand for election?

Opportunities for women to become a member of the clergy?

Is this a cultural question?

A financial question?

A political question?

Or – best of all – an equal opportunities question?

For surely, these questions and their responses must be weighted to males, females, gays, lesbians, able-bodied and disabled to have a tangible value?

Yes?

Sigh.

Look, I know that the BBC is our national broadcaster but there’s something very wrong here.

That the BBC commissioned this survey is one thing.

Laudable perhaps, but flawed because the parameters can not have been adequately set at the point of specification.

That the survey output is so badly flawed it is risible, this is another thing entirely.

That no-one in the BBC has comprehended the magnitude of the wrongness of the survey is inexcusable.

That the BBC news department has published this survey is just plain stupidly wrong.

Yes, I know I’m probably the man who put ‘anal’ into ‘analytical’ but come on.

As a piece of statistical work this really isn’t good enough.

Is it?

Look!

I’ve found someone else who feels similarly.

Chloé says:

I’d like these kids to realize that maths and statistics (and science in general) are not only about calculations, but also about properly asking the right question, cleanly checking that conditions are verified before applying formulas, and drawing conclusions from the numerical results they obtain.

Which is the reason why they are asked about analysis, interpretations and other horrible things that require to actually write this highly suspicious things called, oh, yes, I remember now, words.

And by that I don’t mean tossing words together in a grammatically dubious sentence, hoping that nobody will notice that the beginning contradicts the end – and both of them contradict the middle -, nor that they don’t have the faintest idea what the difference between hypothesis, condition, proof, example and conclusion is.

See?

It’s not just me!

B.