Saturday, January 03, 2009

Lying with (other people's) numbers

Surveys and their illegitimate cousins show up distorted in the press for a lot of reasons. One is game-frame bias: If something looks like a contest, it's told like one, even if the data don't support all the clawing, eking, inching and scrapping in the hed. Another is confirmation bias: If something purports to show that the world is either on or off track (church makes you healthy, Those Kids are Destroying the Language, whatever), it gets emphasized. And there's lab-coat bias, or the tendency to assign extra credibility to whoever is standing in front of the whiteboard and sounding scientific.*

Then there's out-and-out lying, or ignoring the data and inventing conclusions to support your ideological or political stance. That can be hard to tell from confirmation bias, but the little cousins at Fox are sometimes pretty blatant about it, as they are here.

For the record, no. We have no idea whether "troops wary about Obama" is true, because this survey doesn't address the issue. Generalizations about what "more than half of U.S. service members think" are fabricated. As it has done before with this particular instrument, Fox is lying because it has a chance to sow doubt about its political enemies:

Six out of 10 Americans in uniform are concerned about Obama's lack of military experience and their perceived difference in his mission and values from that of Bush.

Military Times is a little less blunt this time about the limitations of its survey, but the information is there if you want it. This is a self-selecting poll, meaning the results can't be generalized to the population sampled.** The "characteristics of Military Times readers" are also mentioned. The explanation a few months ago was pretty specific about this: "The group surveyed is older, more senior in rank and less ethnically diverse than the overall armed services." In short, we don't even know much about what this unrepresentative slice of the military thinks.

Genuine, deliberate lying in the news is a bad idea, owing to the unpleasant consequences of being caught. Fox is lucky here because it picks up some camouflage from the pack; even the big kids are sloppy, gullible or both when it comes to survey data. But there's always a chance that if someone at the Times or the Post or McClatchy complains loudly enough, a burst of bogus statisticking can be stopped in its tracks. That's unlikely to happen at Fox, because Fox is fundamentally dishonest in ways that professional news organizations aren't.

These are rough times for journalism. Remember not to get the real kind mixed up with the Fox kind.

* If somebody out there can explain why the NYT is so goo-goo about FiveThirtyEight.com, please do. I completely don't get that.
** Whenever you see a phrase like "that means it is impossible to calculate statistical margins of error commonly reported in opinion surveys," you should feel free to reach for the firearm of your choice.

Labels: , ,

0 Comments:

Post a Comment

<< Home