How reporting on obesity and car crashes turned into a data journalism train wreck

How reporting on obesity and car crashes turned into a data journalism train wreck Rebecca Goldin, Ph.D., February 4, 2013 Data analysis is the new big thing in journalism; but it’s the little details that count carcrashTo the list of risks associated with being obese, we must – if we are to believe recent news reports in USA Today and Fox News – add dying in an automobile accidents. The source for this claim is a new study published by the Emergency Medicine Journal of the BMJ group, and therefore seems authoritative. But one of the hazards of reporting “new studies” – especially those with alarming findings – without referring back to the research that has been already undertaken on the topic is, one might say, science reporting’s equivalent of driving while under the influence. So, if we leap back a year, we find that there was a similar study published in the American Journal of Emergency Medicine with conflicting results. First, some qualification so we know what the media’s claim actually is about: It’s not that you are more likely to die from an accident if you are obese – it’s that if you are in a serious car accident, you are more likely to die from that accident if you are obese (though the claims are not so immediate in the scientific literature – see the caveats below). In particular, there is no evidence presented that obese people are more likely to be involved in a serious car accident than non-obese people. And so, according to the news, the EMJ study found that among people involved in traffic collisions, obese people were more likely to die of their injuries than non-obese people. USA Today pointed out that underweight people are also at risk. Except that this is not really what the study found. Obesity is defined as having a body mass index (BMI) level of over 30 in adults. In a comparison with death rates for people with normal BMI rates of 18.5 to 24.9, the EMJ in looking at just under 7,000 people, found no correlation between obese BMI levels of 30-34.5 and increased death (RR of 1.21 with 95% CI 0.98 to 1.49); no correlation with overweight people with BMI 25-29.9 and increased death (RR of 0.94 with 95% CI .82 to 1.09); and no correlation with underweight people with BMI less than 18.5 and increased death (RR=1.19, 95% CI 0.86 to 1.63). Instead, the authors of the EMJ study found a correlation only among the severely obese, those with a BMI 35-39.9 (RR= 1.51, CI 1.10 to 2.08) and the morbidly obese, those with a BMI greater than 40 (RR= 1.80, CI 1.15 to 2.84). When broken down into gender-specific deaths, only women who were severely obese (but not that who were morbidly obese with a BMI over over 40, strangely) were found to have a statistically significant elevated risk (RR= 2.2, 95% CI 1.27 to 3.81). Remember that a result is considered statistically significant only if the CI (confidence interval) does not include the value 1.0. In other words, the media accounts of what the study said were largely inaccurate. And this is before we examine the study in light of previous contrasting evidence. Thus, the American Journal of Emergency Medicine study published in January, 2012, found less depressing results for overweight folk, using data from a much larger sample of over 155,000 people involved in traffic accidents. The odds ratios (similar to the relative risk) suggested a statistically significant benefit for overweight people. The adjusted risk of death reported as the odds ratio compared with normal weight people was OR, .952 (95% CI .911-.995). There was no statistically significant benefit or risk to being obese (BMI 30-34.9). In this study, the risk seemed to be localized to those who are underweight (OR =1,115, 95% CI 1.035-1201), severely obese (OR 1.212, 95% CI 1.28-1.302) and morbidly obese (OR 1.559; 95% CI 1.402-1.734). So if one wanted to report the news accurately, the headlines should have said, “Car accidents and severe obesity may be a deadly mix: New study confirms risk” – perhaps with a subhead, “Previous benefit to overweight drivers and risk to underweight drivers is unconfirmed.” The American Journal of Emergency Medicine study refers BMI levels of 35-39.9 “moderately obese” but for consistency with the Emergency Medicine Journal, we use “severely obese” for these levels in order to make appropriate comparisons. Rebecca Goldin, Ph.D. is the Director of Research at STATS.org. For information on the Cardiometabolic Health Congress contact Dina Kouveliotes at 877.571.4700 or visit www.cardiometabolichealth.org