Causation vs. Correlation: A Real World Example

Blogger Dennis Mangan points to a study that suggests the US may have suffered almost 14,000 excess deaths as a result of nuclear fallout (PDF) from the March 2011 Fukushima nuclear core meltdown. Here is the story as it originally appeared in the June 10-12 Weekend Edition of Counterpunch. Although this does not necessarily discredit the authors of the "study," you should probably be aware that Joseph J Mangano (MPH, MBA) and Janette D Sherman (MD), have been involved for many years in anti-nuclear activism.

Here is the abstract of the article as it appeared in the International Journal of Health Services:

The multiple nuclear meltdowns at the Fukushima plants beginning on March 11, 2011, are releasing large amounts of airborne radioactivity that has spread throughout Japan and to other nations; thus, studies of contamination and health hazards are merited. In the United States, Fukushima fallout arrived just six days after the earthquake, tsunami, and meltdowns. Some samples of radioactivity in precipitation, air, water, and milk, taken by the U.S. government, showed levels hundreds of times above normal; however, the small number of samples prohibits any credible analysis of temporal trends and spatial comparisons. U.S. health officials report weekly deaths by age in 122 cities, about 25 to 35 percent of the national total. Deaths rose 4.46 percent from 2010 to 2011 in the 14 weeks after the arrival of Japanese fallout, compared with a 2.34 percent increase in the prior 14 weeks. The number of infant deaths after Fukushima rose 1.80 percent, compared with a previous 8.37 percent decrease. Projecting these figures for the entire United States yields 13,983 total deaths and 822 infant deaths in excess of the expected. These preliminary data need to be followed up, especially in the light of similar preliminary U.S. mortality findings for the four months after Chernobyl fallout arrived in 1986, which approximated final figures. _Source (PDF)

The basic thesis is fairly simple: The authors claim that a comparison of CDC weekly death rates for a select group of US cities -- from just before and just after the Fukushima meltdown -- will provide a valid picture of the fatal effects of fallout from the Japanese nuclear plant on these US cities of the Pacific Northwest.

We will look at this thesis from the standpoint of Hill's Criteria of Causation, recently considered in this Al Fin blog post.

1. Temporal Relationship
The authors do present a "before exposure" vs "after exposure" scenario, which they claim demonstrates a significant rise in deaths in the US Pacific Northwest, thousands of miles downwind in the weeks just after the Japanese incident.
2. Strength
The authors claim nearly 14,000 excess deaths in the US in the 14 weeks after the detection of excess radiation, one week after the meltdown incident.
3. Dose Response Relationship
The authors are unable to present reliable graduated exposure data which might prove or disprove a "dose-response relationship."
4. Consistency
The author's claim that the rate of excess US deaths which they detected for the 14 weeks after Fukushima, is comparable to the rate of excess US deaths detected by US researchers in the first 4 months after the Chernobyl explosion and meltdown in 1986.
5. Plausibility
There is no known biological mechanism to explain this number of excess deaths so quickly, from such relatively low levels of possible radiation exposure. The excess mortality as reported by the CDC were attributed to seasonal infections and SIDS deaths.
6. Considerations of Alternate Explanations
There is no indication that the authors considered alternate explanations for their findings.
7. Experiment
An experimental design to test this hypothesis in humans would encounter ethical difficulties. Animal testing up to this point is unlikely to explain how such low levels of excess radiation could lead to such a rapid elevation of death rates.
8. Specificity
There is no specificity relationship that is detectable in this data. In other words, there is no logical connection between these "excess deaths" and possible exposure to low levels of excess radiation.
9. Coherence
The claims of the authors are not consistent with the existing body of knowledge in regard to human health responses to transitory, relatively low radiation exposures.

A far more devastating criticism of the authors' thesis can be found here.

.....there are several conclusions to be drawn here:
-- There is no spike in infant mortality due to Fukushima. Instead there is an accidental dip during the 4 weeks before the radioactive releases reached the U.S. west coast.
-- The infant mortality rate in the northwest U.S. was actually 23% higher in the first 7 weeks of 2011 than after Fukushima, 108 cases in 7 weeks give a weekly ratio of 15.43. We can thus say, by using Sherman and Mangano's own way of phrasing it, that this amounts to a decrease of 23% and is statistically significant.
-- The data for the full time period of weeks 1 - 21 amount to 272 infant deaths over 21 weeks, i.e. a weekly rate of 12.95. This is slightly higher than the weekly rate after Fukushima (12.50).
-- Janette Sherman and Joseph Mangano have a lot to explain for us...if anybody cares to listen to them after this low point in their so called scientific careers. _Source

Source
Excess deaths from relatively low, transitory radiation exposures should only show up years after the exposure. One would not reasonably expect to find "instant deaths" from such low level, short term exposures. And yet, that seems to be what the authors were trying to find, using quite noisy data from the CDC which was never meant for this type of comparative analysis.

But what is the question that everyone should be asking of the authors? "What does your follow-up data show, over the subsequent time period since the Fukushima event?"

A quick check of the most recent CDC MMWR of 6 Jan 2012 (PDF) shows a total death count for the sample 122 cities, as 9,530. This is down by over a thousand from the 3 June 2011 MMWR (PDF) which shows a death count of 10,839, for the sample 122 cities. Should I then claim that radiation from the Fukushima incident was beneficial to the health of US residents, since the US death rate has declined over time since the meltdown? Of course not. The data bounces up and down every week, for a large number of reasons. Simple correlational studies based upon this type of data should never be used to prove causation.

Index for all MMWRs in 2011, where you can find the tables for each week of 2011.

The remarkable aspect of Mangano and Sherman's paper, is the number of people who were taken in by their shoddy "research." This includes the International Journal of Health Services, as well as a wide range of "green" and anti-nuclear political sites. One of the most disheartening articles linked to this issue is this article in Oilprice.com, by John Daly.

In the real world, people are paid a lot of money to generate such pseudo-scientific "studies", "surveys," opinion polls, marketing research, and the like. Billions of dollars can change hands based upon the results of such shoddy analysis. In the case of the UN's IPCC and global climate policy, monetary redistribution into the $trillions might be easily achieved, by "scientific" methods no more valid than those used by Mangano and Sherman.

At this point in time, most of the global media is willing to cut corners on fact-checking, for the sake of "a good story," or for the sake of "advancing the agenda."

The book, How to Lie With Statistics, is as timely now as ever. If you read it and understand it, you are likely to grow angry more frequently when consuming media pieces. If you prefer bliss, you should probably choose ignorance, like most folks.

More: A quick and dirty takedown of the Mangano / Sherman "study" by Scientific American Technology Fellow Michael Moyers