New Report on Deaths in Iraq

A new study is out purporting to show that over 600,000 Iraqis have died as a consequence of the U.S-lead war in Iraq. John Podhoretz of National Review asked me to compile into a set of questions some of the issues I raised about the study with him, and I do so below.

You can read the study here.

I should note that, while this isn't my first time around a random sample, a survey instrument, or a log-linear regression model, I do not study mortality rates and would defer to the expertise of scholars in the field (i.e. medical research) who have experience with these models. However, I do have some questions from a semi-lay perspective.

1. The survey reports a pre-invasion crude mortality rate of 5.5 deaths per year out of 1000 persons in the population (based on [edit: 82, not 2] 2 pre-invasion deaths reported in their sample). Is this consistent with death rates in countries similar in development to Iraq?

2. A one year period prior to the invasion was chosen for the pre-invasion period comparison baseline in the study. Why was this interval chosen? Note that this is a period with a coalition enforced no-fly zone in effect. Would a different interval have produced a different mortality rate? For example, what about one that included post-Gulf War reprisals? This is very significant. The pre-invasion mortality rate is integral to calculating the # of deaths attributable to the war in Iraq. Higher pre-invasion mortality rate, fewer deaths attributed to the war. And note, they get their estimate of the pre-invasion mortality rate *from* their sample. So a household is asked about deaths from each of the periods. They may be underestimating the deaths in the earliest period simply because it was so long ago.

3. Why are the confidence intervals around their estimates so large? It sends up a pretty big red flag. While sensitivity checks were done according to the authors, I wonder whether outlier clusters might be driving the inflated death totals. Their model assumes that the "variation in mortality rates across clusters is proportional to the average mortality rate." Perhaps non-constant error variance in their data may account for the inflated confidence intervals they estimate?

4. I think there is a fundamental question as to whether this baseline-comparison model is appropriate for assessing the impact of the war on the number of deaths in Iraq. It is very sensitive to the baseline selected as well as the variance in deaths across clusters. While the authors acknowledge several possible sources of systematic bias in their sample (i.e. the possibility of over-sampling higher-mortality clusters as a result of population migration), I don't think they appreciate just how much of an effect that could have on their estimates. Their method does not and cannot account for the essential non-randomness of warfare. Coalition forces do not drop bombs randomly. Insurgents do not plant IED's or car-bombs randomly. As such their sample may be severely biased by the fact that individuals living in Iraq make rational decisions to move away from hot-spots when able to do so. Less people in the household means a higher mortality rate.

5. Why was no effort made to distinguish combatants from non-combatants? I understand that there is concern for risk to the interviewers (as stated in the article), but this is pretty important. The authors argue that this is the "this the deadliest international conflict of the 21st century" and that they have urged investigation into the "excess" mortality rate in Iraq. But what if a large percentage of that 'excess' is, in fact, coalition forces killing insurgents? From a policy perspective, there doesn't seem to be much logic in treating the death of an innocent child from an IED the same as the death of the terrorist planting the IED at the hands of coalition forces...yet that is exactly what the study does. Note the study acknowledges that "Across Iraq, deaths and injuries from violent causes were concentrated in adolescent to middle age men." The exact population from which combatants are drawn.

UPDATE: IUD vs. IED spelling error corrected. Thanks to Craig for noting. The universe of possible jokes as a consequence of that error I leave to the imagination of the reader. ;)