Deltoid

Sometimes I think that there must be a qualifying exam in order to write for Tech Central Station. Fail the exam and you’re in. They seem to have exams in at least physics, economics, statistics, and epidemiology. Tim Worstall, the author of today’s article seems to have failed both the statistics and epidemiology exams.

Worstall is criticizing a recent study published in the Lancet that found very roughly 100,000 excess deaths in Iraq after the invasion, almost all of which were violent. He writes:

At the very least one would have to add The Lancet to that list of mainstream media which are worth 15% (or is it 5% now, the left have never really been any good at numbers) to John Kerry in the polls. What makes it a great deal worse is this, from the findings to the report. In fact, these are the findings in their totality:

“The risk of death was estimated to be 2.5-fold (95% CI 1.6-4.2) higher after the invasion when compared with the preinvasion period. Two-thirds of all violent deaths were reported in one cluster in the city of Falluja. If we exclude the Falluja data, the risk of death is 1.5-fold (1.1-2.3) higher after the invasion. We estimate that 98 000 more deaths than expected (8000-194 000) happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included. The major causes of death before the invasion were myocardial infarction, cerebrovascular accidents, and other chronic disorders whereas after the invasion violence was the primary cause of death. Violent deaths were widespread, reported in 15 of 33 clusters, and were mainly attributed to coalition forces. Most individuals reportedly killed by coalition forces were women and children. The risk of death from violence in the period after the invasion was 58 times higher (95% CI 8.1-419) than in the period before the war.”

Have a look at those confidence levels. Yup, 95%. That is, a one in twenty chance that the effect simply does not exist.

No, a 95% confidence interval does not mean that there is a 95% chance that the true value is in the interval. And even if it did, Worstall would still be wrong, since if there was a 95% chance that the true value was in 1.1–2.3, there would be a 2.5% chance that it was more than 2.3 and only a 2.5% chance that it was less than 1.1.

It gets worse:

Look at the relative risk ratios (leave out Falluja; I don’t think anyone is really very surprised to see a higher mortality rate there): 1.1-2.3. It isn’t just that it is an absurdly wide one (note, a relative risk ratio of 1 would mean no effect whatsoever) it is that if this paper was written to generally accepted statistical standards it would never have been published. With a 95% confidence level a relative risk ratio of anything less than three is regarded as statistically insignificant.

Actually, by “generally accepted statistical standards” the result is statistically significant at the 95% level since the 95% confidence interval does not include 1. The risk ratio does not have to be three or more to be statistically significant. Worstall is trying to tell his readers that if the death rate increased by a factor of 2.9 (which would be about 300,000 dead bodies), statistics could not detect this increase. Really. (Worstall’s factor of three rule is probably a confused version of the GEP scam run by Philip Morris to try to show that cigarette smoke was harmless.)

Worstall goes on:

Just to clarify that, by “insignificant” no one is stating that it is not important to those people who undoubtedly have been killed during the War. What is being said is that we don’t have enough information to be able to say anything meaningful about it. “Statistically insignificant” means “we don’t know”.

The result was, in fact, statistically significant, so we can be reasonably confident that the death rate in Iraq went up. The best estimate (not including Falluja) is 100,000 extra deaths, but the confidence interval is wide, so the estimate should be treated with caution.

Of course, it is obvious why Worstall’s article was published despite his ingorance of basic statistics—Tech Central Station is campaigning for Bush and wants to deny that the invasion of Iraq has had bad consequences.

Update: Daniel Davies has a comprehensive takedown of more defective critiques of the Lancet study, including Worstall’s.

Earlier, I wrote how Joyce Lee Malcolm had doctored a quote from the Textbook of Criminal Law to make it appear that self-defence was illegal in Britain. She wrote:

Now everything turns on what seems to be “reasonable” force against an assailant, considered after the fact. As Glanville Williams notes in his Textbook of Criminal Law, that requirement is “now stated in such mitigated terms as to cast doubt on whether it [self-defense] still forms part of the law.”

The word “it” does not refer to self-defence as Malcolm’s addition to the quote indicates, but to the requirement that the defender’s belief that his actions were necessary had to be a reasonable belief. If this requirement is no longer part of the law it makes it easier to plead self-defence.

On the firearmsregprof list Clayton Cramer disputed this, arguing:

I spent a bit of time reading the full quote, and I don’t think “doctored” is the right word. In fact, I am not even sure that Malcolm has it wrong. The pronoun “it” could be read as referring to either the requirement of “reasonableness” or self-defense. Part of why I suspect that Malcolm has correctly identified what “it” refers to, and that you have not (a easy mistake to make considering the text), is some other text that Malcolm quotes from Glanville Williams in Guns and Violence: The English Experience at p. 187. She quotes Williams at considerable length, without anything bracketed or ellipsed, in which Williams gives examples such as this:

Unfortunately, Malcolm had also doctored this quote to alter its meaning, this time without even indicating the alterations with brackets or ellipses. The table below compares Malcolm’s altered quote with the correct version. Malcolm makes it appear that it was illegal for Shannon to defend himself and that he escaped conviction on a technicality, when, in fact, the Court of Appeal had expanded the right of self-defence by removing the proportionality rule.

Correct version

Malcolm’s version

The facts of Shannon were that the deceased, a heavily built man who had convictions for violence, had been making threats against Shannon for having (as he believed) “grassed” him. Shannon, who had no history of violence or aggression, must have been living in fear of an attack for some time. When the attack came he fought back, the fight (though evidently largely one-sided) being described by a bystander as “pretty frightening.” Shannon’s evidence was that he was being held very tightly by the neck and was being dragged down and “kneed;” he feared that if he fell while in the grip of his attacker he would have “got beat up by his feet.” He lashed out with a scissors and inflicted a fatal blow. On the issue of self-defence the judge left the case to the jury with the bald question: Did the defendant use more force than was necessary in the circumstances?” On this the jury, surprisingly, returned a conviction of manslaughter. The conviction was quashed, as already said, for inadequate direction to the jury; but the Court of Appeal expressed no other criticism of the verdict. We are left with the impression that if in a similar case the judge reads out Lord Morris’s dictum to the jury, who nevertheless convict, the conviction will stand. On the dictum, it is not easy to see how “what the defendant thought” could be evidence of what it was reasonable for him to do. The usual opinion is that the question what is reasonable, in the multifarious applications of that word, is for the unaided vote of the jury, and is not a matter for “evidence” in the ordinary way. It looks very much as though the dictum is a way of escaping from the test of reasonableness without acknowledging the fact. This conclusion is strengthened by the above-quoted remark of the Court of Appeal, which (epitomising a lengthy statement of Lord Morris) distinguishes sharply between “necessary self-defence” on the one hand and “angry retaliation or pure aggression” on the other. The dichotomy allows no place in between for unnecessary but putative self-defence. All putative self-defence, it seems, falls into the category of “necessary self-defence.” In this part of the judgment, the idea that the defendant’s belief is merely evidence of reasonableness has suddenly vanished; indeed, the very word “reasonable” is dropped. It seems, therefore, that the decision makes a radical change in the law. At least where the defender fears death or serious injury, there is no proportionality rule any longer; and a good thing too—in view of the jury’s verdict in Shannon. German law, it seems, gets on without a proportionality rule, and so could we, where the facts are similar to those in Shannon. The reasoning in the decision is fudged, but that is the price one pays for a beneficial change in the law.

In 1980 Shannon was attacked by a bully—a heavily built man who had previous convictions for violence and had threatened Shannon’s life. Shannon fought back and witnesses described the fight (evidently one-sided) as “pretty frightening.” Shannon testified he was held by the neck and was being dragged down and “kneed.” He lashed out with a pair of scissors and inflicted a fatal blow. The jury heard a great deal of questioning about how Shannon happened to be carrying scissors, an issue irrelevant to the charge. In the event the jury found him guilty of manslaughter. The Court of Appeal reversed the decision, not because of the verdict, but because of a fault in the judge’s charge.

I pointed this problem out. In Malcolm’s reply she continued to insist that Glanville Williams was saying that self-defence had been narrowed. And that false quote from Glanville Williams? It was a paraphrase that she “accidently” formatted so that it looked like a quote:

Thanks for sending me the debate with Lambert and your refutation. Of course I never “doctored” anything. To begin with the entire section in which the comments I cited are found is labeled “The Necessity for Defence.” The first sentence reads: “The defence of private defence resembles that of preventing crime in the twin requirements that the act must be immediately necessary…and proportional to the harm feared.” p. 503. He then goes on to say that the requirement of reasonableness “is unhappy.” He cites Holmes about how detached reflection cannot be demanded in the presence of an uplifted knife. He notes the requirement in England is now stated in such mitigated terms….” Self-defence seems the main subject, not reasonableness because in fact the issue of reasonableness is still the touchstone. But it is the expectation, unlike that in Holmes, that great precision is expected in self-defence, and such expected precision is, in fact, unreasonable under extreme circumstances. It is the requirement of reasonableness in self-defence and self defence has been thereby narrowed.

On Shannon I was not quoting, but paraphrasing, to provide a concise example and brief description of the case. I did not use quotation marks although I suppose setting the example apart might, accidently, have made that seem the case. But I summarized the example accurately.

Note, on the subject of William’s intent in his comment on self-defence as I presented it, he adds immediately after the discussion of the Shannon case, p. 507, “For some reason that is not clear, the courts occasionally seem to regard the scandal of the killing of a robber (or of a person who is feared to be a robber) as of greater moment than the safety of the robber’s victim in respect of his person and property…The jury’s verdict in Shannon is a standing warning to all defenders of the legal danger of killing an adversary, even in self-defence.” All of this seems to me to make it clear that it is self-defence that, in Williams’ view, no longer seems to be allowed in practice.

Notice the ellipsis in her quote in the previous paragraph? Malcolm removed a sentence that made it clear that Shannon had expanded the right of self-defence:

Possibly, if the matter were fully argued, the court would apply the same rule as in Shannon, but one does not know.

To help settle the issue, I asked Jim Lindgren to comment:

I reviewed the long sections from Glanville Williams that you posted.

Having not reviewed the entire online exchange (which I hadn’t seen before now), I will confine my comments almost entirely to my own readings of Williams’s passage and Joyce Malcolm’s comment (as you quoted it).

In these sections, Williams is at times crystal clear and at other time, quite opaque. He also has an annoying habit of adding parentheticals or asides that don’t fully match the concept, such as:

“The force used in defence must be not only necessary for the purpose of avoiding the attack but also reasonable, i.e. proportionate to the harm threatened;”

This passage conflates reasonableness with proportionality, which I view as closely related but distinguishable. I might unreasonably view a man as dangerous, but respond proportionally to that perceived threat. Or I might reasonably view a man as dangerous, but respond disproportionately strongly to that real threat. I make this point in part because I think that his writing is indirect enough that it is possible to miss Williams’s main points. Sometimes he is making a legal point, sometimes he is evaluating a result or argument with which he disagrees.

In his discussion of the Shannon case, Williams appears (1) to think that the jury overstepped in not finding self-defense on the facts, (2) to agree with the result of the appeal overturning the conviction, (3) to disagree with the test (dictum) enunciated by the appellate court since (in making self-defense easier to meet) the Court muddles some distinctions, but then (4) to say that if juries are going to be so hostile to what he believes to be legitimate self-defense, then a rule making self-defense easy in one respect (gutting the reasonableness requirement) may prevent bad verdicts such as the trial jury’s in Shannon. For example, he concludes a discussion of Shannon with these words:

“It looks very much as though the dictum is a way of escaping from the test of reasonableness without acknowledging the fact. This conclusion is strengthened by the above-quoted remark of the Court of Appeal, which (epitomising a lengthy statement of Lord Morris) distinguishes sharply between “necessary self-defence” on the one hand and “angry retaliation or pure aggression” on the other. The dichotomy allows no place in between for unnecessary but putative self-defence. All putative self-defence, it seems, falls into the category of “necessary self-defence.” In this part of the judgment, the idea that the defendant’s belief is merely evidence of reasonableness has suddenly vanished; indeed, the very word “reasonable” is dropped. It seems, therefore, that the decision makes a radical change in the law. At least where the defender fears death or serious injury, there is no proportionality rule any longer; and a good thing too, in view of the jury’s verdict in Shannon.”

I have tried to give a taste for just how nuanced Williams’s argument is, which is by way of explaining that it would be quite easy for someone to misread him innocently. Indeed, I had to read the relevant sections three times to get the main moves in his back-and-forth argument (assuming that I have captured them even now).

What situation falls in the “place in between for unnecessary but putative self-defence”? There are two distinguishable situations that Williams might have had in mind: (1) where some self-defense is necessary, but the actions taken (though honestly pursued for defense purposes) are not proportional; and (2) where self-defense is honestly pursued, but unnecessary. Williams appears to think that the court comes close to eliminating the reasonableness requirement, by presenting a choice “between ‘necessary self-defence’ on the one hand and ‘angry retaliation or pure aggression’ on the other. By not requiring explicitly that the force be reasonable (which Williams takes to mean proportional), Williams argues that the defense for self-defense has been expanded by the Court of Appeals in Shannon so that reasonableness/proportionality may have been in effect read out of test. Then he adds the fillip: “It seems, therefore, that the decision makes a radical change in the law. At least where the defender fears death or serious injury, there is no proportionality rule any longer; and a good thing too—in view of the jury’s verdict in Shannon.”

This is all a long way around to explain the passage that Joyce Malcom quoted:

“That willingness was further undermined by a broad revision of criminal law in 1967 that altered the legal standard for self-defense. Now everything turns on what seems to be “reasonable” force against an assailant, considered after the fact. As Glanville Williams notes in his Textbook of Criminal Law, that requirement is “now stated in such mitigated terms as to cast doubt on whether it [self-defense] still forms part of the law.”

Here is the Williams quote, where he foreshadows his Shannon argument (which I discussed above in detail):

The requirement of reasonableness is unhappy. Enough has been said in criticism of it, and the CLRC has recommended that it should be expunged from the law. In practice, as we have seen, the requirement may be construed indulgently to the defendant, for, as Holmes J memorably said in the United States Supreme Court, “detached reflection cannot be demanded in the presence of an uplifted knife.” As we shall see in the next section, the requirement is now stated in such mitigated terms as to cast doubt on whether it still forms part of the law. [emphasis added]

As I read it, the antecedent of “the requirement” is “The requirement of reasonableness,” not “self-defense.” This inference is strongly supported by Williams’s long discussion of Shannon in the “next section,” where Williams writes:

“the dictum is a way of escaping from the test of reasonableness without acknowledging the fact,” and “the very word ‘reasonable’ is dropped,” and “At least where the defender fears death or serious injury, there is no proportionality rule any longer; and a good thing too . . . .”

Since Williams treats reasonableness and proportionality as the same thing, I read him as restating that the reasonableness “requirement is now stated in such mitigated terms as to cast doubt on whether it still forms part of the law.” Williams is saying, not that it is doubtful that self-defense exists, but that it is doubtful that defendants who use force will have to overcome a full-fledged reasonableness requirement. In short, I agree with Tim Lambert’s basic reading of the Williams’s passage (and disagree with Joyce Malcolm’s). Overall, however, Williams appears to see the scope of self-defense in England as not broad enough, a point that I believe Malcolm notes in her response. Williams does not see self-defense as so weak as to be doubtful whether it exists.

Of course, I could be wrong in my interpretation because the Williams passages are so obliquely written, and, even if I am right, it is understandable to miss on first reading what I see as Williams’s main points.

James Lindgren
Northwestern University

Unfortunately, Malcolm has continued to repeat her misrepresentation of Glanville Williams, most recently two days ago. This isn’t just an academic argument. By misinforming the British about the law on self-defence she may succeed in dissuading people from defending themselves. Disgraceful.

Tech Central Station has published Tim Worstall’s admission that his critique of the Lancet Iraq study was completely wrong:

Further to my article of Friday on this subject. I’m afraid I mangled the statistical argument. My inadequate knowledge of the subject led me to make an argument that is incorrect. I stand by my contention that there is something fishy about this study (leaving aside the politically motivated timing of its publication, something the author has been clear about himself) yet have to admit that I have not found it, leaving me with nothing but personal prejudice upon which to stand my argument. I would also like to make clear that this subject was not “assigned” to me, the idea, research, argument and errors were all my own, as was my request for this clarification. Just in case you are wondering, being fact checked by the Pajamahaddin and being found in error does hurt and I hope that future writings will be, where necessary, so corrected.”

On his blog Worstall thanks Daniel Davies and me for the correction. Such decent behaviour is unfortunately not common at Tech Central Station. Authors like John Lott and Iain Murray just repeat their false claims, while Glenn Reynolds posts a correction but does not acknowledge the source.

Tech Central Station didn’t just post Worstall’s correction by itself. They have had a second attempt at debunking the Lancet study, posting an article by Michael Fumento. Fumento argues:

the researchers didn’t feel themselves bound by anything official, like death certificates. Interviews were just fine. “In the Iraqi culture it was unlikely for respondents to fabricate deaths,” they wrote.

Unfortunately, Fumento seems to have missed the immediately preceding sentences in the Lancet paper, where they noted that, when asked, 81% confirmed with death certificates:

In 63 of 78 (81%) households where confirmations were attempted, respondents were able to produce the death certificate for the decedent. When households could not produce the death certificate, interviewers felt in all cases that the explanation offered was reasonable eg, the death had been very recent, the certificate was locked away and only the husband who was not home had the key. We think it is unlikely that deaths were falsely recorded.

Fumento’s “killer” argument is:

Cluster sampling can be valid if it uses reliable data, rather than on inherently unreliable self-reporting. But it can also be easily skewed by picking out hotspots — like determining how much of a nation’s population wears dentures by surveying only nursing homes.

In fact, intentionally or otherwise, that’s pretty much what The Lancet did. Most of the clusters had no deaths whatsoever. But here’s the real bombshell: “Two-thirds of all violent deaths were reported in one cluster in the city of Falluja,” the journal reported. That’s it; game over; report worthless.

Trouble is, Fumento has once more been extraordinarily careless in his reading of the study. Here are the two sentences in the report that follow the one he quoted:

If we exclude the Falluja data, the risk of death is 1.5-fold (1.1-2.3) higher after the invasion. We estimate that 98 000 more deaths than expected (8000-194 000) happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included.

That’s right, they properly excluded the outlier Falluja in their estimate of 98,000 and Fumento didn’t notice this fact. That’s it; game over; Fumento article worthless.

I’m starting to feel embarrassed for Tech Central Station. Do you think they’ll have a third go at the Lancet?

Also of interest is Chris Lightfoot’s demolition of more lame critiques of the study.

I wrote earlier how it seems that you must fail a qualifying exam before you can write on a topic at Tech Central Station. Now the errors in Fumento’s critique of the Lancet study.aren’t errors in epidemiology—they seem to result from not having read the study. Indeed, in comments at TCS, Fumento seems to be asking for help to find out what it said:

You imply rather strongly that you’ve read the report. If so, please inform us of what the extrapolation was that DID NOT rely on the Falluja cluster. I’m waiting.

and then:

I asked the wrong question. I meant to say how many were in the extrapolation that DID include the Falluja cluster?

and also:

We also know there is absolutely no way to randomly select 33 clusters.

Again, if he had read the study, he would have known that they did randomly select 33 clusters, and that they explained the randomization procedure in great detail.

But now, in a comment on my blog, Fumento has proved that he meets the TCS requirement to write about epidemiology—he demonstrates ignorance of a basic epidemiological principle:

The alleged 100,000 deaths were those above the pre-war baseline. That baseline was predicated on a figure of 5.0 deaths per 1,000. BUT the figure for the US at the time was 8.3 deaths per 1,000. Obviously Iraq was one of the safest countries on the face of earth prior to the Yankee imperialist invasion. In fairness, the CIA Worldbook uses a 5.6 per 1,000 figure for Iraq but what was it’s source? Saddam’s ultra-trustworthy government, of course. Thus The Lancet is using figures even lower than the Land of Baghdad Bob was putting out.

OK, some basic epidemiology:

Comparing Mortality in Different Populations: an important use of the rates—recall that the populations may differ in regard to many factors which affect mortality, of which age distribution is the most important—so to compare, we hold the factor (age) constant.

Consider:

Death Rates by Age and Race, Baltimore City, 1965
	All	<1yr	1-4yr	5-17yr	18-44yr	45-64yr	>65yr
White	14.3	23.9	0.7	0.4	2.5	15.2	69.3
Black	10.2	31.3	1.6	0.6	4.8	22.6	75.9

Overall [this is called CRUDE or UNADJUSTED mortality], the death rate for blacks is LOWER than it is for whites. This is unexpected given the conditions at the time for blacks with respect to health care access and living situations. Why would this happen? Well, if we look at the stratified rates by age, we see that the rate for blacks is HIGHER at every age level. This seeming conundrum is explained by the fact that, while the rates increase markedly for both races in the over 65 bracket, there are more whites left in that age group. So many, in fact, that the overall mortality for whites overwhelmingly occurs in the over 65 age group, while the same is NOT the case for blacks.

Hmmm, so do you think Iraq and the US have the same age distribution? From the CIA Factbook—the very source that Fumento cited—we find that the US has a very different age structure with a much larger proportion of old people. And notice that neighbouring Syria has a similar age structure to Iraq and a death rate of 4.96, consistent with the Lancet study.

Country	US	Iraq	Syria
Death Rate	8.34	5.66	4.96
Median Age	36 years	19.2 years	20 years
Age Structure	0–14 years: 20.8% 15–64 years: 66.9% 65 years and over: 12.4%	0-14 years: 40.3% 15-64 years: 56.7% 65 years and over: 3%	0-14 years: 38% 15-64 years: 58.7% 65 years and over: 3.3%

Fumento’s argument that the 5.66 number for Iraq was falsified by Saddam’s regime makes no sense. If Saddam was cooking the numbers he would have made them higher and claimed that the sanctions were killing vast numbers of people.

And his claim that the Lancet number of 5.0 is inconsistent with the CIA Factbook number of 5.66 is also wrong, since he ignores the fact that the 95% confidence interval for the Lancet study was 3.7–6.3, and 5.66 lies in that interval.

Also of interest might be this Chris Mooney post on Fumento’s book on biotechnology.

Update: Fumento replies:

Much of what you write is simply idiotic, as in saying the clusters were “randomly sampled.” How is that even possible, aside from throwing darts at a dartboard? It also implies you can read the researchers’ minds. Ah, The Amazing Lambert. But at least I’ve found one person who really does believe we’ve been violently killing civilians at a rate of over 180 per day. I was afraid the Lancet propagandists weren’t going to get away with that. You can blog all you want, but my next column is also on this. It goes out to over 350 newspapers and lots of websites that each have more traffic than yours. You occupy the pitiful place of the harmless blogger who blogs because nobody in his right mind would punish him. Rant, rant, rant. The world careth not.

Looks like Fumento flunks statistics as well as epidemiology. Random sampling is a fundamental concept in statistics and despite Fumento’s incredulity it is actually possible to carry it out. I didn’t have to read the researchers’ minds to find out how they randomized the clusters; I just had to read their paper, something that Fumento still has not done. And no, I don’t believe that we are killing civilians at over 180 a day—that isn’t what the paper found. I do believe that Fumento’s critiques of it are worthless, if that is any help.

Update 2: Fumento replies again:

You cannot randomly choose physical households in a place like Iraq. Those data are not in a computer. Do you think it’s like campaign polling in the States?

Here is the methodology that is beyond Fumento’s comprehension:

We assigned clusters to individual communities within the Governorates by creating cumulative population lists for the Governorate and picking a random number between one and the Governorate population. Once a town, village, or urban neighbourhood was selected, the team drove to the edges of the area and stored the site coordinates in a global positioning system (GPS) unit. We assumed the population was living within a rectangle, with the dimensions corresponding to the distances spanned between the site coordinates stored in the GPS unit. The area was drawn as a map subdivided by increments of 100 m. A pair of random numbers was selected between zero and the number of 100 m increments on each axis, corresponding to some point in the village. The GPS unit was used to guide interviewers to the selected point. Once at that point, the nearest 30 households were visited.

And if you are interested in more of the Fumento follies, look here and here.

The Anchorage Daily News has published a new version of Michael Fumento’s attempt to debunk the Lancet study on deaths in Iraq. How does it differ from his previous attempt? Well his key argument was that their estimate was skewed by the inclusion of the Falluja cluster. But it is perfectly clear from the report that Falluja was excluded from their estimate. Fumento knows this because he responded to my post with a comment, and he specifically asked questions about the inclusion of Falluja in the comments to his TCS article. In his new version he tacitly admits his error by quietly dropping his false claim. Of course, he hasn’t corrected his TCS article. He also aware that they did look at death certificates, so it is dishonest for him to repeat his claim that “the researchers didn’t feel bound by anything official like death certificates”.

Unfortunately he has replaced his bogus claim about Falluja with something worse—his ridiculous comparison of crude death rates between the US and Iraq. (I have explained what is wrong with that here.)

It is disgraceful that someone so wilfully ignorant of basic science has published a column on a scientific question in a newspaper. You can send a letter to the editor at the Anchorage Daily News here. (Leave a comment if you do write a letter.) Contact information for Scripps Howard News Service is here. I think it would be particularly helpful if any epidemiologists in my readership contacted SHNS.

Update: Fumento replies:

Actually, the major changes were additions — including quite legitimately pointing out that The Lancet insisted on using as its baseline pre-war mortality a number far lower than Saddam had used. That gave a range in the paper is inconsequential; the figure they used for their all-important 100,000 figure was five per 1,000.

5.0 is not “far lower” than 5.66. The post-war mortality rate was 7.9 (that’s excluding Falluja—if you include Falluja it was 12.3), so whether the pre-war rate was 5.0 or 5.66, it is still a substantial increase. I notice you offered no defence of your ridiculous comparison of crude death rates between the US and Iraq.

Fumento continues:

I pulled the section about Falluja being included because it confused people — like you. Find in the paper where they provide an equivalent to the 100,000 figure but exclude Falluja deaths. You can’t, because it’s not there. The Lancet has lied and you support it because you happen to like its conclusions, not because those conclusions where arrived at scientifically.

I already gave the exact quote from the paper—it was from the same paragraph that Fumento quoted in his TCS piece. Here it is again, with extra emphasis:

We estimate that 98 000 more deaths than expected (8000 194 000) happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included.

And no, I do not like the conclusions. I find them most unpalatable. But rather than inventing specious grounds for dismissing the study, I think it is better to face up to reality.

Yet another person has tried to refute the Lancet article. John Brignell dismisses the study just because:

A relative risk of 1.5 is not acceptable as significant.

Actually the increased risk was statistically significant. You won’t find support for Brignell’s claim in any conventional statistical text or paper. To support his claim he cites a book called Sorry, wrong number!. Trouble is, that book was written by…. John Brignell. Not only that, it was published by… John Brignell. Brignell is a crank who dismisses the entire field of modern epidemiology as some sort of plot by scientists to scare people. We encountered him before in this post where, armed with no evidence whatsoever, he insisted that the ozone hole had always been present.

To see how silly Brignell’s “relative risk of 1.5 is not acceptable as significant” claim is, consider this: Suppose we had perfect records of every death in Iraq and there were 200,000 in the year before the invasion, and 300,000 in the year after. Then the relative risk would be 1.5 and Brignell would dismiss the increase as not significant even though in this case we have absolutely certainty that there were 100,000 extra deaths.

One interesting feature of blogspace discussion of the Lancet study has been the comments from warbloggers, who, despite not even knowing what cluster sampling is, have been absolutely certain that the methodology of the study has been discredited. For instance, Arthur Chrenkoff admits:

I’m not a statistician

but none the less concludes that Shannon Love had demolished the study. (Daniel Davies deals with that “demolition”).

Or Michael Totten at Instapundit, who is certain that the study uses very bad methodology. Bill Trippe sent him a correction:

Did you even read the paper before you decided Shannon Love’s argument was so brilliant? Probably not, because if you had, you would see that she clearly did not understand what she was reading. Case in point: she makes a grand pronouncement (even putting it in bold), about the inclusion of Fallujah in the conclusions. Guess what? Fallujah was excluded from the results as an outlier.

Totten, of course, did not correct the erroneous post.

Or Cori Dauber. I corresponded with her last year because she bought Lott’s ridiculous claim that the murder rate in Baghdad was very low. and she admitted that she wasn’t good with numbers. But she is sure that the methodology is “garbage” and calls the study a “lie”.

Our last example, by Anthony Rickey, is like a Bizarro world version of this post. He takes issues with the folks who accepted the Lancet’s study, reckoning that their prejudices have blinded them to the obvious (to him) flaws. He even believes that the Lancet study will be another scandal like the affair of Dan Rather and the forged memos. Unfortunately it is Rickey’s prejudices that have blinded him to the flaws in the criticism of the study. (See Daniel Davies again.)

But what about all the people who accepted the study’s result who also didn’t know much about statistics? Well if you don’t know enough to evaluate the study yourself, you’d have to trust the experts on statistics, and I don’t think that it would be unreasonable for you to suppose that a journal with the prestigious reputation of the Lancet would have checked those statistics thoroughly. And you would have been right.

John Fleck commented on my exchange with Fumento here and here. He responded to Fumento’s silly charge that I “occupy the pitiful place of the harmless blogger who blogs because nobody in his right mind would punish (sic) him” with:

That’s of course ad hominem, something of a poor refuge in any argument. But it’s worse than that. It’s plain dumb in this age of Dan Rather and Little Green Footballs for a writer of Fumento’s stature to expect us to think he wins the argument because his work is published in mainstream media.

Sure enough, Fleck got an email from Fumento:

Subject: Ah, another worthless observation from somebody that can’t get published so he blogs.

Earth to Inkstain and Lambert: Other than Inkstain caring what Lambert says and Lambert caring about what Inkstain says (perhaps), nobody cares what either of you says. Not only are you fully contained in the blogosphere, you’re actually in a much tinier realm than that. Meanwhile of the many places my piece on the Lancet trash appeared is today’s Daily News, weekend population above 500,000. You attack not out of a sense that injustice has been done regarding the Lancet report, but out of jealousy. But if you cleaned up your act, you might just find that somebody somewhere, even with a circulation of ten, would occassionally print you. Alas, you will not. You are a lost cause.

Which is pretty funny, since if Fumento had bothered to click on Fleck’s About me link he would have discovered that Fleck is the science writer for the Albuquerque Journal.

Fumento then sent Fleck another abusive email:

First, it seems to me that any nationally syndicated columnist, including those I can’t stand, is a journalist — whether John Fleck acknowledges it or not. Second, I dropped one of my arguments from the TCS piece only because it confused people with simple minds. Like you. As it happens, I also had to cut 200 words even as I added in new information. So we are faced with two possibilities here, neither pleasant. A) You’re not particularly bright; B) You’re not particularly bright.

I wonder if Fumento has managed to read the Lancet paper yet?

On a slightly more serious note: Fumento has managed to get his attack on the Lancet paper published in the Sacramento Bee, the Arizona Daily Star and the Minneapolis-St Paul Star Tribune. Just think how much it would bug him if you wrote a Letter to the Editor about his column.

Oh, and my humble blog is now the second site returned by a Google search for “Fumento”.

The defective refutations of the Lancet study just keep on coming. First, we have Gerard Alexander writing in the Weekly standard:

But the study’s researchers were sure to survey in Falluja, far and away the most violent city in post-invasion Iraq. Falluja turned out to be such a wild statistical outlier that they offer two estimates, one with Falluja included and one with it kept out. But questions about just how representative the sample sites were go deeper than this. The researchers selected their survey sites households for such unclear reasons that we simply can’t extrapolate to the whole country with any confidence. What are the chances that they have over-sampled the most violent parts of the Sunni Triangle and under-sampled the calmest Kurdish and Shia areas? Without better statistics about population and violence, we can’t know, and neither can they. The fact that they don’t explain their strategy doesn’t build confidence in their research design or their conclusions.

Alexander comes from Fumento’s don’t-bother-to-read-the-study school. If Alexander had bothered to read the Lancet study he could not possibly have missed their lengthy and detailed explanation of how they randomly selected the clusters. After this gross deviation from the standards of scholarship, Alexander then has the hide to accuse the Lancet of jettisoning “their standards of fairness, restraint, objectivity, and integrity.”

Next, we have Robert Lichter of STATS who writes:

The crucial assumption is that any increase in deaths after the invasion began on March 19, 2003 is associated with the conflict and subsequent occupation, to the exclusion of any other factor. Specifically, their sample included 46 reported pre-invasion deaths, only one of which was violent, and 89 post-invasion deaths outside Falluja, 21 of which were violent. According to a table that breaks down the causes of death, fewer than half of the “excess deaths” (45 percent) resulted from violence. One in five was accidental, one in six was due to heart attack or stroke, just under one in 10 was caused by infectious disease, and the same proportion consisted of neonatal or infant deaths. Yet all these deaths without exception were attributed to the war and occupation.

It seems pretty unlikely that a doubling in the number of deaths was just a coincidence. If Lichter wants to suggest that something else caused it, he needs to explain what. And I hope it is clear how the disruption to medical services and electricity and water could cause increases in deaths from diseases.

The Economist and Stephen Soldz have discussions of the study that are more balanced than the hatchet jobs by Alexander and Lichter.

Update: I sent my comments to Alexander and he replied:

I appreciate your certainty that I haven’t read the study, but am afraid its misplaced. I’m even sorrier that you missed my point: in the absence of better stats, especially on violence, no-one—no matter how lengthily they think they’re explaining their methods—can survey a sample in whose representativeness we can have confidence. It also means we can’t exclude the study’s findings as accurate, which is why I don’t dismiss them, though I question them. What I got most specific about was the Lancet’s apparent disinterest in their own previous published estimates of child/infant mortality before March 2003. I’m not dismissing those previous findings without explanation; they are.

So Alexander read the extensive explanation of their strategy for selection and then accused the authors of not explaining their strategy. Readers will have to think of the best word to describe Alexander’s conduct. Now he claims that it is not possible at all to get a representative sample. He doesn’t explain why it is impossible, probably because he cannot. Nor do they dismiss previous findings on infant mortality without explanation. They clearly state:

The Coalition Provisional Authority and the Iraqi Ministry of Health have identified the halving of infant mortality as a major objective. In the absence of any surveys, however, they have relied on Ministry of Health records. These data have indicated a decline in young child mortality since February, 2001, but because only a third of all deaths happen in hospitals, these data might not accurately represent trends. No surveys or census-based estimates of crude mortality have been undertaken in Iraq in more than a decade, and the last estimate of under-five mortality was from a UNICEF sponsored demographic survey from 1999.

In other words, infant mortality has decreased since it was last measured.

Alexander claimed that the Lancet chose to throw away its reputation with the article, but it is Alexander who has thrown his own reputation away with his disgraceful article in the Weekly Standard.

Fumento left a comment on my earlier post. Instead of discussing the Lancet article, he boasted how his column had been published in the on the web site of the Lake Wylie Pilot, which is a free weekly newspaper serving a town of 3,000 people. Hey, my little blog has a greater circulation than that.

Eye Doc linked to Fumento’s attack on the Lancet, so I left a comment explaining what was wrong. Fumento replied:

Tim Lambert is on a personal Jihad to debunk my debunking. I did not say death certificates were not used, they were. But so was alleged personal recall. That means that if a family recalled ten deaths of people who were alive and well, they went straight into the pot.

Nowhere in his piece did Fumento say that they used death certificates, instead he implies they did not with this: “the researchers didn’t feel themselves bound by anything official, like death certificates.”

Fumento continues:

The authors claimed to have come up with one set of numbers including Falluja, another without. But strangely, they never present the “without numbers.” Lambert knows this because I told him directly. Anyway, it’s in the study — or rather, it’s NOT in the study.

Who are you going to believe, Fumento, or your lying eyes?

And David Mason piles on here and here, opining: “Michael Fumento is a bitter, bitter man.”.

Correction: In comments, Fumento complains:

you are now lying about what I wrote to you. I didn’t say I appeared in the Lake Wylie Pilot, I said my column is picked up by the McLatchey News Service that posts it automatically to the sites of about a dozen papers.

My apologies. When he wrote that “it goes to” the Lake Wylie Pilot he meant that it appeared on their web site, which is apparently different from appearing in the Lake Wylie Pilot. I apologize unreservedly to Mr Fumento for stating that his column had appeared in the Lake Wylie Pilot when it had merely appeared on the Lake Wylie Pilot web site. I hope that Mr Fumento’s reputation has not been harmed by my erroneous statement.

Daniel Davies has an excellent roundup of the Lancet discussion.

I’ve added an update to my post about Gerard Alexander’s attack on the Lancet.

Chris at Mixing Memory takes down another Lancet critique, this one by John Ray.

The fun continues in this comment thread. Highlights:

Michael Fumento:

The authors claimed to have come up with one set of numbers including Falluja, another without. But strangely, they never present the “without numbers.” Lambert knows this because I told him directly. Anyway, it’s in the study—or rather, it’s NOT in the study.

John Fleck:

A quick refresher on where the Lancet study’s authors included the “without Falluja” numbers. It’s in the paper’s abstract. That’s the thing that comes right at the beginning: “We estimate that 98,000 more deaths than expected (8,000–194,000)happened after the invasion outside of Falluja and far more if the outlier Falluja cluster is included.”

It would be one thing if Fumento simply misspoke in his original TCS piece when he said the inclusion of the Falluja cluster biased the study and made it worthless. But in the face of Lambert and others repeatedly quoting the precise, clear and unambiguous wording with which the study did exactly the opposite, excluding the Falluja data as an outlier, Fumento continues to misrepresent it

Jonathan Dursi:

There is nothing in Fumento’s analysis which even comes close to disputing the Lancet article—indeed, it’s not even clear he read the abstract.

Michael Fumento:

Lambert, Fleck, and Dursi (an epidemiologica expert because his field is astronomy) just won’t let go. The study did not present numbers that included Falluja, either in the abstract or text. Yet they accuse ME of not reading it.

Mark Tyrrell Frank:

Michael Fumento is quite extraordinary. How can he continue to say this? The text in the abstract and in the body quite clearly gives figures both with and without Falluja. E.g. maximum likelihood estimate of death risk is increased 2.5 times including Falluja and only 1.5 times without. This is not so much deception as sheer madness. Or maybe he has a different copy of the paper?

Sheer madness or deception? I report, you decide.

The Times Higher Education Supplement has produced a list of the world’s top universities. They must have used a good methodology because UNSW came in at number 36. The United States dominated the list, with 20 out of the top 50 places. I wasn’t quite sure exactly where all the American universities were, so I marked the locations on a map of the United States that I found here. Probably most of my readers already know where they all are, but I thought I’d share the map with you.

The Site Meter counter just ticked over to 300,000 visits. I really appreciate all the visitors, especially the ones who have left comments.

For someone who holds blogs in contempt, Michael Fumento sure spends a lot of time posting comments to blogs. Here he is again: (Hat tip: John Fleck, now the third site on a Google search for “Michael Fumento”)

My writing on the Lancet article has been Fleck’s obsession for over a week, and everything he says is wrong including this latest posting. First, simple subtraction tells you in 19 percent of the households death certificates were NOT used. But that’s not the equivalent of 19 percent of the deaths. If a household said a bomb killed five family members, that’s worth five death certificates. For all we know, half the deaths reported came from recall. Second, apparently Lambert has never seen a death certificate and thinks coroners or medical examiners are psychics. A death certificate will indicate death by violence, but it won’t say something like “500 lb. JDAM dropped by U.S. F-18.” Every day we see pictures of Iraqis blown up by Iraqis and other Arabs. Yes, the people were blown up but no we didn’t do it and the death certificate isn’t going to say one way or another. Finally, he ignores evidence outside of the paper that I presented, including two anti-war groups that said the Lancet figures were far too high and a certain individual who pegged them too high by 85,000. His name? Osama bin Laden.

Note that after his apparent flip-flop on the question of whether the study reported an estimate excluding Falluja, Fumento is silent on that question. Anyway, let me address the points that Fumento does raise. First, hardly any households had multiple deaths, so his notion that half the deaths were not confirmed is wrong. Second, yes, the death certificate will just say that the cause of the death was an explosion without saying who was responsible. For that we must rely on the word of the family. But the person is still dead as a result of the invasion, no matter who did the killing. Third, the numbers from groups like Iraq Body Count are measuring something different from the Lancet survey–the number of confirmed civilian deaths directly caused by the war. The IBC number is certainly an underestimate since not all deaths will be reported. Not does it count indirect effects like increase in disease because of breakdowns in medical infrastructure. The IBC does not say that the Lancet estimate is too high:

Others have asked us to comment on whether the Lancet report’s headline figure of 100,000 is a credible estimate. At present our resources are focused on our own ongoing work, not assessing the work of others.

Fumento, of course, claims otherwise.

Kevin Drum is displeased that the LA Times published another op-ed from John Lott:

The man is a fraud and the Times demeans itself by allowing him space on their pages.

Sadly, it looks as if Michael Fumento has retired from the field. All I can offer any folks suffering withdrawal symptoms is this thread. James M describes it like this:

I noticed a truly spectacular example of what I suppose is the unarmed kamikaze approach to debate carrying on in the comments boxes. Not so much being savaged by a dead sheep, as seeing someone punch themselves repeatedly in the face. It is painful to watch.

But yet, like a car wreck, you must look.

Tech Central Station has an article by Robert McHenry criticising Wikipedia for innacuracy. Yes, this Tech Central Station. McHenry found an error in the Wikipedia article on Alexander Hamilton. Of course, within hours of his pointing out the error, it was fixed. Unlike the numerous and far more serious errors you see in Tech Central Station.

Incidently Tech Central Station is published by a lobbying company that works for Microsoft. And Microsoft Encarta is a competitor to Wikipedia. And McHenry was Editor in Chief of the Encyclopædia Britannica, another competitor to Wikipedia.

From Kieran Healy I learn that Otis Dudley Duncan has died. It was Duncan who started the investigation into John Lott’s mysterious survey. When Duncan first contacted me with his concerns, I found it almost impossible to conceive that someone would fabricate a survey rather then admitting to a careless error, but Duncan’s insight into human nature has proven to be much better than mine. Although he preferred to stay out of the limelight, Duncan gave me the benefit of his wisdom on the Lott affair. If my writings on Lott’s mysterious survey have been astute, Duncan deserves the credit. It has been an honour and a privilege to work with him.

Although he was long since retired, Duncan also recently worked on debunking some other dodgy polls, ones that purported to show wide support for the teaching of Creationism. He will be sadly missed.

Update:The New York Times has an obituary. The Humanist Society of Santa Barbara’s newsletter has an appreciation.

The latest pundit to have a go at the Lancet study is Andrew Bolt. Like most of the critics, Bolt just does not have the statistical background to produce a competent critique. In Bolt’s case this is even less excusable, since he had the benefit of the Economist’s excellent article, but unfortunately Bolt does not seem to have understood it.

Bolt writes:

Just ask yourself: Have more than 180 Iraqis, mainly women and children, really died every day, on average, for the past 18 months, usually at the hands of the Americans?

If so, where are all the funerals? Where are the pictures? Where are the news reports

The CIA World Factbook reports that the death rate in Australia is 7.38 per 1,000. That’s more than 400 deaths per day. Just ask yourself: Have more than 400 Australians, really died every day, on average, for the past 18 months? If so, where are all the funerals? Where are the pictures? Where are the news reports?

Bolt continues

But few of the commentators who seized on the survey bothered to ask such basic questions, or even to heed Human Rights Watch, which warned: “The numbers seem to be inflated.”

Perhaps they should read what Human Rights Watch said later:

“I hate the interview I did for The Washington Post,” he says. “I was on the train, I hadn’t read the report yet [when the Post’s reporter called for comment]. In general, I’m not as negative as that [Post] report made me seem. This is raising issues that are not heard of much in the U.S.”

Bolt continues:

Its researchers interviewed 7868 Iraqis in 988 households in 33 neighbourhoods around Iraq, allegedly chosen randomly, and asked who in the house had died in the 14 months before the invasion and who in the 18 months after.

“allegedly chosen randomly”? The researchers carefully explain, in detail, how the randomization worked. If Bolt wants to accuse them of lying about this, why does he bother with the rest of his attack? Why not just accuse them of making up all the results?

Bolt argues that the pre-war infant death rate that the survey found (29 per 1,000) is wrong because:

2002 figures from UNICEF, which in a much bigger survey of 24,000 households found the infant mortality rate in Iraq before the war was actually a tragic 108 deaths per 1000 infants.

But the 2002 UNICEF figures are based on a survey conducted in 1999 (which meant that it was counting deaths that happened in 1998). And the 1999 survey used similar methodology to the new survey, so the difference can’t be because there was something wrong with the methodology of the new survey—the most likely explanation is that the oil-for-food program lowered the infant mortality rate to between 1998 and 2002.

In March 2002 Matt Welch wrote an article debunking extreme claims of deaths caused by sanctions:

Sanctions critics almost always leave out one other salient fact: The vast majority of the horror stats they quote apply to the period before March 1997, when the oil-for-food program delivered its first boatload of supplies (nearly six years after the U.N. first proposed the idea to a reluctant Iraqi government). …

As the U.N. Office for the Iraqi Program stated in a September 28, 2001 report, “With the improved funding level for the program, the Government of Iraq is indeed in a position to address the nutritional and health concerns of the Iraqi people, particularly the nutritional status of the children.” Even two years earlier, Richard Garfield noted in his survey that “the most severe embargo-related damages [have] already ended.” …

Those who get past the initial frustrations of researching the topic usually end up on Richard Garfield’s doorstep. His 1999 report—which included a logistic regression analysis that re-examined four previously published child mortality surveys and added bits from 75 or so other relevant studies—picked apart the faulty methodologies of his predecessors, criticized the bogus claims of the anti-sanctions left, admitted when the data were shaky, and generally used conservative numbers.

So Richard Garfield is the go-to guy on Iraqi infant mortality. Does he think that there is something wrong with the numbers in the Lancet study? Probably not, since he is one of the authors.

Andrew Bolt then demonstrates a lack of understanding of basic statistics. After noting the high death rate in the Falluja cluster he writes:

Truly, these statistics are unbelievable. I suspect the study’s authors thought so, too, which may be why they left the Fallujah figures out—calling them unrepresentative—when they calculated Iraq’s death toll since the invasion.

But the survey techniques they used to give clearly wrong figures in Fallujah are the same ones they used in the other 32 clusters of households that they interviewed elsewhere in Iraq.

An estimate based on Falluja is unreliable because it is based on one cluster, and there is a good chance that that cluster might not be representative. But it does not follow that because an estimate based on one cluster is unreliable that an estimate based on 32 clusters is unreliable. In fact, the size of the sample is the crucial difference—you need a reasonable sample size to get a reliable estimate. If you believe Bolt’s argument, then all surveys can be similarly dismissed. For example, a poll on voting intentions that just surveyed one random person is clearly going to give an unreliable result. Does it follow that a poll of a 1000 people must also give an unreliable result?

Bolt finishes with:

Too many commentators seem too desperate to believe the worst of the Americans and to belittle the liberation of Iraqis from a tyrant.

That desperation means even junk surveys such as this will find many eager believers, ready to hear the very worst. And to recklessly repeat it.

Too many war supporters have a desperate need to deny that the Iraq war has had some bad consequences. This desperation means that even clueless critiques, such as Bolt’s will find credulous believers. This is unfortunate. I believe we need to be clear-eyed about the consequences of the war and face up to reality.

Update: Bolt’s response:

I’m sure you don’t need me to point out the obvious omissions, red herrings, false comparisons of additional deaths to total deaths, and the telling failure even to cite the shocking figure of Iraqi child deaths given even by the “expert” cited as the last word on the issue.

Of course I did cite the infant mortality figures. I don’t know why Bolt would write something so plainly false. And did you notice the clever clever way he used square quotes to disrespect Richard Garfield?

Update 2: Tim Blair thinks that Bolt’s hopelessly innumerate criticism proves that Peter FitzSimons is stupid for citing the 100,000 figure. Unfortunately, all Blair has proved is that he himself has no understanding of basic statistics.

Darp and Jess organized a get together of bloggers in Sydney last Friday. I wandered along and, well, there were a lot of people there. Despite having been to two previous such gatherings, the only bloggers I had met before were Jason Soon and c8to. I’m too lazy to link to everybody so I’m going to reward TimT’s shameless link whoring by suggesting you go to his post and follow the links from there to all the photos and descriptions and more importantly, a whole bunch of interesting blogs you might not have seen before.

One anecdote: That evening, at dinner, when I told the family I was going out to meet some bloggers, the 16-year-old looked at me with a “could my Dad be any more lame?” expression. Naturally, I took that as a challenge and printed out a graph of my blog traffic to take along. Not only was the 16-year-old forced to acknowledge that his Dad was the uber-lamer, but my fellow bloggers found the graph fascinating, and Suki even took a picture. Oh well, I haven’t hit rock bottom yet—that comes when you post pictures of your pets on your blog.

Our dog has developed a fear of traffic. Since we live on a busy street, this is a problem. It all started when he was crossing the street with Carmen and a car went through the red light without even slowing, passing less than a metre in front of him and smashing into the side of another car just a couple of metres away. After that he wouldn’t cross the street at that intersection, which is sort of understandable, but his fear quickly grew so that he would freeze up walking on the footpath beside a busy road. Did I mention that we live on a busy road?

Naturally I searched my blogroll for the solution and found this Brad Delong post—the answer is dog treats. Bribe him with yummy food and he’ll forget about being being afraid. So I buy some Schmackos Bacon Bitz (”Dogs go wacko over Schmackos” it says on the bag.) I try bribing the dog. I let him smell the treat and stand five metres down the street and offer it to him. He comes forward and takes the treat. Success! Then he spits out the treat. Now this dog will eat anything vaguely foodlike and some things that aren’t even food. But he won’t eat these stupid Schmackos treats. They are the first things we have discovered that he won’t eat. If you have a suggestion for a dog treat we could try, please leave a comment.

PS We don’t actually know what breed he is, but the most popular theory is that he is a Great Dane/Bull Mastiff cross.

Chris Bertram points out that a new study suggests that the Lancet’s finding of an increase in infant mortality following the invasion of Iraq is correct. The Washington Post reports:

After the rate of acute malnutrition among children younger than 5 steadily declined to 4 percent two years ago, it shot up to 7.7 percent this year, according to a study conducted by Iraq’s Health Ministry in cooperation with Norway’s Institute for Applied International Studies and the U.N. Development Program….

International aid efforts and the U.N. oil-for-food program helped reduce the ruinous impact of sanctions, and the rate of acute malnutrition among the youngest Iraqis gradually dropped from a peak of 11 percent in 1996 to 4 percent in 2002. But the invasion in March 2003 and the widespread looting in its aftermath severely damaged the basic structures of governance in Iraq, and persistent violence across the country slowed the pace of reconstruction almost to a halt.

Via Juan Cole I find that Rod Nordland and Babak Dehghanpisheh have written:

The leading British medical journal, The Lancet, recently published a study that used interviews and extrapolations to estimate the total figure at 100,000 or more, mostly from aerial bombardment. Other statisticians have since dismissed the study’s conclusions as unreliable and speculative.

No no no! The people who dismissed the study’s conclusions were not statisticians and displayed profound ignorance about statistical methodology.

Update: David Adesnik attacks the malnutrition report, calling the 11% and 4% acute malnutrition figures “pseudo-statistics”. Adesnik offers no criticism of the methodology—the only reason he offers for rejecting the figures is that they were compiled under Saddam. But his reason makes no sense at all. Even if, for the sake of argument, we believe that Saddam could force UNICEF into cooking the statistics, why would Saddam have been artificially lowering the figures? Surely he would have been raisning them so that he could point to the harm that the sanctions were inflicting on Iraqi children.

In an update, Adesnik puts his money on the scenario that the Washington Post is belatedly reporting the results of this study conducted 29 April–3 May 2003 which found that 7.7% of Baghdad children were acutely malnourished. Well, he’s lost his money, because that was a different survey. The Post is reporting the results of a national survey conducted in April/May 2004.

Adesnik also argues that even if there is a new survey the 2003 survey, “conducted less than three weeks after the invasion of Iraq”, proves that “the increase from 4 to 7.7 percent was Saddam’s doing.” Adesnik does not seem to have bothered to read the conclusion of the 2003 study (which was conducted six weeks after the start of the war, not “less than three” as Adesnik claimed):

Seven out of 10 children reported had suffered from diarrhoea at some time during the previous 5 weeks. Diarrhoea is likely a major factor in the rise in malnutrition since the war, linked mainly to the poor quality and low quantity of water, poor sanitation, large amounts of uncollected garbage and frequent electricity cuts.

Adesnik seems to be unaware that a sick child can lose a lot of weight in a few weeks.

Now the 2003 study just covered Baghdad, while the 2004 one was a national one, so the numbers are that comparable, but it does seem that malnutrition got worse after the invasion and things still have not improved.

Update 24/11: David Adesnik has replied. First, he is unconvinced that the Post is reporting the results of the IAIS survey:

Hoping to track down the data, I sent an e-mail to IAIS on Sunday asking for further information about their work. In addition, I spent a considerable amount of term searching for related information on Google and Lexis-Nexis, yet found absolutely nothing.

Of course, it may turn out that IAIS really has done a new survey. But for the moment, there is hardly enough evidence to substantiate Tim’s allegation.

Well, I didn’t spend a considerable amount of time searching, I just looked at the IAIS home page, which says:

Malnutrition in Iraq
In recent days data on malnutrition from the IMIRA survey have been published in Iraq and elsewhere (see Washington Post ). According to our findings 7.5 percent, or 216,000, children between 6 months and 59 months of age at the time of interview are acutely malnourished. Acute malnutrition is more widespread in the south than the north of Iraq. Acute malnutrition is measured by comparing a child’s height and weight to a standard reference population. See the Fafo IMIRA web pages.

I trust that clears things up.

Originally, Adesnik wrote that the 2003 study was conducted “less than three weeks after the invasion of Iraq”. He now says that this was an “ambiguity” and that he was counting from the fall of Baghdad. Forgive me, but “invasion of Iraq” is not an ambiguous way to refer to “fall of Baghdad”. And it makes no sense to count from the fall of Baghdad in any case, since the disruptions to water supplies and so on would have started with the bombing campaign at the start of the war.

Next Adesnik writes:

what I doubt is that 200,000 thousand children can get that sick in the space of a few weeks. Major combat operations were fairly localized and coalition bombing raids did not target civilian infrastructure.

While most Iraqis probably were dependent on official food rationing programs that may have been disrupted during the war, I tend to doubt that such a disruption would translate so immediately into a national epidemic of malnutrition. Of course, that is just speculation—but Tim is only offering more of the same.

I am perplexed by this response. I didn’t offer “just speculation”, but quoted the conclusion of the study, which found that “Seven out of 10 children reported had suffered from diarrhoea at some time during the previous 5 weeks.” Adesnik can doubt that the children got sick, but the doctors who examined them seem to think otherwise.

Adesnik next argues that Saddam was cooking the malnutrition figures and that we don’t know whether he was artificially increasing them or decreasing them because “speculation about Saddam’s motives is futile.” Well, it doesn’t seem likely that Saddam could get Unicef to cook their numbers, but we have independent evidence that malnutrition was lower before the war—the recent Lancet study found that infant mortality was quite a bit lower before the war (and was conducted when Saddam was no longer in a position to influence the results).

Moreover, the infant mortality and malnutrition figures seem to show a consistent pattern and support each other. If you study the table below, you will see that things were very bad in the late 90s, then the oil-for-food program made things a lot better and that things have gotten worse since the war.

Acute Malnutrition Rate	Infant Mortality Rate
1996 11% (1996 MICS)	1999 10.8% (Unicef 1999)
2002 4% (Nutrition survey 2002)	2002 2.9% (Lancet 2004)
2004 7.5% (IMIRA 2004)	2003-2004 5.8% (Lancet 2004)

Update 26/11 Further discussion with Adesnik is here.

One of the arguments made against the Lancet study was that the study had greatly underestimated the pre-war mortality rate, because the study found that it was about 29 per 1000 live births, while UNICEF estimated that it was 108. Now the 108 dates from 1999, but sceptics doubted that it could have declined dramatically by 2002. However, other studies (see table below) show that the incidence of acute malnutrition declined dramatically between the late 90s and 2002, so it seems likely that infant mortality would have done so as well.

Acute Malnutrition Rate	Infant Mortality Rate
1991 3% (GOI-UNICEF 2000c)	1990 4.7% (CMMS)
1996 11% (1996 MICS)	1999 10.8% (Unicef 1999)
2002 4% (Nutrition survey 2002)	2002 2.9% (Lancet 2004)
2004 7.5% (IMIRA 2004)	2003-2004 5.8% (Lancet 2004)

Update: Added figures for situation pre-sanctions to the table.

David Adesnik has replied to my post on malnutrition in Iraq. He has conceded that the Washington Post was reporting the results of a new survey rather than the results of one from 2003. But he is still arguing that the war did not cause the increase in malnutrition seen in the 2003 study:

The question isn’t whether a certain child had some diarrhoea during the invasion, but whether that child started to have diarrhoea (or whether the condition intensified) during that five week period.

If we look at this UNICEF press release (which Adesnik already linked to) we find (my emphasis):

UNICEF says that unsafe water from disrupted water services may be playing a significant role in the findings. Poor water quality is largely to blame for a rapid increase in cases of diarrhoea among children in recent weeks.

Speaking from Baghdad, UNICEF Health and Nutrition Officer Dr. Wisam Al-Timini said that the survey found that more than 1 in 10 children were in need of treatment for dehydration.

“This suggests exactly what we know: Poor water and sanitation leads to diarrhoea, and then to dehydration and malnutrition. These children need treatment to stop their bodies from wasting because of an inability to retain vitamins and nutrients from ordinary foods. Those severely malnourished who do not get treatment are at very high risk of dying.”

Adesnik continues:

In order to show that the invasion was the primary cause of rising malnutrition, one has to show that the preponderance of the children’s severe weight loss took place during the six weeks of major combat operations, rather than the preceding year or so.

But the UNICEF report says:

“If we compare these results with earlier findings, we note that children who have generally grown over the past few years because of improved nutrition have suddenly and dramatically wasted. This coincides with war and the breakdown of social services.

Adesnik then looks for trouble by writing:

On a brighter note for OxBlog, Tim doesn’t seem to challenge my assertion that the similar results of the UNICEF and IAIS studies demonstrate that the malnutrition rate has been essentially stable since the beginning of the occupation. Thus, the WaPo was still very wrong to report that malnutrition “shot up…this year”.

Well, they’re not strictly comparable since one is for Baghdad and one is national, but more importantly, let’s look at what the Post said:

Acute malnutrition among young children in Iraq has nearly doubled since the United States led an invasion of the country 20 months ago, according to surveys by the United Nations, aid agencies and the interim Iraqi government.

After the rate of acute malnutrition among children younger than 5 steadily declined to 4 percent two years ago, it shot up to 7.7 percent this year,

The Post did not say that the increase happened this year as Adesnik seems to believe, but that it happened since the invasion. Clearly, “this year” refers to when the new survey was conducted, not to when all of the increase happened. Point to the Washington Post.

I haven’t commented on Kaplan’s shoddy critique of the Lancet because Daniel Davies already demolished it here. Kaplan did have one argument that Davies did not address, so I will deal with that in this post. Kaplan wrote:

The survey team simply could not visit some of the randomly chosen clusters; the roads were blocked off, in some cases by coalition checkpoints. So the team picked other, more accessible areas that had received similar amounts of damage. But it’s unclear how they made this calculation. In any case, the detour destroyed the survey’s randomness; the results are inherently tainted. In other cases, the team didn’t find enough people in a cluster to interview, so they expanded the survey to an adjoining cluster. Again, at that point, the survey was no longer random, and so the results are suspect.

It’s quite remarkable how many things that Kaplan got wrong in just one paragraph:

1. It is not true that they could not visit some clusters because the roads were blocked off.
2. They did not pick other more accessible regions. To reduce travel, they paired governorates with similar violence levels and move all the clusters in the pair into one of the governorates.
3. It was not unclear how they did this calculation since the report explains it in excruciating detail.
4. The moving was clusters was randomized so that it did not destroy the survey’s randomness.
5. It is not true that they expanded the survey to an adjoining clusters if there weren’t enough people in a cluster. The sampling unit was households, not people. And a cluster was the thirty households that were nearest a randomly chosen location.

The last mistake is particularly egregious. The first four seem to have resulted from a bizarre misreading of the study’s Governorate pairing procedure, but the last one seems to have been made up out of whole cloth.

The latest pundit to attack the Lancet study is somebody called John Lott. He writes:

I haven’t spent a lot of time going through the methodology used in this survey by Lancet, but I don’t know how one could assume that those surveyed couldn’t have lied to create a false impression. After all, some do have a strong political motive.

Well, unlike surveys of defensive guns use, where the people questioned can make anything up that they liked, the researchers tried to verify the deaths with death certificates and were succesful in 81% of the times that they asked.

There is also the question of the comparability of the before and after war fatality rates. Andrew Bolt has a very extensive and interesting critique of the Lancet paper:

As I explained earlier, Bolt’s article contains some basic statistical errors. But Lott seems to be endorsing it. What does that say about Lott’s knowledge of statistics?

Lott also links to this New York Times article, claiming

If the New York Times critiques you (even with caveats) from the right, you know that you are in trouble

Which is pretty weird, since the article defends the Lancet study:

Other critics referred to the findings of the Iraq Body Count project, which has constructed a database of war-related civilian deaths from verified news media reports or official sources like hospitals and morgues.

That database recently placed civilian deaths somewhere between 14,429 and 16,579, the range arising largely from uncertainty about whether some victims were civilians or insurgents. But because of its stringent conditions for including deaths in the database, the project has quite explicitly said, “Our own total is certain to be an underestimate.”