Deltoid

gzuckier explains, in detail, what is wrong with Lott’s criticism of Kellermann.

For some reason, Kellermann’s work seems to provoke badly flawed criticism. In another posting gzuckier demolishes three other critiques. In an earlier posting I noted that a critique by Kopel and Reynolds got all its facts wrong. And in a AEI event promoting Lott’s new book, Carl Moody claimed:

The second cut is, as you say, is the data available to other researchers [inaudible], and the answer is no for Kellermann, so I think he’s lying. He’s refused repeated requests for his data. So, no, I don’t trust him and I don’t think anybody here in this room should trust him, or anywhere else, for that matter.

I pointed out to Moody that Kellermann’s data was available for download from the ICPSR. Moody had his slander of Kellermann edited out of the transcript. I suggested that he should publicly apologize to Kellermann. He initially said that he would be happy to do so, but reneged, saying that he thought Kellermann would prefer not to get an apology (false, in fact).

Both Roger Ailes and Tom Spencer found the following exchange from Hardball rather bizarre:

MATTHEWS: It could happen—teacher is at the blackboard chalking up some math lesson, kid runs up real fast, grabs it out of her drawer, he knows it’s there.

LOTT: It’s not going to be in her drawer

MATTHEWS: Where would it be?

LOTT: It would be on her in some way.

LEAR: How does a teacher in a summer dress carry a gun on her?

LOTT: You can carry it inside your thigh. There’s lot of places you can carry it where somebody is not going to see where it is.

After reading my posting yesterday, Alan Schussman was able to get Kellermann’s data in seven seconds and wonders

Evidence. Right there. Data. Available. How do people get away this this crap?

At least Moody admitted that Kellermann released his data. In discussion on Usenet, Lott just switched to the claim that Kellermann hadn’t released all of his data. “What data was missing?” we asked. Lott wouldn’t say.

A deltoid is the concave triangular curve formed when a small circle rolls around the inside of a circle three times as big. Eric Weisstein’s Mathworld has a nice animation as well as a description of its properties.

I use deltoids for the ends of the cartouches in the sidebar and the icon that marks the end of each post.

So why did I call this thing “deltoid”? Well, it’s named after the computer I use to write these postings. How geeky is that? I name all my computers after plane curves (I have another one called ‘cycloid’). My wife insisted on regular names for our children, however. (And she vetoed ‘Amber’ as a name as well.)

If you’ve ever played with Spirograph you might have drawn this curve. Here’s a spirograph applet (courtesy of Anu Garg) that lets you create this curve and other pretty ones:

(more…)

Jesse Taylor has written a letter to the editor about John Lott and also gives his take on John Lott’s suggestion that Iraqi civilians should have more guns. Kaimipono is deeply skeptical. Bill Berkowitz describes it like this:

Perhaps the weirdest bit of advice came from John Lott Jr., the now-discredited resident scholar at the American Enterprise Institute. Lott, who believes the American people would be safer if we all are armed, suggested that trying to force Iraqis to turn in their guns was a mistake.

Also commenting on Lott’s advice was Handgun-Free America Director Chris McGrath. Handgun-Free America have also created a web site critical of Lott.

Science has printed a letter from Lott (subscription required) responding to Science’s editorial suggesting that the AEI should deal with Lott the same way that Emory dealt with Bellesiles:

Donald Kennedy’s editorial “Research fraud and public policy” (18 April, p. 393) alleges that I made up a computer hard disk crash when challenged about the loss of data on a 1997 survey. Unfortunately, Science did not contact me about these allegations. I have provided editors with statements from nine different academics, verifying the hard disk crash. Four of them were coauthors who also lost data with me.

Lott attempts his usual sleight of hand here, pretending that Kennedy alleged that Lott invented the disk crash. Of course, as Lott very well knows, what Kennedy actually suggested was that Lott invented the survey. And while Lott has been able to find nine academics who verify the disk crash, he hasn’t been able to find any academics to verify the existence of the survey.

When the disk crashed on 3 July 1997, I lost all my data for virtually all the research projects that I had conducted up to that point in time, including the text and data files for my book More Guns, Less Crime. With the help of other academics, primarily David Mustard (University of Georgia), I replaced all the massive crime data sets so that academics at dozens of universities could replicate and reexamine every single regression reported in my book. All the additional data have also been supplied for the book’s second edition.

Lott artfully avoids mentioning the additional data for his reply to Ayres and Donohue. The data which contained systematic coding errors. Nor does he mention that when the coding errors are corrected his results go away.

The survey data Kennedy mentions involve merely one number in one sentence in my book, and he fails to note that I later redid the survey on a smaller scale and obtained similar results. Those data have also been released (www.johnlott.org).

The later survey is too small for meaningful results and if you examine the data you will find that the results are not what Lott claims.

Kennedy discusses criticisms that I made of Ian Ayres and John Donohue’s work (only Donohue is mentioned in the Editorial), but fails to note that I have provided them with my different city, county, and state level crime data sets both before and after they refused to provide me with data for their own work. I feel that the comments that I posted about their paper were entirely accurate.

Lott creates the impression that Ayres and Donohue haven’t released their data and that his “Mary Rosh” comments on their work were about this. In fact, their data is available here and what “Rosh” actually said was:

The Ayres and Donohue piece is a joke. I saw it a while ago. Their own county level data that did the year by year breakdown actually showed that Lott and Mustard were correct, but they weren’t smart enough to know it. A friend at the Harvard Law School said that Donohue gave the paper there and he was demolished on this and other points. I haven’t checked their paper again, but do they still have the county level breakdown by year or did they remove it because it was the most general test and it went the wrong way from their perspective? What academic journal are they going to get it publshed in?

Lott feels that it is “entirely accurate” to state that their paper is “a joke” and that Ayres and Donohue “weren’t smart enough” to understand their own results. And he wrote this after their paper and his reply had been accepted by the Stanford Law Review. (I dealt with Lott’s claim that their results supported him earlier.)

In his letter Lott continues:

I used a pseudonym in Internet chat rooms because earlier postings under my own name elicited threatening and obnoxious telephone calls.

This isn’t true either, as I explained before.

My first Blosxom plugin. It creates a menu to let a user choose to see the current page in a different flavour, or the whole weblog in a different flavour. You can see it in action in the sidebar.

Download it here.

Lott was on MSNBC’s Buchanan & Press on May 26. From the transcript:

PRESS:

After that book came out, there was a person who showed up on the Internet by the name of Mary Rosh, who said you were the best professor she ever had in college. She praised the book in her review on the Internet. She said any critics of your book should slink away into a hole and hide. And it turns out this Mary Rosh is a total invention of yours.

Now, why should I believe anything you say in this book if you are lying to people on the Internet?

LOTT:

Well, first of all, not all of those were from me.

PRESS:

She doesn`t exist, does she?

LOTT:

I used a pseudonym in Internet discussions, in chat room discussions.

PRESS:

Doesn’t that just destroy your credibility?

LOTT:

Well, nothing there was false, in the sense that everything I said about guns was correct. And even the other stuff was all based upon some type of truth. I originally used my own name in Internet discussions. But I would get phone calls from people who would be threatening and other types of things.

And, look, we all make silly mistakes from time to time. I admit that I did that.

So when “Mary Rosh” promoted Lott to a chaired professor:

“I had him for a class at the Wharton Business School where he was a chaired professor”,

it wasn’t a lie, but “based upon some type of truth”. And when she wrote:

“There were a group of us students who would try to take any class that he taught. Lott finally had to tell us that it was best for us to try and take classes from other professors more to be exposed to other ways of teaching graduate material”,

that was “based on some kind of truth” as well. And I’d really like to know the kind of “truth” that this:

I am 114 lbs. and 5′6″.

and this:

Even if I am not wearing heels, I don’t think that there are many men that I could outrun

were based on.

And some of Rosh’s statements about guns were incontrovertibly false. For example,

These are not surveys for self defense as you claimed, but surveys involving any type of carrying guns for any reason (e.g., hunting, moving residences, etc.).

And the story about the threatening phone calls doesn’t hold water, as I explained before.

skippy isn’t impressed by Lott’s sleight of hand in proving he had a disk crash instead of offering evidence that he did a survey. skippy is rightly skeptical about the existence of witnesses to the disk crash. There are none. However, we do have plenty of evidence that Lott lost data in 1997 and told lots of people at the time he had lost the data in a disk crash, so I’m prepared to let him have the disk crash.

In the July 14 issue of National Review Michael Potemra has a review of The Bias Against Guns. He writes:

Each of us has a favorite part of the Bill of Rights; for me—as for many others—it’s the First Amendment. But a good rule of thumb is to consider that particular freedom most important which, at a particular time, is most under attack. And that’s why John R. Lott Jr. of the American Enterprise Institute deserves the status of Hero of the Constitution in our time: He stands up for the embattled Second Amendment, the section of the Bill of Rights most hated by today’s smart set.

Lott does not stand up for the Second Amendment. He hardly even mentions it in his book. Lott’s actions have damaged the cause of gun rights. Instead of an argument about what rights the Second Amendment protects, he has made the argument about whether gun controls increase or decrease crime and then lost that argument by using cooked statistics.

Try the following thought experiment. Imagine a fellow who goes on TV and says, “Muslims tend to be violent and creepy,” and another who says, “Gays tend to be violent and creepy.” In both cases, there would be a justifiable explosion of outrage at the proclamation of such unfair and bigoted stereotypes. But now try to imagine a third fellow, who declares that “gun advocates tend to be violent and creepy.” His outburst would probably occasion, at most, a press release from the National Rifle Association; the mainstream media and the public at large would likely see nothing exceptionable in his statement.

Rather than guess at how people might react to such a declaration, I thought I’d see how they did in practice. However, when I searched Google and Factiva, I couldn’t find anyone who’d ever said that or anything similar. The closest examples I could find (via a Google search on “gun creepy”) were a review by Michael Brown, who wrote:

“A good anti-gun propagandist must be ruthless, cold and calculating, like those creepy folks at the Violence Policy Center.”

and a page that described gun control advocates as “creepy-crawly enemies of liberty”. No-one on Potemra’s side of the argument objected at all to this.

Potemra continues:

In his new book, The Bias Against Guns: Why Almost Everything You’ve Heard About Gun Control Is Wrong (Regnery, 349 pp., $27.95), John Lott explains how the defenders of gun ownership have been saddled with this undeserved reputation—and provides the statistical truths that the anti-gun activists don’t want you to know.

Actually, Lott makes statistical claims that are not supported by his data, but pro-gun folks tend to accept these claims uncritically because they suit their agenda.

The picture he paints is quite striking. Gun ownership is an important factor in reducing the crime rate; it makes ours a less, not more, violent society. For example, states with concealed-carry laws have seen large decreases in the number of multiple-victim public shootings; which only makes sense, because a violent criminal intent on a murder spree is more likely to shoot at targets he can confidently assume to be unarmed. This is part of the more general benefit of allowing citizens to engage in defensive gun use. One study found that in the ten states that adopted concealed-carry laws between 1977 and 1992, overall murder rates fell after the laws were passed. Lott’s book is full of information of this kind—which is highly inconvenient for media outlets that want to traffic in gun scaremongering.

Lott’s multiple-victim public shooting study has been contradicted by Duwe et al. Lott does not mention Duwe’s study at all in his book. And his claims about murder rates falling have been contradicted by Ayres and Donohue. Lott misrepresents Ayres and Donohue’s findings in his book.

In response to my comments about Potemra’s review of The Bias Against Guns, Paul Blackman points out that Lott actually mentions the Second Amendment twice in his book. I’ve corrected my earlier posting to say “barely even mentions” instead of “never mentions”.

The first mention is when Lott quotes Benjamin Civiletti:

“The nation can no longer afford to let the gun lobby’s distortion of the Constitution cripple every reasonable attempt to implement an effective national policy towards guns and crime.”

Lott does not dispute Civiletti’s interpretation of the Constitution—he just uses the quote to show that Civiletti supports gun control

The second mention is a paraphrase of Italian defence minister Antonio Martino, who:

“suggested that Italy model its laws after the U.S. Constitution’s Second Amendment, which protects the right of citizens to bear arms.”

Lott only mentions the Second Amendment in passing and only when he is quoting or paraphrasing someone else.

In footnote 40 of his article The Impact of Concealed-Carry Laws (in Evaluating Gun Policy), John Donohue comments on Lott’s claims about multiple-victim public shootings:

In the wake of a recent school shooting in Germany that killed 14, Lott summarized his finding from the Lott and Landes study:  “multiple-victim public shootings fell on average by 78 percent in states that passed [right-to-carry] laws.”  John Lott, “Gun Control Misfires in Europe,” Wall Street Journal, April 30, 2002, A16.  Although the results may at first seem persuasive, there is a major problem with the Lott and Landes data.  The FBI Supplemental Homicide Report (SHR) reveals over 800 such multiple-victim deaths per year while Lott and Landes use a Lexis search that generates only about 20.  While it may be that not all 800 should be included (e.g., Lott and Landes would eliminate some of the murders in the FBI data because they are not committed in public places), the true number of cases is vastly greater than the number that Lott and Landes employ.  Indeed, Lott and Landes have now found that when they use the SHR data, their results “were rarely statistically significant.”  Consequently, if their story doesn’t emerge when they use the best data, why should we believe their results using much less accurate data? 

Whenever a major public shooting occurs, Lott writes an op-ed claiming that concealed-carry laws would largely eliminate them. He never mentions the problems with his data or the contrary research. No doubt he is working on one now in response to the Lockheed shootings. This one occured in a concealed-carry state, but I imagine he’ll be able to come up with something.

I’ve switched from hand-crafted html to using Blosxom for my blog. This lets me add some nice features like grouping postings by topic, an RSS feed, comments, search and so on.

After a couple of hundred postings about John Lott, I also feel like posting on something else for a change, so it’s now Tim Lambert’s weblog rather than a weblog examining Lott’s research. For readers who are just interested in that topic, you need to visit cgi.cse.unsw.edu.au/~lambert/cgi-bin/blog/guns/Lott and you’ll only see the postings on Lott.

The old address (http://www.cse.unsw.edu.au/~lambert/guns/lott98update.html) still works, and will continue to work but it will only contain the postings about Lott.

I’m a computer scientist, not a graphic designer and it probably shows in the design of this page. Believe it or not I actually looked at a lot of designs before deciding that the one I liked best was this one, which happens to be the default design in Blosxom. I did add a couple of small flourishes like the pointy boxes in the sidebar.

Another deliberate design choice is that the text size is not set. I find an annoying number of websites specifying font sizes like 9 pixels—on a high resolution monitor that is really small type. This page should display text in the size you choose with your browser.

This is an annotated list of John Lott’s on line reviews at Amazon and at Barnes and Noble.

(more…)

Lott has an article in the National Review Online criticizing the UN’s efforts to combat the illicit trade in small arms, which, according to the UN, are involved in 300,000 deaths in armed conflicts each year. That seems like a lot to me, but here’s Lott’s take:

“Political scientist Rudy Rummel estimates that the 15 worst regimes during the 20th century killed 151 million of their own citizens. Even assuming that the 300,000-gun-deaths-per-year-in-armed-conflicts figure is accurate, the annual rate of government-sanctioned killing is five times higher.”

So Hitler and Stalin killed more. That hardly makes the 300,000 dead each year insignificant.

Lott goes on to claim:

“In the U.S., the states with the highest gun-ownership rates have by far the lowest violent-crime rates.”

He offers no evidence in support of this claim. Of course, in an op-ed like this there isn’t always enough space to provide supporting cites. Trouble is, he makes exactly the same claim on page 76 of The Bias Against Guns and doesn’t offer any cites or evidence to support his claim there either. Nor does he mention all the studies that have found that states with higher gun-ownership tend to have more homicides.

Lott then claims

“And similarly, over time, states with the largest increases in gun ownership have experienced the biggest drops in violent crime.”

This is not correct—his analysis used data from polls that were totally unsuited for his purpose and were contradicted by all other polls including those he used in subsequent research. More details are here.

Lott continues:

“Research by Jeff Miron at Boston University, examining homicide rates across 44 countries, found that countries with the strictest gun-control laws also tended to have the highest homicide rates.”

Lott does not mention that none of Miron’s correlations between gun-control laws and homicide were statistically significant. (In his book it’s worse—Lott falsely claims that half of the correlations are significant.) Nor does he mention that Miron’s measure of gun laws was badly flawed. Their measure was a simple three point scale with 0 being no laws, 1 being some controls, and 2 a complete ban. On this scale the US and the UK have the same gun laws, and France and Germany have less restrictive laws than the US.

And Lott unaccountably fails to mention all of the studies that have examined the relationship between gun ownership and homicide with international data. The most sophisticated of these, with a vastly better measure of gun ownership and more statistical controls than Miron is by Anthony Hoskin in Justice Quarterly 18:3 pp. 569-592 (2001). Hoskin found:

“Two-stage least squares regression, which controls for homicide’s effect on firearm availability in addition to a number of other confounding factors, reveals a statistically significant positive effect of firearm availability on national homicide rates. The magnitude of the association is considerable. The observed relationship is found to be insensitive to sample composition. Results also indicate that homicide rates do not influence levels of firearm availability.”

That’s more guns, more crime. Don’t expect Lott to admit that this study exists any time soon.

Lott tries to make it look as if a gun ban in Britain caused a crime increase:

“News reports in Britain showed how crimes with guns have risen 40 percent in the four years after handguns were banned in 1997.”

Lott, of course, does not mention that violent crime in Britain has declined significantly since the 1997 law. Now, it is not likely that the 1997 law caused either the increase in gun crimes or the decrease in violent crime, but Lott is cherry picking which facts to present to try to make it look like the law caused crime to increase.

Lott continues:

Passive behavior is much more likely to result in serious injury or death than using a gun to defend oneself.

As I explained earlier, Lott is misrepresenting the results of Kleck’s study, which found that passive behaviour is not more likely to result in serious injury than using a gun, and did not say anything about death.

A reader reminds me of another problem with Lott’s attack on UN gun control efforts that I discussed yesterday. Lott argues that the UN’s regulations would prevent people from obtaining small arms to resist totalitarian regimes. This is rather undercut by the Iraqi experience. Tim Noah observes that Iraqis seemed to have been well armed without ever overthrowing a repressive dictatorship. Also of interest are follow-up postings by Tim Noah and Jim Henley and the ensuing blogspace discussion ably covered by Henley here.

Terry Krepel criticizes WorldNetDaily’s unbalanced coverage of Lott.

Lott has a piece in the Los Angeles Times, which Elton Beard summarizes here.

To me, the most interesting thing Lott writes is this:

But the more than 2 million times each year that Americans use guns defensively are never discussed. In more than 90% of those cases, simply brandishing a weapon is sufficient to cause a criminal to break off an attack.

After claiming on dozens and dozens of occasions that brandishing was sufficient 98% of the time, and finally reducing the number to 95% in April, Lott has now reduced his number to 90%. If he keeps on reducing it like this he may eventually end up with a number like that found by competent surveyors. So why did he reduce the number? It’s possible that he finally noticed that his 2002 survey does not yield 95% he claims. Of course, he’s unlikely to ever admit that there was anything wrong with the his previous 98% and 95% claims so we may never know the reason. And while the 2002 survey does find about 90% brandishing, the sample size is too small for a statistically reliable estimate.

Lott also writes:

Examining all the multiple-victim public shootings in the United States from 1977 to 1999 shows that on average, states that adopt right-to-carry laws experience a 60% drop in the rates at which the attacks occur, and a 78% drop in the rates at which people are killed or injured from such attacks.

Lott is well aware of the paper by Duwe, Kovandzic and Moody that found carry laws had no significant effect on mass public shootings, but he won’t even acknowledge the existence of their paper. Not to mention the other problems with his claims pointed out by Donohue and by me.

Lott continues:

To the extent such attacks still occurred in right-to-carry states, they overwhelmingly take place in so-called “gun-free zones.” Indeed, the attack last week in Meridian, Miss., in which five people were killed took place in a Lockheed Martin plant where employees were forbidden to have guns.

Last Friday I wrote:

Whenever a major public shooting occurs, Lott writes an op-ed claiming that concealed-carry laws would largely eliminate them. He never mentions the problems with his data or the contrary research. No doubt he is working on one now in response to the Lockheed shootings. This one occured in a concealed-carry state, but I imagine he’ll be able to come up with something.

Was I right, or was I right?

Lott continues:

My research also shows that citizens with guns helped stop about a third of the post-1997 public school shootings, stepping in before uniformed police could arrive.

Why does Lott specify uniformed police here? Well, the armed students in the Appalachian Law School shootings were off-duty police. And Lott does not mention that that shooting actually stopped because the shooter was out of ammunition. Nor does he mention that most witnesses said that the shooter put his gun down before the armed police arrived.

Lott also claims:

Annual surveys of crime victims in the United States by the Justice Department show that when confronted by a criminal, people are safest if they have a gun.

No, they don’t show that at all.

A 1998 Lott op-ed called “The Cold, Hard Facts About Guns” has been getting some blogspace attention, with links from Margi Lowry, Joshua Claybourn and pecksnif. pecksnif, apparently unaware of Mary Rosh, even asserts:

unlike Michael A. Bellesiles, who also wrote about guns, Lott is not a known liar and fabricator of facts, nor was he forced to resign his position at the University of Chicago.

Unfortunately, many of Lott’s claims in his op-ed are inaccurate. Since his piece was written in 1998 when debunking his claims I won’t use anything from after 1998. He writes:

Myth No. 1: When one is attacked, passive behavior is the safest approach.

The Department of Justice’s National Crime Victimization Survey reports that the probability of serious injury from an attack is 2.5 times greater for women offering no resistance than for women resisting with a gun. Men also benefit from using a gun, but the benefits are smaller: offering no resistance is 1.4 times more likely to result in serious injury than resisting with a gun.

As I detailed earlier none of the differences he presents are statistically significant and he has no business making such a claim from the NCVS data.

Lott also says:

Myth No. 3: The United States has such a high murder rate because Americans own so many guns.

There is no international evidence backing this up. The Swiss, New Zealanders and Finns all own guns as frequently as Americans, yet in 1995 Switzerland had a murder rate 40 percent lower than Germany’s, and New Zealand had one lower than Australia’s. Finland and Sweden have very different gun ownership rates, but very similar murder rates. Israel, with a higher gun ownership rate than the U.S., has a murder rate 40 percent below Canada’s. When one studies all countries rather than just a select few as is usually done, there is absolutely no relationship between gun ownership and murder.

In fact, a 1993 study by Killias did find such a relationship. (You see an on-line summary of the study here.) Here is the data on homicide and gun ownership from that study:

	homicide	gun ownership
USA	7.6	48
Switzerland	1.2	27
Finland	3.0	23
New Zealand	2.0	22
Australia	1.9	19
Sweden	1.3	15
W.Germany	1.2	9

We see that contrary to Lott’s claims, the Swiss, New Zealanders and Finns all own guns much less frequently then Americans, Switzerland and Germany have the same homicide rate, and New Zealand had the same rate as Australia. Finland has more gun ownership than Sweden and also a higher homicide rate.¹

Lott got pretty well every single fact wrong in his passage.

Lott also writes:

Myth No. 5: The family gun is more likely to kill you or someone you know than to kill in self-defense.

The studies yielding such numbers never actually inquired as to whose gun was used in the killing. Instead, if a household owned a gun and if a person in that household or someone they knew was shot to death while in the home, the gun in the household was blamed. In fact, virtually all the killings in these studies were committed by guns brought in by an intruder. No more than four percent of the gun deaths can be attributed to the homeowner’s gun. The very fact that most people were killed by intruders also surely raises questions about why they owned guns in the first place and whether they had sufficient protection.

This is wrong from beginning to end. It is not true that “virtually all the killings … were committed by guns brought in by an intruder”. If you look at the data (available from the ICPSR, study 6898), you will find that less than 10% of the killings were of this type. And it is not true that “No more than four percent of the gun deaths can be attributed to the homeowner’s gun” In fact, the majority of them can be.

¹ Lott is apparently talking about homicide rates in 1995, while Killias’ data is 1983–86, but the picture does not change if you use homicide rates from 1995. It is possible the discrepency is caused by the use of different data sources, but while Killias tells us where his data comes from (ICVS and WHO), Lott does not

Lott has an op-ed in the Kansas City Star which is recycled from a previous op-ed in the Columbus Dispatch that I commented on here. He has made an interesting change—in earlier versions he wrote:

Other research, by David Olson at Loyola University and Michael Maltz at the University of Illinois, found that when law-abiding citizens carried concealed handguns, criminals were much less likely to carry guns. In fact, they found gun murders fell by 20 percent.

This prompted a rebuke from Michael Maltz, who wrote:

In an effort to promote laws permitting the carrying of concealed weapons, in the April 24th Dispatch John Lott supports his own position by citing research I did with David Olson, purporting to show that these laws reduce homicide. In doing so, Lott bends the truth so much that he breaks its back. Specifically, Lott is well aware of a paper Joseph Targonski and I published in September of last year that points out that the data that we used in that study was problematic and should not be relied upon. The conclusion of our more recent paper is that Lott’s data (and ours) were so error-laden that they cannot be used with any degree of reliability.

So how did Lott change his op-ed in response to this? Does he withdraw the claim? No, he replaces the passage with:

Several studies find that as law-abiding citizens are allowed to defend themselves, criminals are much less likely to carry guns.

This seems to be Lott’s standard practice. After Ayres and Donohue showed that his data contained significant coding errors, Lott did not admit to the errors or correct them, but he did switch to citing Olson and Maltz’s research in his pieces. Now that Maltz has rebuked him for his misrepresentation of Maltz’s work, Lott does not admit to do anything wrong but modifies his claim so that it is so vague that it is difficult to refute.

On July 12 The Columbus Dispatch published a letter from Paul van Doorn replying to an earlier letter from David Mayer that I commented on. Here is an extract (hyperlinks added by me):

Mayer claimed the research of economist John Lott establishes that “violent-crime rates fall after right-to-carry laws are adopted.”

Lott is the darling of the pro-gun movement; he has written two books based upon studies he has conducted on crime rates and gun laws and frequently testifies before legislative bodies that statistical evidence establishes that laws permitting the carrying of concealed weapons result in reduced crime.

Other researchers, most notably scholars John Donohue and Ian Ayres, have conducted their own research and have analyzed Lott’s data, methods and conclusions. Donohue is a Stanford Law School professor and associate dean for research there. Ayres is a Yale Law School professor.

Among their conclusions: Correctly interpreting Lott’s data yields opposite conclusions than those reached by Lott—i.e., concealed-carry laws are actually associated with modest increases in various categories of crime.

Mayer claimed in his letter that Lott’s book The Bias Against Guns refutes Donohue and Ayres. In other words, according to Mayer, Lott’s conclusions are still correct: Letting people carry concealed weapons reduces violent crime.

Mayer omitted the subsequent history of the dispute between Lott and Donohue and others. He likewise omitted other information about Lott that bears directly on his credibility.

For instance, Mayer makes no mention of the fact that Donohue and Ayres recently published a second response to Lott’s work in the Stanford Law Review . They write that published “refutations” of their previous work were nothing of the kind, because they simply ignored the authors’ primary points.

A recent review of The Bias Against Guns, written by Harvard’s David Hemenway, included a partial list of 10 instances in which statements made and conclusions reached by Lott in his latest work are wrong or misleading, citing academic sources and studies that contradict what Lott writes.

At the end of January, Lott admitted to a Washington Post reporter that he had engaged in blatant acts of deception intended to lend credence to his status and work. For three years, in a series of letters, a woman by the name of Mary Rosh defended Lott against critics. She described herself as an academic and a former student of Lott’s.

It turns out Rosh doesn’t exist. Lott used this pseudonym to create a false impression of independent support for his research.

Lott’s “mea culpa” came only after a journalist was unable to track down the phantom Rosh. Lott insists there is nothing wrong with what he did, asserting somewhat perplexingly that the deceptions were necessary in order to present “another point of view.” But if Lott was writing the letters, how could they possibly express views other than his own?

Dispatch readers should know that one of Lott’s most-often cited “findings” may be only a product of his imagination. He claims again and again that in those states where concealed-carry laws are in effect, armed defenders have thwarted criminals 98 percent of the time that they were threatened (more recently he inexplicably revised the figure to 95 percent).

Lott has attributed this statistic to “national surveys,” to other researchers, and even to his own research. But it can be found nowhere in his own published data, nor in any national surveys, nor in the data of anyone else.

I should note that Lott claims that the 95% figure comes from his 2002 survey, although that survey does not yield 95%.

Last week I commented on Lott’s LA Times editorial where he claimed that

Examining all the multiple-victim public shootings in the United States from 1977 to 1999 shows that on average, states that adopt right-to-carry laws experience a 60% drop in the rates at which the attacks occur, and a 78% drop in the rates at which people are killed or injured from such attacks.

I pointed out that he once again failed to mention the results in Duwe, Kovandzic and Moody. Homicide Studies 2002 6:4. Here is the abstract:

Right-to-carry (RTC) laws mandate that concealed weapon permits be granted to qualified applicants. Such laws could reduce the number of mass public shootings as prospective shooters consider the possibility of encountering armed civilians. However, these laws might increase the number of shootings by making it easier for prospective shooters to acquire guns. We evaluate 25 RTC laws using state panel data for 1977 through 1999. We estimate numerous Poisson and negative binomial models and find virtually no support for the hypothesis that the laws increase or reduce the number of mass public shootings.

They did find some evidence that carry laws increase the number of people killed and wounded in mass public shootings, but this was in only one of the models.

Well, in his reply to comments on the LA Times article, Lott finally mentions the Duwe article. He writes:

for a recent paper that gets the same results I do when this more narrow definition is used see Duwe, Kovandzic and Moody, “The Impact of Right-to-Carry Concealed Firearm Laws on Mass Public Shootings,” Homicide Studies, November 2002

That’s right, Lott is claiming that Duwe got “the same results” as Lott. Most people would, I think, reckon that with Lott saying “78% drop” and Duwe saying “virtually no support” for an increase or a decrease, that their results were not the same.

So, what tortured reasoning does Lott use to claim that Duwe got “the same results”? Here is what Lott says:

Whether one looks at two or more people killed or injured as well as three or more people killed, the drops are huge and quite statistically significant. Only when you examine multiple victim public shootings involving four or more people killed is the number of deaths so small that the drops are no longer statistically significant (for a recent paper that gets the same results I do when this more narrow definition is used see Duwe, Kovandzic and Moody, “The Impact of Right-to-Carry Concealed Firearm Laws on Mass Public Shootings,” Homicide Studies, November 2002, for a similar result when examining the few cases where four or more people are killed).

This misrepresents both Lott’s findings and Duwe’s paper to make it look like Duwe’s work supports Lott. First, in the Lott and Landes paper, they write that when they examined shootings with four or more people killed:

The results are similar to our earlier ones. We find that right-to-carry laws reduce the number of deaths, and that these deaths were increasing before passage of the law and falling thereafter.

Duwe et al could not reproduce this result. When they used a Poisson model as Lott did here, they found statistically significant increases. Buried in an endnote on page 302 of The Bias Against Guns Lott admits that the results are not significant if he uses a negative binomial model (instead of Poisson) but insists that the results still show decreases. And while Duwe et al also found that the results were not significant, in some specifications there were increases and some decreases.

Lott’s description of the differences between looking at shootings with two injuries and those with four killed is also misleading. He makes it seem that the only difference is that the sample size is smaller, causing the “huge” drops to stop being statistically significant, but if there were huge drops with the smaller sample, then they would still have been statistically significant. In fact, the drops are smaller and are actually increases in some specifications. Duwe et al also point out that the smaller sample has a big advantage—it is possible to collect ever single such shooting, while the larger sample could be biased by the omission of some cases.

Lott’s description of Duwe as getting “the same results” is reminiscent of the way he has described Ayres and Donohue’s work. Most observers would say that their results contradict his, but Lott insisted that:

Their own county level data that did the year by year breakdown actually showed that Lott and Mustard were correct, but they weren’t smart enough to know it.

Tom Spencer finds Lott’s misrepresentation of Duwe et al hilarious.

The Wyeth Wire takes Lott to task for his completely unsupported claim that Baghdad has fewer murders than Washington DC. Of course, Lott’s defence will be that he was just reporting Donald Rumsfeld’s claims and how was Lott to know that Rumsfeld was no criminologist?

In an interesting parallel to this Daniel Davies post, Randy Barnett, guest blogging at GlennReynolds.com thinks that the Left lives in their own world where the “facts” are different. He writes:

On legal historian e-mail lists to which I subscribe, the Left took forever to abandon Michael Bellesiles (of Arming America disrepute), perhaps because his story fit their world. Or perhaps it was because the worst possible thing is to admit the evil right-wingers are right about anything. I raise the Bellesiles affair not because I think he is typical of the Left, but because of the dogged refusal to admit his story was a fabrication when the evidence of fraud was visible for all to see.

And asks:

I am interested in hearing your thoughts. Have you noticed it too?

Well, funny you should ask.

On a firearms e-mail list to which Randy Barnett subscribes, the Right still has not abandoned John Lott (of mystery survey disrepute), perhaps because his story fits their world. Or perhaps it was because the worst possible thing is to admit the evil left-wingers are right about anything. I raise the Lott affair not because I think he is typical of the Right, but because of the dogged refusal to admit his story is a fabrication when the evidence of fraud is visible for all to see.

Chris Lawrence defends Lott against the charge Wyeth made yesterday. James Joyner also comments.

Lawrence is correct when he points out that Lott’s claims about Baghdad murders are not lies unless Lott knows them to be false, and, in the absence of reliable data we don’t know whether they are false or true. However, what Lott did was write with reckless disregard for the truth. Rumsfeld was actually comparing combat deaths of US soldiers in Baghdad with murders in Washington, DC, so Lott had absolutely no basis for his claim. The important point for someone reading Lott is that any event, you should not believe anything he writes unless you have an independent source for it.

Lawrence also writes:

Indeed. And, that would be a worthwhile critique of Lott’s analysis, which gets to the whole “causal mechanism” thing I discussed above. The best I can say for Lott (if you accept his claims about the dispensation of the survey data, which I find dubious but not entirely improbable) is that he’s a sloppy social scientist—albeit perhaps not an not extraordinarily sloppy one, given the pure sludge that often is passed off as strong evidence in many peer-reviewed journals.

* Lott’s missing data only affects a small part of his overall argument; it may speak to his overall credibility, but the vast majority of his data is available and has been analyzed by other scholars.

And when Ayers and Donohue analysed that data they found systematic coding errors, which, when corrected, caused his results to go away. Lott still will not admit to the errors or deny them.

Good responses by Henry Farrell and Ted Barlow to Randy Barnett’s post that I mentioned yesterday. Saul Cornell (via email) writes:

The comparison between the way historians have responded to the Bellesiles affair and legal scholars have responded to the Lott affair is instructive.  The William and Mary Quarterly  held a round table on Arming America in a timely fashion.  While most historians withheld final judgment until the forum appeared and Bellesiles was given a chance to respond, most historians were convinced by Arming America’s critics and were willing to go on the record to state that his claims about Americans having few guns had been refuted. Historians have also endorsed the Emory report. The double standard being applied to Lott is shocking. Kopel, Reynolds,and Barnett should be leading the charge to have a proper scholarly inquiry into the Lott affair. Given the outrage about Arming America, the failure to demand the same kind of scrutiny for Lott raises the disturbing prospect that this was never about scholarship, but was always about politics. I hope that I am wrong about this. It would be terrible if Kopel, Reynolds, and Barnett would allow their reputations to be tarnished by Lott’s behavior. Failure to follow the example set by historians in the Bellesiles case will discredit them on the gun issue. If they believe Lott is honest what do they have to lose from a serious inquiry? If they do not demand that AEI begin such an inquiry then their earlier claims that this was never just about politics will have been discredited.

Barnett responds to some of the comments he received, and posts a comment from a reader

that merits careful consideration by those who are more skeptical that the phenomenon I was asking about even exists.

Well, it certainly does, since it provides another example of the Right “living a lie”. His reader refers to “researchers such as Kellermann, and Bellesiles behave”. The belief that Kellermann has been somehow discredited like Bellesiles is common among the Right, but has no basis in objective reality. In fact, his most famous result (from the case-control study on gun ownership and homicide) has been replicated by three other independent studies (Wiebe, Cumings et al, and Kleck and Hogan). This fact does not exist in the Right’s fantasy world.

Paul Bruno writes:

John Lott should be below anybody’s standards, but since Lew Rockwell insists on publishing Lott’s work, I cannot in good conscience keep the link to his site.

Roger Ailes is rather unfair to Lott, implying that Lott would deny guns to sixth graders. Actually, in The Bias Against Guns Lott argues that sixth graders should have access to guns for self-defence.

Vladimir Kushnir describes how Symantec’s censorware blocks access to many pro-gun websites such as nra.org, under the category ‘Weapons’. I checked, and he has accurately described which sites are blocked. (Though the Lott site that is blocked is actually a collection of Lott’s articles—Lott’s own site is not blocked.) I agree with Kushnir that most people would not expect ‘Weapons’ (blocked by default) would block access to sites like gunscholar.com.

Kushnir also found that many more pro-gun sites were blocked than pro-control sites. I found that 13 of 48 sites in Google Directory’s Pro-Gun Rights category were blocked by the ‘Weapons’ category, while only 5 of the 78 sites in Google’s so called Anti-Gun Rights category were blocked. There certainly seems to be some bias against pro-gun sites here. Kushnir believes that this

could only be achieved as a result of a conscious effort to block conservative websites, dedicated to the protection of the Constitution, while not blocking the liberal ones.

I don’t think this is correct. They do block some pro-control sites and the pattern observed could be the result of unconscious bias on the part of the employees selecting sites, or that pro-gun sites are more likely to be submitted for blocking.

In any event, I don’t think any of the sites should be blocked (and perhaps the NRA and VPC can agree on something here—they are both blocked). But then, I think the whole idea of censorware is a bad idea.

Update: Guncite (one of the blocked sites) has more information. Symantec’s explanation for the blocking was:

Basically the logic behind gun filtering lists came about after the Columbine school shootings. It was decided that I-Gear would start filtering gun sites that promote gun use so schools can monitor their students in hopes of preventing future school shootings. This is the reasoning behind filtering sites that promote gun “use” vs. guns in general.

Lott responds on his blog to Wyeth’s accusation that he had no evidence for his claim about Baghdad murders. (My earlier comments are here.)

Notice that Lott responds on a minor point, once again ducking the question of the coding errors. And while he links to Wyeth and responds to some of my comments he doesn’t link to me, but pretends my comments were emailed to him. I link to Lott’s comments and have him in my blogroll because I want my readers to see what he says and what I say and make upi their own minds about who is right. Lott doesn’t link here or even let on that this site exists.

Anyway, what evidence does Lott offer that Baghdad has fewer murders than Washington DC? He states:

Baghdad is a city with a population some 8.5 times greater than Washington. While it might be difficult to keep track of the number of property crimes or robberies these days in Iraq, presumably Rumsfeld knows whether the number of murders is greater or less than 200 a month.

Actually, metropolitan Washington has a population close to that of Baghdad and about 30 murders a month (UCR table 6). Lott (as well as Rumsfeld) is inappropriately comparing part of metropolitan Washington with metropolitan Baghdad. Nor is assuming that Rumsfeld has evidence the same as actually having evidence. But more importantly, Rumsfeld never said that Baghdad was experiencing fewer murders than Washington, D.C.. You can see the entire transcript here. Rumsfeld, while responding to this question:

Q: Mr. Secretary, is there any sign in this department of any central control over these dead-enders, as you call them, who have now killed—well, 42 American troops have now died since the president declared major hostilities at an end. Any sign of—

eventually said:

Rumsfeld: Look, you got remember that if Washington D.C. were size of Baghdad, we would be having something like 215 murders a month. And it is—there’s going to be violence in a big city. It s five and a half million people. For the most part, it’s in that area I described. That’s where the active—and it tends not to be, at this stage, random killings. It’s not the kind of rioting you saw on television last night in Michigan, or that type of thing. What you’re seeing instead is what we believe is purposeful attacks against coalition forces as opposed to simply crime and that type of thing.

His “215 murders a month” is in comparison to the 42 US troops dead in a month or so. Nowhere does he say that Baghdad has fewer murders than Washington DC. Now, Rumsfeld is playing fast and loose with his statistics here, since the relevant population is not the size of Baghdad, but the number of US soldiers in Iraq, which is about 1/4 the number of people in Washington DC, and therefore US soldiers in Iraq are considerably more likely to be murdered than DC residents. But he is not saying what Lott claims he is.

Lott has an op-ed in the New York Post where he comments on the recent NYC City Hall shooting. Lott seems to believe that concealed handguns would have prevented the shooting. Lott’s logic is especially hard to follow in this case—the victim was actually carrying a concealed handgun and the killer was shot almost immediately by a police officer. Having more people with concealed handguns present could not have prevented this crime, and might have led to more people getting shot in the general confusion. On the other hand, the policy that Lott dismisses, requiring everyone to pass through metal detectors, could have prevented this shooting.

Lott also claims, yet again:

Examining all the multiple-victim public shootings from 1977 to 1999 shows that, on average, states that adopt right-to-carry laws experience a 60 percent drop in the rate at which the attacks occur, and a 78 percent drop in the rate at which people are killed or injured from such attacks.

As usual, Lott fails to mention the results of the study by Duwe et al, which found no support for the hypothesis that such laws decrease mass public shootings. (He’s only ever mentioned it once, and then to claim that their study “gets the same results I do”.)

Lott continues:

the vast majority of academic research finds that concealed handguns reduce violent crime generally.

This is not true. The table below shows that there are more works that don’t find that they reduce violent crime. Note also that fully half of the ones finding that they reduce crime have Lott as an author, and that Michael Maltz, the only researcher with papers on both sides of the table has repudiated the paper that found that carry laws reduced crime. In any case, not all papers should be weighted equally, and Ayres and Donohue, the most comprehensive paper, found that carry laws tended to increase violent crime.

works that do not find that concealed handguns generally reduce violent crime	works that do find that concealed handguns generally reduce violent crime
Alschuler, Two Guns, Four Guns, Six Guns, More Guns: Does Arming the Public Reduce Crime? Valparaiso University Law Review (1997) link	Bartley and Cohen, The effect of concealed weapons laws—an extreme bound analysis. Economic Inquiry (1998) link
Ayres and Donohue, Shooting Down the More Guns, Less Crime Hypothesis. Stanford Law Review (2003) link	Bronars and Lott, Criminal Deterrence, Geographic Spillovers, and Right-to-Carry Laws American Economic Review (1998) link
Ayres and Donohue, The Latest Misfires in Support of the “More Guns, Less Crime” Hypothesis Stanford Law Review (2003) link	Lott, Plassmann and Whitley, Confirming “More Guns, Less Crime” Stanford Law Review (2003) link
Black and Nagin, Do Right-to-carry Laws Deter Violent Crime? Journal of Legal Studies (1998)link	Olson and Maltz, Right-to-Carry Concealed Weapon Laws and Homicide in Large U.S. Counties: The Effect on Weapon Types, Victim Characteristics, and Victim-Offender Relationships Journal of Law & Economics (2001) link
Rubin and Dezhbakhsh, The Effect of Concealed Handgun Laws on Crime: Beyond the Dummy Variables. International Review of Law and Economics link draft	Plassmann and Tideman, Does the Right to Carry Concealed Handguns Deter Countable Crimes? Only a Count Analysis Can Say Journal of Law & Economics (2001) link
Duggan, More Guns, More Crime. Journal of Political Economy (2001) link	Lott, More Guns, Less Crime
Harrison, Kennison, and Macedon, Crime and Concealed Gun Laws: A Reconsideration unpublished link	Lott and Mustard, Crime, deterrence, and right-to-carry concealed handguns. Journal of Legal Studies (1997) link
Helland and Tabarrok, Using Placebo Laws to Test “More Guns, Less Crime” link (draft)	Moody, Testing for the Effects of Concealed Weapons Laws: Specification Errors and Robustness Journal of Law & Economics (2001) link
Goertzel, Myths of Murder and Multiple Regression. The Skeptical Inquirer (2002) link
Hood and Neeley, Packin’ in the Hood?: Examining Assumptions of Concealed-Handgun Research. Social Science Quarterly (2000) link
Kovandzic and Marvell, Right-to-carry Concealed Handguns and Violent Crime: Crime Control Through Gun Decontrol? Criminology & Public Policy (2003) draft
Ludwig, Concealed-gun-carrying Laws and Violent Crime—Evidence from State Panel Data. International Review of Law and Economics (1998) link
McDowall, Loftin and Wiersema, Easing Concealed-Firearm Laws: Effects on Homicide in Three States Journal of Criminal Law and Criminology (1995) link
Maltz and Targonski,A Note on the use of County-level UCR Data. Journal of Quantitative Criminology (2002) link
Daniel W. Webster, Jon S. Vernick, Jens Ludwig, and Kathleen J. Lester. Flawed Gun Policy Research Could Endanger Public Safety. American Journal of Public Health (1997) link
Zimring and Hawkins,Concealed Handguns: The Counterfeit Deterrent. The Responsive Community (1997) link

Lott also claims:

Despite all the national studies that have been done, there is not a single refereed academic journal publication concluding the opposite.

How clever. Since Ayres and Donohue’s article was published in the Stanford Law Review, which is not refereed, Lott gets to ignore it. Of course earlier in his piece, Lott cites his own research on multiple-victim public shootings which has not been published in a journal, let alone a refereed one, while ignoring Duwe et al’s study which was published in a refereed journal.

Update: Following a discussion with David Friedman I changed the title of the first column. It used to say “concealed handguns do not reduce violent crime”. I changed it because given the way statistical tests work, we can’t prove that concealed guns don’t reduce crime. Getting a “no result” on a statistical test doesn’t prove that the null hypothesis is true. It does mean that you can’t conclude that guns reduce crime.

John Lott informs us about askjohnlott.org but warns us that it is not run by him and is “trying to create a negative impression”.

Wyeth has his own response to Lott’s defence that I dissected earlier.