More Deeply Flawed “Stand Your Ground” Research from JAMA
It seems that Dr. David K. Humphreys was never taught the first rule of finding yourself in a hole: stop digging. Dr. Humphreys has chosen to follow up his deeply flawed November 2016 "science" paper in the Journal of the American Association: Internal Medicine on Florida's "Stand-Your-Ground" law by publishing a response to criticism of that paper in the form of a "Research Letter" just published in JAMA Internal Medicine. The "Research Letter" is behind a firewall, but can be purchased/rented for $30 if anyone is interested. I caution that it's only about a page-and-a-paragraph in length, however, and interestingly enough JAMA Internal Medicine makes the whole first page plainly accessible by clicking here, so I'm not sure the last paragraph is worth the extra $30. But it's your money.
You may recall that I previously critiqued Humphreys' November 2016 "Stand-Your-Ground" “science” paper published last November, "Evaluating the Impact of Florida’s 'Stand Your Ground' Self-defense Law on Homicide and Suicide by Firearm," (pay-walled), in a piece I wrote at National Review, “What to Make of the New Study of Florida’s ‘Stand Your Ground’ Law” (free).
The gist of my criticism is accurately recounted by Humphreys in his purportedly responsive "Research Letter":
Results from our study of the influence of Florida’s “Stand Your Ground” self-defense law on homicide and suicide by firearm1 have been questioned for not distinguishing between “unlawful” homicide (ie, murder) and “justifiable” homicide (ie, lawful use of lethal force).2,3 ... Readers have suggested that if the increase in homicide rates resulted from an increase in homicides that were justifiable, the law may be working as intended. We investigated this possibility by acquiring additional data and conducting new analyses.
In fact, nowhere in Dr. Humphrey's "Research Letter" does he cite any actual "additional data" that supports any such "new analysis."
As an interesting aside, footnote  cited above is purportedly a reference to a previous post I had published in National Review on Dr. Humphrey's earlier deeply flawed "Stand-Your-Ground" paper.
- Branca A. What to make of the new study of Florida’s “stand your ground” law. Natl Rev. http://inresco.org/firearmsf/jama_junk_science.html. Published November 16, 2016. Accessed June 26, 2017.
It is notable however, that even the simple act of accurately citing others' work apparently falls outside of Dr. Humphreys' capabilities. The link Dr. Humphreys' provides in footnote  is not to my post on National Review of the indicated title, but to a favorable third-party review of my post: “More JAMA Flim-Flam: Flawed Study Takes Aim at "Stand Your Ground.”
In the very next paragraph, entitled "Methods," Dr. Humphreys writes:
We obtained monthly counts of justifiable homicides, broadly defined as the killing of an assailant or intruder, during the commission of a criminal act, by a civilian, from the Florida Department of Law Enforcement.4,5
No kidding: the Florida Department of Law Enforcement. Impressive.
Upon further examination, however, we find this claim to be misleading at best and outright false at worst. Specifically, footnotes  and  do not reference the Florida Department of Law Enforcement at all, but rather reference the FBI and the Violence Policy Center:
4. Federal Bureau of Investigation. Uniform Crime Reporting Handbook. Washington, DC: US Dept of Justice; 2004.
5. Violence Policy Center. Firearm Justifiable Homicides and Non-Fatal Self-defense Gun Use: An Analysis of Federal Bureau of Investigation and National Crime Victimisation Survey Data. Washington, DC: Violence Policy Center; 2015. http://www.vpc.org/studies/justifiable15.pdf. Accessed July 26, 2016.
So, no actual reference to the Florida Department of Law Enforcement, much as the footnote referencing my National Review piece didn’t actually link to my National Review piece.
Let's continue by taking a closer look at the second of those two footnotes. Footnote  cites a research report from the Violence Policy Center. Is this an unbiased source of data on this subject? Hardly. This outfit is a well-known anti-Second Amendment organization, with an explicit anti-"gun violence" bias, that advocates for increased restrictions on the ownership of firearms by law-abiding Americans and against the very Stand-Your-Ground policy under “study” by Dr. Humphreys. Yet Dr. Humphreys shares none of this relevant context with us.
Furthermore, although this is the only cited data source in Dr. Humphrey’s entire “Research Letter,” the Violence Policy Center does not produce it’s own crime statistics data. Rather, the cited report merely repackages the actual crime data from the FBI Uniform Crime Reports. But the primary-source FBI UCR is itself readily and publicly accessible via their web site, with preliminary data through the first half of 2016, whereas the secondary-source Violence Policy Center study reports on data only through 2012. Why would Dr. Humphrey link to an interpretation of the UCR data from a biased secondary source limited to 2012 rather than make use of the primary UCR data itself through 2016?
Even if we were to pretend that the Violence Prevention Center study’s data was unbiased and valid, we discover something remarkable when we look to the Florida justifiable homicide data—the very subject of Dr. Humphreys’ research--provided in that report: there isn’t any. Literally.
When one looks at “TABLE ONE: FIREARM JUSTIFIABLE HOMICIDES BY STATE, 2008-2012” found on page 8 of the report, we see the states listed vertically and beside each state are columns listing the purported justified homicides for each state for the years 2008 to 2012 in a series of columns.
For the state of Florida, every column—all of them—is labeled “N/A.”
Further, footnote five in the Violence Prevention Center report states: “5 The state of Florida did not submit any data to the FBI Supplementary Homicide Report (SHR) for the years 2008 through 2012,” the span of years covered by the report. So, the only cited source of even secondary data for Dr. Humphreys’ newest Florida justifiable homicide data analysis for this “Research Letter” literally contains no Florida justifiable homicide data. None.
I would also note in passing that even if the Violence Prevention Center’s report did have data for Florida homicides, which it does not, the report covers only the period from 2008 through 2012, and thus would be useless in any analysis that purports to compare pre-2005 homicides with post-2005 homicides.
Let's now take a look at footnote , or more particularly the citation to the FBI UCR Handbook. Note that title carefully: FBI UCR Handbook. This is not the actual data-containing FBI Uniform Crime Report. Rather, this is the handbook intended to guide police departments in correctly gathering and reporting crime data to the FBI UCR. It is literally a procedural instructional manual.
There is not a single piece of crime data contained in the referenced document. As a result, as was the case with the Violence Policy Center report, it could not possibly have been the source for any of the purported data used in Dr. Humphreys purported statistical analysis. So why is it cited? Perhaps because it provides the working definition of justifiable homicide used by the FBI UCR.
Sure enough, when we look at pages 17-18 of the FBI UCR Handbook (2004) we find a discussion of what constitutes justifiable and non-justifiable homicide. Specifically, the Handbook provides a helpful illustration of a scenario that it says should not be reported as a justifiable homicide:
"The following scenario illustrates an incident known to law enforcement that reporting agencies would not consider Justifiable Homicide:
- While playing cards, two men got into an argument. The first man attacked the second with a broken bottle. The second man pulled a gun and killed his attacker. The police arrested the shooter; he claimed self-defense."
With all respect to the FBI, this is simply mistaken. If you are having a mere verbal argument with somebody, and they decide to escalate matters to the point of attacking you with a broken bottle, they have become a deadly force aggressor against whom you would be lawfully entitled to use deadly-force in self-defense—including shooting and killing that attacker. Contrary to what the UCR states, this scenario would constitute a justifiable homicide, and it would do so in every state in the country.
In other words, the FBI UCR Handbook of 2004 apparently does not understand what does and does not constitute a justifiable homicide, and thus the data gathered on justifiable homicide using this handbook as the procedural standard is necessarily worthless. By extension, to the extent Dr. Humphreys relied on this mistaken definition of justifiable homicide, he also does not understand what does and does not constitute a justifiable homicide, and thus his analysis based on data gathered using the FBI UCR Handbook definition is patently worthless.
Even if we set aside that profound misunderstanding of what constitutes justifiable homicide, however, there is another inherent characteristic of the FBI UCR that makes it almost useless for assessing justifiable homicides. That is the fact that the UCR is, as the name suggests, based on crime reports, and not on any final determination of whether a homicide was justified or not. This is a difference that matters, if reality matters at all, especially in the context of justifiable homicide.
Just because a police officer at a homicide scene believed that a use-of-force may not have been justified doesn’t mean that’s the final word on the matter. Indeed, it is very common for a defendant charged with a crime of violence—and thus by definition perceived by the arresting officer as having acted unlawfully—to later have the charges against him dropped by prosecutors, or to be acquitted at trial, precisely on the basis of lawful justification. In other words, the officer’s crime report of non-justifiable homicide was ultimately determined to be mistaken.
To illustrate: If I am attacked by a man in a parking lot, and I shoot and kill him, it’s very likely that I’m going to be arrested by police when they arrive in response to my 911 call, regardless of the merits of my self-defense claim. Many departments do this as simple policy, and so as not to task their officers with making a final legal determination of self-defense at the scene. There’s been a shooting, I concede it was me who did the shooting, that’s probable cause for an arrest—let the lawyers handle the self-defense issues at their leisure, is how the thinking goes.
In my hypothetical case, sure enough, when the prosecutors look at the investigative report it seems a plain case of lawful self-defense, and the charges against me are dropped. Or, if I’m unlucky, I’m compelled to go to trial and get acquitted. In both of those cases, the final determination of my killing of that other person is that it was a justifiable homicide. How does the UCR reflect that final adjudication? It doesn’t. It goes by the initial crime report, and thus my use-of-force is incorrectly recorded by the UCR as an unjustified killing.
That’s not even considering whether a typical patrol officer tasked with the decision of making an arrest has the training and expertise to make a legal determination of lawful self-defense in the immediate aftermath of a use-of-force event and before the totality of the circumstances and evidence are likely to be known.
This challenge of relying on such sourcing of data as determinative was noted by the British economist Sir Josiah Stamp, in an observation that has since become known as “Stamp’s Law”:
The government are very keen on amassing statistics. They collect them, add them, raise them to the nth power, take the cube root and prepare wonderful diagrams. But you must never forget that every one of these figures comes in the first instance from the village watchman, who just puts down what he damn pleases.
Further, the FBI UCR data on homicides in Florida in particular is deeply defective for the years relevant to Dr. Humphreys’ “research.” Even beyond the utter absence of relevant data for Florida from 2008 to 2012, as already discussed above, economists out of George Mason University have found that Florida homicide data reporting to the UCR was either entirely unavailable or severely underreported from 1996 to 2005—in other words, for the full tens years of the pre-SYG period being compared by Dr. Humphreys to the post-SYG period.
One wonders exactly what kind of peer review JAMA Internal Medicine conducted on this “Research Letter,” that a small-town lawyer in Colorado could so readily expose the deep flaws present in Dr. Humphrey’s research.
For shame, JAMA.