Citing Recent Court Cases, Professor Praises U-M Press Title
In a recent blog post considering high court decisions in the United States and Britain, Michael Smithson, Professor of Psychology and Decision Sciences at Australia National University, called Stephen T. Ziliak and Deidre N. McCloskey's Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives "a swinging demolition of the unquestioned application of statistical significance in a variety of domains." Like Ziliak and McCloskey, Dr. Smithson notes that there are meaningful applications of statistical significance, but context is important--what is being measured, what is the sample size, and what a non-significance finding might actually mean are all important factors. To illustrate this, Smithson gives an unusual example:
For a quick intuitive, if fanciful, example let’s imagine randomly sampling one person from the world’s population and our hypothesis is that s/he will be Australian. On randomly selecting our person, all that we know about her initially is that she speaks English.
There are about 750 million first-or second-language English speakers world-wide, and about 23 million Australians. Of the 23 million Australians, about 21 million of them fit the first- or second-language English description. Given that our person speaks English, how likely is it that we’ve found an Australian? The probability that we’ve found an Australian given that we’ve picked an English-speaker is 21/750 = .03. So there goes our hypothesis. However, had we picked an Australian (i.e., given that our hypothesis were true), the probability that s/he speaks English is 21/23 = .91.
In the case of drug manufacturers involved in the Supreme Court case Smithson cites (and for which Ziliak contributed an amicus brief), it is also essential to take into account the severity of side effects, whether or not they occur with a frequency that is statistically significant.
