“A group of mathematicians has been trying for years to have a core statistical concept debunked. Now, the Supreme Court might have done it for them.”
– Carl Bialik, The Wall Street Journal, April 2, 2011, page A5
“There’s not much we can say here about the perils of p-values and significance testing that hasn’t been said already for decades (Ziliak and McCloskey 2008; Hubbard 2016).”
– Ronald L. Wasserstein, Allen L. Schirm, and Nicole A. Lazar (Editorial), Moving to a World Beyond p < 0.05 The American Statistician (2019 73 sup1), page 1
Selected Works on “Statistical Significance”
By Stephen T. Ziliak, 1996-
Ziliak How Large Are Your G Values Gosset’s Guinnessometrics When a Little p Is Not Enough The American Statistician (2019) 73 sup1 (Special Issue on Statistical Inference in the 21st Century: A World Beyond p < 0.05)
[Media press release (April 24, 2019)]
Ziliak Guinnessometrics Economic Foundation Student’s t JEP 2008
Ziliak William-S-Gosset-and-Neglected Concepts in Experimental-Statistics-JWE 2011 (Journal of Wine Economics, Special issue on Beeronomics)
(Comment by Tim Harford, Financial Times 2009)
(Interview with Tim Harford, BBC Radio 4 “More or Less” 2009)
(Comment in Mark Thoma’s Economist’s View 2009)
(Comment by The Economist and Economist’s View 2009)
(Comment by Andrew Leonard, Slate 2009)
(Comment by Ezra Klein, Sarah Kliff, The Washington Post 2012)
(Comment by Whet Moser, Chicago Magazine 2012)
(Comment by The Irish Times 2012)
(Comment by Freakonomics 2012)
(Comment by Dan Kopf, Priceonomics 2015)
McCloskey and Ziliak 1996 The Standard Error of Regressions JEL
Ziliak and McCloskey Size Matters 2004a Journal of Socio-Economics
“A clear trade-off: how much confidence [in a result] is “enough” depends on the costs of further research and the benefits of extra precision. Ziliak and his co-author Deirdre McCloskey argue in The Cult of Statistical Significance that most academic disciplines have forgotten this trade-off . . . A sharp line for statistical significance makes no sense, and it has a cost.”
– Tim Harford, Financial Times (2009)
Size Matters Ziliak and McCloskey 2004 AEA Plenary on Statistical Significance
Ziliak McCloskey Significance Redux 2004b Reply to Granger Zellner Leamer Horowitz Wooldridge et al
(Comment in The Economist, Jan. 2004, “Signifying Nothing?”)
(Comment in Nature, 2008, Statistical significance disputed definition in Nature)
(Comment by Pedro R. Lowenstein, Molecular Therapy 2009)
Statistically significant journey _Ziliak 2017
Ziliak McCloskey 2004a Size Matters article published simultaneously by Econ Journal Watch
Ziliak McCloskey reply to Thomas Schelling 2004 EconJournalWatch
Ziliak McCloskey Abstract of book The Cult of Statistical Significance JSM 2009
(Better yet, order the book)
Advance Praise for The Cult of Statistical Significance (2008)
“McCloskey and Ziliak have been pushing this very elementary, very correct, very important argument through several articles over several years and for reasons I cannot fathom it is still resisted. If it takes a book to get it across, I hope this book will do it. It ought to.”
– Thomas C. Schelling, Distinguished University Professor School of Public Policy, University of Maryland and 2005 Nobel Laureate in Economics
“With humor, insight, piercing logic and a nod to history, Ziliak and McCloskey show how economists–and other scientists–suffer from a mass delusion about statistical analysis. The quest for statistical significance that pervades science today is a deeply flawed substitute for thoughtful analysis. This hollow pursuit, kept alive by mechanical, conformist thinking, has led to grave and obvious errors. Yet few participants in the scientific bureaucracy have been willing to admit what Ziliak and McCloskey make clear: the emperor has no clothes.”
– Kenneth Rothman, Professor of Epidemiology, Boston University Distinguished Fellow, Research Triangle Institute, and founding editor of Epidemiology
Articles, Chapters, Edited Volumes
Associate Editor, Stephen T. Ziliak, The American Statistician (2019) 73 sup1 (Special issue on Scientific Inference in the 21st Century: A World Beyond p < 0.05). 43 articles by leading statisticians and scientists, helping to pave the way forward
Contributing author and ASA P-values Committee Member, Stephen T. Ziliak, The American Statistician 70 (2016) “Statement on Statistical Significance and P-values,” American Statistical Association (ASA), Alexandria, VA
Ziliak Significance of ASA Statement on P-values Stat Sig 2016 The American Statistician 70, supplemental
(Comment in FiveThirtyEight, 2016)
(Ziliak interview, Tim Harford, BBC Radio 4 “More or Less”, 2009)
(Comment by Tim Harford, BBC More or Less and Financial Times, 2011)
(Comment by Regina Nuzzo, Nature, 2016)
Statistical Significance Ziliak McCloskey Rejoinder to Mayer EJW 2013
Signifying nothing reply to Hoover and Siegler 2008 (Journal of Economic Methodology 2008)
(Comment by Andrew Gelman, Statistical Modeling blog 2007)
Science is Judgement reply to Aris Spanos _ Ziliak McCloskey 2009 (Erasmus Journal of Economics and Philosophy)
McCloskeyZiliakSept2012 Reply to Old Tom New Tom Mayer
McCloskey and Ziliak Supreme Court brief Matrixx v Siracusano 2011
(Comment in Wall Street Journal, Supreme Court Makes a Math Case – Carl Bialik aka The Numbers Guy 2011)
(Comment in Wall Street Journal, A Statistical Test Gets Its Close Up, by Carl Bialik aka The Numbers Guy 2011)
Teaching Graduate Econometrics Jnl Economic Education 2019 McCloskey Ziliak (Symposium on Teaching Econometrics, Journal of Economic Education (2019, forthcoming)
Statistical significance Lipitor and Type 2 Diabetes by Cranor McCloskey Ziliak_amici curiae brief 2017 (U.S. Court of Appeals, Fourth Circuit)
US Supreme Court Finds Statistical Significance Not Necessary for Causation_Matrixx v Siracusano Student v Fisher_Ziliak PDF (Joint Statistical Meetings Late-Breaking Session with Joseph Kadane, Danny Kaplan, Donald Rubin, organized by Milo Schield, Miami Beach 2011).
This plaque is in honor of William Sealy Gosset aka “Student” – he of “Student’s test of statistical significance”, at Gosset’s home for many years in Monkstown, Dublin, Ireland
Levitt List _Balanced vs Randomized Field Experiments Ziliak Rev Behavioral Economics 2014
Ziliak The Lancet 2010 Validus Medicus and a New Gold Standard
Significant errors Lancet Ziliak reply to Stephen Senn 2010
The Unreasonable Ineffectiveness of Fisherian Tests – Biological Theory 2009
Ziliak and Teather-Posadas_Unprincipled Randomization Econ Med 2016
(Comment by Casey Mulligan, The New York Times 2014)
(Comment by Tim Harford, Financial Times 2014)
(Comment by Jed Friedman, World Bank Development Impact Blog 2014)
Ziliak and McCloskey Oxford essay Supreme Court and Ethics of Oomphless Science 2016 (Oxford Handbook of Professional Economic Ethics, Oxford University Press, Eds. George DeMartino and Deirdre N. McCloskey)
Selected Reviews of The Cult of Statistical Significance
The Cult review in Science by Theodore Porter 2008
The Cult of Stat Sig JEL June 09 by Saul Hymans Review
The Cult of Stat Sig _ Monthly Labor Review- Bergman
Review of The Cult in Nature by Jessica Ancker 2009
The Cult of Stat Sig _ Nature Medicine Advert Ancker Review
The Cult of Stat Sig review Notices American Mathematical Society _ Haggstrom
The Cult of Stat Sig review by Philip Coehlo EH.net (Economic History Association 2008)
Comment on The Cult of Statistical Significance by Timothy Wojan et al, Applied Economic Perspectives and Policy
“The book is a model of scholarship, transparent in its method, wide-reaching in its disciplinary expertise, and highly literate, including occasional haiku poems and humor such as, ‘If the variable doesn’t fit/you may not have to acquit.’ The authors convincingly argue that environmental quality, jobs, and even lives are at stake.”
—M. H. Maier, Choice
“The Cult of Statistical Significance has virtues that extend beyond its core message. It is clearly written and should be accessible to those who have neither formal training in statistics nor a desire to secure any. It is full of examples that illustrate why it is the strength of relationships and not their statistical significance that mainly matters.”
—Richard Lempert, Law and Social Inquiry
“Despite appearing to be a book of limited appeal – it is after all a book that looks at a set of statistical techniques – it is one that has immense social implications. We live in an age where ideologies have largely been cast aside and instead we are governed increasingly by a class of politicians and civil servants who aim for ‘evidence-based’ policy-making. When that evidence is based on statistically significant results that ignore any quantification of results then we all have reason to pay attention.”
—London Book Review
“Nevertheless, the message remains: Even employees of major U.S. statistical agencies might take statistical significance for granted. After all, we and other statistical practitioners and data disseminators know all about estimate formulation and sample error. We can analyze data and present our survey results and research findings to the public, providing valuable information about our economy. Relatively few of us, however, know the history of significance analysis, the controversy that surrounds its use, and the “substantive” strength added by considerations of power and other analytical methods.
Cult’s strength is that it fills that void….and then some.” —Bruce Bergman, Bureau of Labor Statistics, New York Office, Monthly Labor Review
