Statistical Significance and U.S. Law

This plaque in honor of William Sealy Gosset aka "Student" - he of Student's test of significance - is now displayed at Student's family home of many years, in Monkstown, Dublin, Ireland.

Articles, essays, and interviews on statistical significance and law, by Stephen T. Ziliak 

 

 

A group of mathematicians has been trying for years to have a core statistical concept debunked.  Now, the Supreme Court might have done it for them.

 – Carl Bialik, Making a Stat Less Significant, The Wall Street   Journal, April 2, 2011, p. A5

Gosset Student drawings of kurtosis

The Unprincipled Randomization Principle_ Ziliak Teather-Posadas_ Oxford (March 29 2014)

  • McCloskey and Ziliak Supreme Court brief of amici curiae on statistical significance, securities law, and adverse event reporting (vol. No. 09-1156, pp. 22).  Washington DC: Supreme Court of the United States.  With Edward Labaton et al. Counsel of Record (Ed.), on adverse effect reporting and statistical significance, Matrixx et. al. v. Siracusano and NECA-IBEW Pension Fund, filed Nov. 12, 2010.

Stephen T ZiliakDeirdre McCloskey in Amsterdam

See also: Tim Harford, Statistical Significance and a U.S. Supreme Court Decision  (Matrixx v. Siracusano), BBC Radio 4, “More or Less” program, interview, April 15, 2011.

Kevin McConway, Statistically Significant: The US Supreme Court Takes a View, OpenLearn, Open University (Milton Keynes, UK) in cooperation with BBC Radio 4’s “More or Less”, hosted by Tim Harford.

Carl Bialik (aka “The Numbers Guy”), “Making a Stat Less Significant,” The Wall Street Journal, April 2, 2011, p. A5.

Carl Bialik, “A Statistical Test Gets Its Closeup,” The Wall Street Journal, The Numbers Guy blog, April 1, 2011.

Tim Harford (aka “The Undercover Economist”),Enlightened Research Fueled By The Dark Stuff,” Financial Times, Feb. 7, 2009.

Tim Harford,Statistical Significance,” BBC Radio 4, “More or Less” program, interview, Feb. 2, 2009

  • Article by Stephen T. Ziliak, “Matrixx v Siracusano and Student v Fisher: Statistical Significance on Trial,” Significance 8 (3, 2011), published by the Royal Statistical Society and the American Statistical Association, pp. 131-134.
"This statistical significance always works and always doesn't work." - U.S. Supreme Court Justice Stephen Breyer, Matrixx v. Siracusano Oral Arguments, January 2011.
“This statistical significance always works and always doesn’t work.” – U.S. Supreme Court Justice Stephen Breyer, Matrixx v. Siracusano Oral Arguments, January 2011.

“The Significance of Statistical Significance” reviews THE CULT OF STATISTICAL SIGNIFICANCE: How the Standard Error Costs Us Jobs, Justice and Lives by Stephen T. Ziliak and Dierdre N. McCloskey. The book’s core message is that statistical significance should not be equated with substantive significance and that empirical researchers should convey more information about the magnitude of relationships and effects than many now do. This review summarizes, approves of and elaborates on Ziliak and McCloskey’s message with special attention to concerns of the legal academy. It clarifies appropriate uses of significance tests within the research framework of controlling for plausible rival hypotheses.”

Stephen T Ziliak statistical significance law

Other legal cases and law review articles discussing the Ziliak-McCloskey research (hat tip to A.L.):

35 Seattle U. L. Rev. 1165
Seattle University Law Review Summer, 2012 NEVADA AND THE MARKET FOR CORPORATE LAW Bruce H. Kobayashi, Larry E. Ribstein

80 Geo. Wash. L. Rev. 505
George Washington Law Review February, 2012 BEYOND IDEOLOGY: AN EMPIRICAL
STUDY OF PARTISANSHIP AND INDEPENDENCE IN THE FEDERAL COURTS Corey
Rayburn Yung

56 St. Louis U. L.J. 363
Saint Louis University Law Journal Winter 2012 DRUG DEVELOPMENT–STUCK IN A
STATE OF PUBERTY?: REGULATORY REFORM OF HUMAN CLINICAL RESEARCH TO RAISE RESPONSIVENESS TO THE REALITY OF HUMAN VARIABILITY Michael J. Malinowski,Grant G. Gautreaux

45 Law & Soc’y Rev. 195
Law and Society Review March, 2011 HAS LEGAL REALISM DAMAGED THE LEGITIMACY OF THE U.S. SUPREME COURT? James L. Gibson, Gregory A. Caldeira

6 Fed. Cts. L. Rev. 1.                                                                                                                Federal Courts Law Review 2011 TWOMBLY AND IQBAL’S MEASURE: AN ASSESSMENT OF THE FEDERAL JUDICIAL CENTER’S STUDY OF MOTIONS TO DISMISS Lonny Hoffman

“[Statistical significance] is not the same as practical importance, the “cult of statistical significance” persists and appears very hard to correct, as Stephen Ziliak and Deirdre McCloskey have recently demonstrated. By emphasizing only whether the differences they observed were statistically significant, the FJC researchers fell into …”

Columbia Law Review October, 2010 AN OFFENSIVE WEAPON?: AN EMPIRICAL ANALYSIS OF THE “SWORD” OF STATE SOVEREIGN IMMUNITY IN STATE-OWNED PATENTS Tejas N. Narechania

19 Wm. & Mary Bill Rts. J. 75
William & Mary Bill of Rights Journal October, 2010 ORAL DISSENTING ON THE SUPREME COURT Christopher W. Schmidt, Carolyn Shapiro

7 Ohio St. J. Crim. L. 307
Ohio State Journal of Criminal Law Fall, 2009 LEGITIMACY AND CRIMINAL JUSTICE: THE BENEFITS OF SELF-REGULATION Tom R. Tyler

50 B.C. L. Rev. 685
Boston College Law Review May, 2009 THE DISTORTING SLANT IN QUANTITATIVE STUDIES OF JUDGING Brian Z. Tamanaha

33 Law & Soc. Inquiry 841
Law and Social Inquiry Summer, 2008 BOOK NOTE

41 San Diego L. Rev. 177
San Diego Law Review February-March 2004 THE ROLE OF SOCIOECONOMICS IN
TEACHING FAMILY LAW Margaret F. Brinig

64-AUT Law & Contemp. Probs. 5
Law and Contemporary Problems Autumn 2001 SCIENTIFIC IGNORANCE AND RELIABLE PATTERNS OF EVIDENCE IN TOXIC TORT CAUSATION: IS THERE A NEED FOR LIABILITY REFORM? Carl F. Cranor [FNa1]David A. Eastmond

53 Indus. & Lab. Rel. Rev. 363
Industrial & Labor Relations Review April, 2000 THE CRAFT OF LABORMETRICS Daniel S. Hamermesh