A PERSONAL HISTORY OF BAYESIAN STATISTICS
 
Thomas Hoskyns Leonard
 
Retired Professor of Statistics, Universities of Wisconsin-Madison and Edinburgh
 
To Leighanne  (1989-2014)
In sadness and admiration
 
ABSTRACT
 

The history of Bayesian Statistics is traced, from a personal perspective, through various strands and via its re-genesis during the 1960s to the current day. Emphasis is placed on broad-sense Bayesian methodology which can be used to meaningfully analyse observed data sets. Over 750 people in Science, Medicine and Socio-Economics, who have influenced the evolution of the Bayesian approach into the powerful paradigm that it is today, are highlighted. The Frequentist/Bayesian controversy is addressed, together with the ways in which many Bayesians combine the two ideologies as a Bayes/non-Bayes compromise e.g. when drawing inferences about unknown parameters or when investigating the choice of sampling model in relation to its real-life background. A number of fundamental issues are discussed and critically examined, and some elementary explanations for non-technical readers and some personal reminiscences are included. Some of the Bayesian contributions of the 21 st. century are subjected to more detailed critique, so that readers may learn more about the quality and relevance of the ongoing research. A recent resolution of Lindley’s paradox by Baskurt and Evans is reported. The Axioms of Subjective Probability are reassessed, some state-of-the-art alternatives to Leonard Savage’s Axioms of Utility are discussed, and Deborah Mayo and Michael Evan’s refutation of Allan Birnbaum’s 1962 justification of the Likelihood Principle in terms of the Sufficiency and Conditionality Principles is addressed.

 

KEYWORDS: History of Bayesian Statistics, Thomas Bayes, Bayesian Inference, Bayesian model choice, Bayesian methodology, Frequentist/Bayesian controversy, Bayes/non-Bayes compromise,  Bayesian model choice, Lindley’s Paradox, Marginalization paradox,  Baskurt-Evans, axioms of subjective probability, Leonard Savage. Axioms of Utility. Michael Evans, Birnbaum’s proof, likelihood principle, sufficiency principle, conditionality principle, Dennis Lindley, subjective probability, statistical probability, inverse probability, Bayes factor, hierarchical Bayesian, empirical Bayesian, random effects models, spatial processes, multiple time series,  MCMC, Metropolis-Hastings algorithm, Laplacian approximation, Daniel Bernoulli, Richard Price, Pierre-Simon de Laplace, Occam’s razor, expected utility hypothesis, Allais Paradox, decision theory, De Finetti axioms, Savage axioms, improper prior, Jeffreys prior, reference prior, Kalman filter, conjugate analysis, M-group regression, Bayesian analysis of categorical data, Bayesian semi-parametric methods, roughness penalty, Dirichlet prior, inverted Wishart prior,  Importance Sampling, smoothing splines, cross-validation, AIC,BIC,DIC. statistical model checking and comparison, Bayesian Econometrics, stochastic volatility, measure of evidence, Bayesian genomics, legal statistics, Sally Clark case, DNA evidence, Alfred Dreyfus case, O.J. Simpson case, Adams rape case, Nurse Atkinson case, the Rosie and the ten construction workers case, Essen-Möller formula.

 

Address for correspondence: Thomas Leonard, 4/3 Hopetoun Crescent, Edinburgh EH7 4AY, Scotland

E-mail: leonardthomas70@googlemail.com,   website: www.thomashoskynsleonard.co.uk

 

ACKNOWLEDGEMENTS

 

Many thanks to Michael Evans, Angelika van der Linde, Peter Wakker, Tony O’Hagan, Fabrizio Ruggeri, Jim Smith, David Scott, Timothy Wallstrom, Murray Aitkin, David Balding, and John S.J.Hsu for their detailed advice and valuable contributions, and to Jim Dickey, Irwin Guttman, Nick Polson, Ming-Mei Wang and George E.P. Box for their various academic accounts from the past.

 

   My former Ph.D. supervisor Dennis V. Lindley sadly passed away in Minehead, Somerset on 14th December 2013 shortly after this history was completed. I enthusiastically acknowledge the wonderful way he and Phil Dawid educated me at University College London, together with his unique insights about our enigmatic discipline.

 

In Memorium, Dennis V. Lindley (1923-2013)

 

   Thanks also to James Carter, Scott Forster, Adam Trettel, Felix Waldmann, Cassie Strickland, Richard A. Johnson, Daniel Gianola, Stephen Fienberg, Grace Wahba, Norman Draper, Jim Albert, Kam-Wah Tsui, Michael Hamada, Julian Stander, Andrew Lawson, Richard Burton, Philip Brown, George Stretfaris, Colin Aitken, Philip Prescott, Orestis Papasouliotis, Matthew Stephens, Thomas Tallis, Deborah Mayo, Neil Chue Hong, Colin Reid, Dee Raspin, Adam Logan and various members of the Edinburgh All-Comers Writers Club for their helpful comments and input, the Celtic scholar Eystein Thanisch for translating a quote in medieval Latin by William of Ockham, and Ernest Gujer of Zurich for his helpful advice about the Swiss mathematician Daniel Bernoulli. The unexpectedly dramatic events surrounding my consultations with my colleagues have themselves become part of the history of Bayesian Statistics.

 

CONTENTS

 

THE FIRST FOUR ITEMS ARE PUBLISHED IN THE JOHN WILEY ON-LINE LIBRARY

 
Please click on WILEY INTERDISCIPLINARY REVIEWS to access these chapters.
 
Chapter 1: Introduction
Chapter 2: Origins
Chapter 3: The Glorious Sixties
Chapter 4: The Pragmatic Seventies  (First Few Pages)
   

THE REMAINDER OF THIS HISTORY IS PUBLISHED ON THIS WEBSITE

Please click on heading to access these chapters
Chapter 4: The Pragmatic Seventies  (Continued)
Chapter 5: The Enigmatic Eighties
Chapter 6: To the Stars in the Nineties
Chapter 7: Towards a Bayesian-Fisherian Twenty-First Century
Chapter 8: Overview
Appendix A: Bayesian Methods for the Simultaneous Estimation of Several Parameters
Appendix B: The Bayesians of Auld Edinburgh (Poem)
Appendix C The Moral High Ground (Poem)
Appendix D Wherefor Dennis? (Poem)
References  
Correspondence and Alternative Viewpoints
   

Please note:

 

Chapter 2 includes a resolution of LINDLEY’S PARADOX.

Chapter 4 includes an  apparent resolution of the Stone-Dawid-Zidek MARGINALIZATION PARADOX controversy.

Chapter 5 includes a re-evaluation of the AXIOMS OF SUBJECTIVE PROBABILITY and a description of the state-of-the-art AXIOMS OF UTILITY.

Chapter 7 includes a description of Michael Evans refutation of the 1962 proof of BIRNBAUM’S THEOREM, a discussion of Sara Martino and Håvard Rue’s INLA PACKAGE, a critique of the landmark Donnelly-Stephens-Balding BAYESIAN GENOMICS methodology, and a RESPONSE to some of the issues raised in Dennis Lindley's 2013 Youtube interview by Tony O'Hagan.

 

IMPORTANT NOTICE:  Every effort has been taken by the author to ensure that this history is accurate from both an historical and a scientific perspective, and fairly stated. If you think that any part of the website portion of the manuscript is inaccurately or unfairly stated, then please contact the author on leonardthomas70@googlemail.com, and he will make every effort to correct the manuscript. He will also consider publishing non-technical contributions regarding his Personal History in the Correspondence and Alternative Viewpoints Section.

 

Tom's literary advisor, Eystein Thanisch

 
 
 
 
  © Thomas Hoskyns Leonard, 2014