Errors in Proofs

Errors in Proofs —
A Correction and Further Data

Leslie Lamport

Last modified 22 October 2019
In a few talks I've given over the past 10 years, I said that there was evidence to suggest that about 1/3 of all published, refereed math papers contain incorrect theorems.  This was wrong.  I should have said that they contained incorrect proofs or theorems.  I am very sorry about this mistake. 

I have never before documented what evidence I have for my assertion.  Since I don't know of any other data on how common errors are in the mathematics literature, I am explaining here what that evidence is.

George Bergman is a mathematician who was at the University of California at Berkeley before he retired.  He wrote many reviews for Mathematical Reviews, a publication that contains reviews of published mathematical articles.  Bergman read the papers he reviewed carefully, which led him to find errors in many of them.

At one time, he showed me the 51 reviews he had by then written in Math Reviews.  Based on those reviews, I judged that exactly 1/3 of the papers he reviewed contained an error, which I define to mean an incorrect statement in a proof or result that the author believed to be correct.  I published this result in the paper How to Write a Proof that I wrote in 1993.

I recently learned that all the reviews in Math Reviews are available on-line (by subscription) from the American Mathematical Society's MathSciNet service.  I found that Bergman has written a total of 84 reviews.  In what I believe were the original 51 reviews, I again judged that 17 of them reported errors.  (I found one review to be borderline and couldn't decide if a mistake it reported was an error.  I classified it as not an error in order to obtain the same number of errors as before.)  Eight of those 17 reviews reported incorrect results.  In two of those eight, theorems were incorrect because they were based on results from two of the other papers Bergman had found to be incorrect.  I also found that two of the error-free papers should not have been counted, since they just corrected errors reported in Bergman's earlier reviews. 

The statistics for all 84 of the reviews were quite similar to those for the original 51.  The 28 reviews reporting errors are an almost identical fraction of the total number of reviews.  The 11 reviews reporting incorrect results is a reduction in the ratio of incorrect results to total errors from 47% to 39%.

These numbers should not be considered accurate for the entire mathematics literature.  They were all in Bergman's area of expertise. (He is an algebraist specializing in ring theory.)  However, I see no reason to expect the statistics for other fields of math to be significantly different.

These statistics are questionable for another reason. Whether a mistake in a paper meets my definition of an error, rather than just being a typo, depends on what the author believed.  These statistics therefore rest on a subjective judgement.  My belief that they are significant is enhanced by the consistency in the classifications of the first 51 reviews I made about 25 years apart.  It would be useful to have an independent classification of the reviews, and I encourage others to provide one.  The reviews can be obtained by searching Math Reviews for reviewer “Bergman,G” on MathSciNet.