Under the auspices of the Center For the Study of Rationality, Hebrew University,
the Aumann committee began its work in 1996. By 1998 it had a list of instructions to
be provided to experts who would determine the list of birth and death cities and their Hebrew spellings
to be associated with each of the rabbis of the Gans experiment.
Two sets of instructions were generated. One, called the "Fresh Test," leaves more
discretion to the experts. The other, called the "Replicative Test," follows more closely
to the Gans experiment.
The committee finished its work on July 16, 2002. Both experiments produced non-significant
results. The committee report (with its appendices) is the first document provided.
Aumann Committee Report
Appendix 1: Gans Original Report
Appendix 2: WRR Statistical Science Paper:
Appendix 3: Instructions to the Experts (Hebrew)
Appendix 4: (Hebrew) The City Names Provided By the Experts
Appendix 5: The Bar Natan and Rips Minority Reports
Unfortunately, one expert was not even
given the written instructions, but briefed orally. His work, in fact, deviated from
the instructions. Furthermore, there were glaring errors in the so-called expert data, errors that are not just
one opinion rather than another, but documentable factual errors that the expert himself
acknowledged after being shown them. These are discussed in Witztum's
report (in Hebrew). In a 159 page report, he documents
the blatant errors in the preparation of the Aumann committee data.
Because of these errors,
Professor Lapides, a committee member, refused to sign the final committee report.
Witztum's report Part 1:
Preface Pages 1-4 (In Hebrew)
Pages 5-11 (In Hebrew)
Pages 12-17 (In Hebrew)
Pages 17-24 (In Hebrew)
Appendix 1 (In Hebrew)
Appendix 2 (In Hebrew)
Appendix 3 (In Hebrew)
Appendix 4 (In Hebrew)
Appendix 5 (In Hebrew)
Appendix 6 (In Hebrew)
Appendix 7 (In Hebrew)
Witztum's report Part 2:
Preface (In Hebrew)
Pages 90-96 (In Hebrew)
Pages 97-105 (In Hebrew)
Appendix 1 (In Hebrew)
Appendix 2 (In Hebrew)
Appendix 3 (In Hebrew)
Appendix 4 (In Hebrew)
Appendix 5 (In Hebrew)
Appendix 6 (In Hebrew)
When the errors became publicly known, including the use of data for a mis-identified rabbi,
the committe had to decide what to do. Instead of
playing the role of the scientist, who when shown errors in his experiment would immediately
correct the errors and redo the experiment, the Aumann committe issued a statement.
In this statement Aumann acknowledges problems with the execution of the committee's instructions
and acknowledges errors in the data. But he sums it all up by saying that mistakes are taken into
account in the experimental design!
Doron Witztum writes that when he first checked a subset of the expert data he found:
"17 instances in which the expert's data did not exist in the references given to him to support his
decisions. Moreover, in 6 cases, the information given in the references contradicted the data he
claimed to derive from them."
Professor Lapides writes in his statement:
"A thorough check has shown that the work of the experts and of the representatives of the committee
was plagued with extreme carelessness, resulting in dozens of errors, and in many essential violations
of the instructions of the Committee. The experiments based on those data therefore lacked any
scientific meaning."
It is worthwhile to read the full statements of Professor Aumann, Professor Furstenberg, Professor
Lapides and Doron Witztum. Their statements can be found below. All are taken unedited from
the Aumann Committee post-experiment report.
Lapides Statement
Witztum Statement
Furstenberg Statement
Aumann Statement
Our summary of the Aumann committee experiment,
--
To err is human. A true blundering failure is when we do not rise up
to correct our errors.
Professor Haralick performed his own independent experiment with the committee's data (corrected)
and obtained interesting results.
The Haralick Experiment