Flaws in Richard Lenski Study

Richard Lenski rejected a request to release his data to the public, but the following serious flaws are emerging about his work even without a full disclosure of the data. Note that the peer review on Lenski's paper took somewhere between 0 (non-existent) and at most 14 days (including administrative time), and Lenski himself does not have any obvious expertise in statistics. In fact, he admits in his paper that he based his statistical conclusions on use of a website called "statistics101".

1. Lenski's "historical contingency" hypothesis, as specifically depicted in Figure 3, is contradicted by the data presented in the Third Experiment in Table 1 of his paper. Figure 3 proposes a step-up in mutation rate to Cit⁺ due to a historical contingency (potentiating mutation) occurring at about the 31,000th generation, yet the Third (and largest) Experiment in Table 1 shows Cit⁺ arising just as often before the 31,000th generation as after. The abstract, in further contradiction with Figure 3, suggests that the historical contingency (potentiating mutation) occurred prior to the 20,000th generation.

2. Lenski incorrectly included generations of the E. coli already known to contain Cit⁺ variants in his experiments. Once these generations are removed from the analysis, the data disprove Lenski's hypothesis.

3. Lenski's largest experiment (Third Experiment) failed to support his hypothesis with statistical significance. Even though this largest experiment was nearly ten times the size of his other experiments, Lenski did not weight this largest experiment correctly in combining his results.

4. Lenski's two alternative hypotheses suggest a fixed mutation rate, but the failure of the mutations in his experiments to increase based on scale (number of samples) tends to disprove both of Lenski's alternative hypotheses. Yet Lenski's paper fails to address adequately this obvious flaw in the paper.

5. Lenski's paper is not clear in explaining how the results of his largest experiment (Third Experiment) failed to confirm his hypothesis with statistical significance, even with the incorrect inclusion of the Cit⁺ variant generations. Instead, his paper refers to his largest experiment as "marginally ... significant," which serves to obscure its statistical insignificance. Other works published in PNAS are clear in defining statistical significance in the traditional way, which Lenski's Third Experiment (even with incorrect inclusion of the above-referenced generations) failed to satisfy.