Significance of E. Coli Evolution Experiments

Blount, Borland, and Lenski claimed that a key evolutionary innovation was observed during a laboratory experiment. That claim is false. The claim was based on incorrect measurements of statistical significance. Rather than using a test from the statistics literature, a flawed test was contrived and used to measure significance. The flawed test (“mean mutation generation”) produced artificially low p-values.

The data from experiment one of the paper is shown below (see Table 1 of the paper). The expected outcomes under the null hypothesis (no evolutionary innovation occurs) are also shown.

Generation	Trials	Mutants	Statics	Expected Mutants	Expected Statics
0	6	0	6	0.333	5.667
10000	6	0	6	0.333	5.667
20000	6	0	6	0.333	5.667
25000	6	0	6	0.333	5.667
27500	6	0	6	0.333	5.667
29000	6	0	6	0.333	5.667
30000	6	0	6	0.333	5.667
30500	6	1	5	0.333	5.667
31000	6	0	6	0.333	5.667
31500	6	1	5	0.333	5.667
32000	6	0	6	0.333	5.667
32500	6	2	4	0.333	5.667
Total	72	4	68	4	68

When the flawed test is used to compute the significance of this data, the p-value is 0.0085 (see Table 2 of the paper). This p-value is considered statistically significant. However, when the data is analyzed using a standard method (the chi-square test) the p-value is 0.19. This p-value is much larger than the one from the paper and indicates that there is no reason to reject the null hypothesis. The chi-square test p-value for experiment two is small (0.0004). However, experiment three is not statistically significant because its p-value is 0.22.

The chi-square test is a common statistical method. It can be implemented in Microsoft Excel. If the numbers from the last four columns of the table above (excluding the “totals” row) are entered into Excel in rows 1-12 and columns A-D, then the p-value can be computed by entering “=CHITEST(A1:B12,C1:D12)” into any empty cell of the spreadsheet.