37
edits
Changes
Jump to navigation
Jump to search
Significance of E. Coli Evolution Experiments (view source)
Revision as of 14:26, March 4, 2009
, 14:26, March 4, 2009Created page about the incorrect p-values in the E. Coli evolution paper
Blount, Borland, and Lenski<ref> http://www.pnas.org/content/105/23/7899.full.pdf</ref> claimed that a key evolutionary innovation was observed during a laboratory experiment. That claim is false. The claim was based on incorrect measurements of statistical significance. Rather than using a test from the statistics literature, a flawed test was contrived and used to measure significance. The flawed test (“mean mutation generation”) produced artificially low p-values.
The data from experiment one of the paper is shown below (see Table 1 of the paper). The expected outcomes under the null hypothesis (no evolutionary innovation occurs) are also shown.
{|class="wikitable" style="text-align:center"
!Generation
!Trials
!Mutants
!Statics
!Expected Mutants
!Expected Statics
|-
|0
|6
|0
|6
|0.333
|5.667
|-
|10000
|6
|0
|6
|0.333
|5.667
|-
|20000
|6
|0
|6
|0.333
|5.667
|-
|25000
|6
|0
|6
|0.333
|5.667
|-
|27500
|6
|0
|6
|0.333
|5.667
|-
|29000
|6
|0
|6
|0.333
|5.667
|-
|30000
|6
|0
|6
|0.333
|5.667
|-
|30500
|6
|1
|5
|0.333
|5.667
|-
|31000
|6
|0
|6
|0.333
|5.667
|-
|31500
|6
|1
|5
|0.333
|5.667
|-
|32000
|6
|0
|6
|0.333
|5.667
|-
|32500
|6
|2
|4
|0.333
|5.667
|-
|-
!Total
!72
!4
!68
!4
!68
|}
When the flawed test is used to compute the significance of this data, the p-value is 0.0085 (see Table 2 of the paper). This p-value is considered statistically significant. However, when the data is analyzed using a standard method (the chi-square test) the p-value is 0.19. This p-value is much larger than the one from the paper and indicates that there is no reason to reject the null hypothesis. The chi-square test p-value for experiment two is small (0.0004). However, experiment three is not statistically significant because its p-value is 0.22.
The chi-square test is a common statistical method.<ref>''Mathematical Statistics with Applications'' by Wackerly, Mendenhall, and Scheaffer, Section 14.4.</ref> It can be implemented in Microsoft Excel. If the numbers from the last four columns of the table above (excluding the “totals” row) are entered into Excel in rows 1-12 and columns A-D, then the p-value can be computed by entering “=CHITEST(A1:B12,C1:D12)” into any empty cell of the spreadsheet.
==References==
<references/>
The data from experiment one of the paper is shown below (see Table 1 of the paper). The expected outcomes under the null hypothesis (no evolutionary innovation occurs) are also shown.
{|class="wikitable" style="text-align:center"
!Generation
!Trials
!Mutants
!Statics
!Expected Mutants
!Expected Statics
|-
|0
|6
|0
|6
|0.333
|5.667
|-
|10000
|6
|0
|6
|0.333
|5.667
|-
|20000
|6
|0
|6
|0.333
|5.667
|-
|25000
|6
|0
|6
|0.333
|5.667
|-
|27500
|6
|0
|6
|0.333
|5.667
|-
|29000
|6
|0
|6
|0.333
|5.667
|-
|30000
|6
|0
|6
|0.333
|5.667
|-
|30500
|6
|1
|5
|0.333
|5.667
|-
|31000
|6
|0
|6
|0.333
|5.667
|-
|31500
|6
|1
|5
|0.333
|5.667
|-
|32000
|6
|0
|6
|0.333
|5.667
|-
|32500
|6
|2
|4
|0.333
|5.667
|-
|-
!Total
!72
!4
!68
!4
!68
|}
When the flawed test is used to compute the significance of this data, the p-value is 0.0085 (see Table 2 of the paper). This p-value is considered statistically significant. However, when the data is analyzed using a standard method (the chi-square test) the p-value is 0.19. This p-value is much larger than the one from the paper and indicates that there is no reason to reject the null hypothesis. The chi-square test p-value for experiment two is small (0.0004). However, experiment three is not statistically significant because its p-value is 0.22.
The chi-square test is a common statistical method.<ref>''Mathematical Statistics with Applications'' by Wackerly, Mendenhall, and Scheaffer, Section 14.4.</ref> It can be implemented in Microsoft Excel. If the numbers from the last four columns of the table above (excluding the “totals” row) are entered into Excel in rows 1-12 and columns A-D, then the p-value can be computed by entering “=CHITEST(A1:B12,C1:D12)” into any empty cell of the spreadsheet.
==References==
<references/>