17
edits
Changes
Jump to navigation
Jump to search
Line 152:
Line 152: + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
Talk:Significance of E. Coli Evolution Experiments (view source)
Revision as of 20:58, March 14, 2009
, 20:58, March 14, 2009→Unreferenced Claims: Provided some references
There was a typo in my edit summaries on the talk page and the main page. I meant to say "Removed unsupported claims" rather than "Removed supported claims". [[User:SJohnson|SJohnson]] 13:13, 14 March 2009 (EDT)
There was a typo in my edit summaries on the talk page and the main page. I meant to say "Removed unsupported claims" rather than "Removed supported claims". [[User:SJohnson|SJohnson]] 13:13, 14 March 2009 (EDT)
I do not believe that anyone has claimed that 'chi-square test p-values are ''always'' conservative'. The claim that has been made is that ''under certain circumstances'', namely low n and low individual cell values, the chi-square test is an invalid test; that under those circumstances the power of the test is low and it becomes impossible to reject the null hypothesis even when it is false. You may have missed the pertinent sections in my links above, so I will directly quote the relevant sections. All the quoted sections refer to chi-square testing in particular. Any bolding below is mine.
<blockquote>
"Assumptions:<br />
Even though a nonparametric statistic does not require a normally distributed population, there still are some restrictions regarding its use.<br />
1. Representative sample (Random)<br />
2. The data must be in frequency form (nominal data) or greater.<br />
3. The individual observations must be independent of each other.<br />
4. '''Sample size must be adequate. In a 2 x 2 table, Chi Square should not be used if n is less than 20. In a larger table, no expected value should be less than 1, and not more than 20% of the variables can have expected values of less than 5'''.<br />
5. Distribution basis must be decided on before the data is collected.<br />
6. The sum of the observed frequencies must equal the sum of the expected frequencies."
</blockquote>
<ref>http://www.okstate.edu/ag/agedcm4h/academic/aged5980a/5980/newpage28.htm
</ref>
<blockquote>
"Assumptions:
* Random sample data are assumed. As with all significance tests, if you have population data, then any table differences are real and therefore significant. If you have non-random sample data, significance cannot be established, though significance tests are nonetheless sometimes utilized as crude "rules of thumb" anyway.
* A sufficiently large sample size is assumed, as in all significance tests. '''Applying chi-square to small samples exposes the researcher to an unacceptable rate of Type II errors. There is no accepted cutoff. Some set the minimum sample size at 50, while others would allow as few as 20'''. Note chi-square must be calculated on actual count data, not substituting percentages, which would have the effect of pretending the sample size is 100.
* '''Adequate cell sizes are also assumed. Some require 5 or more, some require more than 5, and others require 10 or more. A common rule is 5 or more in all cells of a 2-by-2 table, and 5 or more in 80% of cells in larger tables, but no cells with zero count'''. When this assumption is not met, Yates' correction is applied.
* Independence. Observations must be independent. The same observation can only appear in one cell. '''This means chi-square cannot be used to test correlated data (ex., before-after, matched pairs, panel data)'''.
* Similar distribution. Observations must have the same underlying distribution.
* Known distribution. The hypothesized distribution is specified in advance, so that the number of observations that are expected to appear each cell in the table can be calculated without reference to the observed values. Normally this expected value is the crossproduct of the row and column marginals divided by the sample size.
* Non-directional hypotheses are assumed. Chi-square tests the hypothesis that two variables are related only by chance. If a significant relationship is found, this is not equivalent to establishing the researcher's hypothesis that A causes B, or that B causes A.
* Finite values. Observations must be grouped in categories.
* Normal distribution of deviations (observed minus expected values) is assumed. Note chi-square is a nonparametric test in the sense that is does not assume the parameter of normal distribution for the data -- only for the deviations.
* Data level. No assumption is made about level of data. Nominal, ordinal, or interval data may be used with chi-square tests."
</blockquote>
<ref>http://faculty.chass.ncsu.edu/garson/PA765/chisq.htm</ref>
<blockquote>
"Assumptions:<br />
-None of the expected values may be less than 1<br />
-No more than 20% of the expected values may be less than 5"</blockquote> <ref>http://www.wellesley.edu/Psychology/Psych205/chisquareindep.html</ref>
<blockquote>
"When performing a chi-square test, your data must satisfy important assumptions. Although these assumptions may be stated differently in different textbooks, they generally assert that:<br />
1)The sample must be randomly drawn from the population<br />
'''2)The sample size, n, must be large enough so that the expected cell count in each cell is greater than or equal to 5.'''<br />
Both assumptions must be met in the process of collecting your data, and violations of the second assumption will appear in the Minitab output when you run the analysis.<br />
...<br />
'''You may wonder why the second assumption is necessary for performing the chi-square test. The second assumption arises because the distribution of counts under the null hypothesis is multinomial, and the normal distribution can be used to approximate the multinomial distribution if the sample size is sufficiently large and the probability parameters aren't too small. It can be shown via the Central Limit Theorem that the multinomial distribution converges to the normal distribution as the sample size approaches infinity; however, there is no easy way to show mathematically how and when the convergence fails.'''"
</blockquote>
<ref>http://www.minitab.com/support/docs/Answers/Chi-Square%20Test%20Assumptions.pdf</ref>
<blockquote>
"'''The chi-square test is simpler to calculate but yields only an approximate P value. ... You should definitely avoid the chi-square test when the numbers in the contingency table are very small (any number less than about six)'''."
</blockquote> <ref>http://www.graphpad.com/www/Book/Choose.htm</ref>
<blockquote>
"The most important things to remember to get a valid χ2 test are that the expected values are not too small in any bin (certainly 5 or more), and that the degrees of freedom are properly evaluated. '''Unless you have a very large amount of data, the test is not very sensitive and errs on the side of safety. If you get a significant result, however, it is not likely to be wrong.'''"
</blockquote> <ref>http://mysite.du.edu/~jcalvert/econ/chisquar.htm</ref>
<blockquote>
"The critical assumptions of the chi-square test for k independent samples are similar to those for the chi-square test for two independent samples.<br /> ...<br />
'''4. No more than 20% of the cells may have expected frequencies of less than 5, and no cell should have an expected frequency of less than 1. <br />
The rule given in Assumption 4 is particularly important for a contingency table that is larger than 2X2'''"
</blockquote> <ref>http://books.google.com/books?id=yU15rUiLRI8C&pg=PA201&lpg=PA201&dq=chi-square+test+assumptions&source=bl&ots=FRY0LwQ3z_&sig=FyIvzJx3hjQ8nWlu2cpmZj3pwXY&hl=en&ei=fm-1SayaNI_MMKX5tO4E&sa=X&oi=book_result&ct=result#PPA185,M1
</ref>
<blockquote>
"Special problems with small expected cell frequencies for the chi-square test:<br />
The chi-square test involves using the chi-square distribution to approximate the underlying exact distribution. The approximation becomes better as the expected cell frequencies grow larger, and '''may be inappropriate for tables with very small expected cell frequencies.'''<br />
'''For tables with expected cell frequencies less than 5, the chi-square approximation may not be reliable. A standard (and conservative) rule of thumb (due to Cochran) is to avoid using the chi-square test for tables with expected cell frequencies less than 1, or when more than 20% of the table cells have expected cell frequencies less than 5.'''<br />
Another rule of thumb (due to Roscoe and Byars) is that the average expected cell frequency should be at least 1 when the expected cell frequencies are close to equal, and 2 when they are not. (If the chosen significance level is 0.01 instead of 0.05, then double these numbers.)<br />
Koehler and Larntz suggest that if the total number of observations is at least 10, the number categories is at least 3, and the square of the total number of observations is at least 10 times the number of categories, then the chi-square approximation should be reasonable.<br />
Care should be taken when cell categories are combined (collapsed together) to fix problems of small expected cell frequencies. Collapsing can destroy evidence of non-independence, so a failure to reject the null hypothesis for the collapsed table does not rule out the possibility of non-independence in the original table.<br />
'''As with most statistical tests, the power of the chi-square test increases with a larger number of observations. If there are too few observations, it may be impossible to reject the null hypothesis even if it is false.'''"
</blockquote> <ref>http://www.basic.northwestern.edu/statguidefiles/gf-dist_ass_viol.html</ref>
== References ==
{{reflist}}