This link has been bookmarked by 1 people . It was first bookmarked on 26 Apr 2009, by Charlie Gibbons.
-
26 Apr 09
-
The confidence level is set in advance. The resulting p-value is a random variable. To adjust the level in response to the p-value is an improper ex post move. So if the pre-set level of significance is 95% and the resulting p-value is 0.0001, the proper response is to say "The results are significant at the 95% level (p = 0.0001)." To readjust the claim to "The results are significant at the 99% level (since) p = 0.0001" is simply incorrect.
Thus: there should only be one star per table, at whatever level the analyst sets in advance.
-
And so it is disappointing that rarely if ever do we see a paper wrestle with the proper level of significance, such as by asking whether this is a case where a false negative is better or worse than a false positive. After all, it is not always clear that we are best served by the conservatism of the 95% confidence interval. A false positive may be worse than a false negative in criminal law ("better 10 guilty men go free..."), but a false negative may be worse in some medical situations, such as whether a particular pill works.
-
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.