Benjamini and Hochberg's false
discovery rate (FDR)-controlling
method is applied to the problem of
testing infinitely many contrasts
in linear models. Usual asymptotics for FDR-controlling methods fail
in this case, because the test
statistics are highly dependent. An
alternative formulation using
randomly sampled contrasts allows
direct application of the Glivenko-Cantelli theorem, and exact,
easily calculated critical values
are derived, defining a new
multiple comparisons method for
testing contrasts in linear models.
The new method makes no assumptions about the
data-generating process
other than nonzero sample
variance. The method is seen to be
adaptive, depending on the data
through the usual ANOVA F-statistic,
like the Waller-Duncan Bayesian
multiple comparisons method.
Comparisons with Scheffé's method
are given, and the method is
extended to the simultaneous
confidence intervals of Benjamini and
Yekutieli.