Power comparisons of four post-MANOVA tests under variance-covariance heterogeneity and non-normality in the two group case

Multivariate statistical methods have been strongly recommended in behavioral research employing multiple dependent variables. While the techniques are readily available, there is still controversy as to the proper use of the methods that have been developed for analyzing and interpreting data after finding a significant pairwise difference with a multivariate analog of the two group t-test, known as Hotelling's T².

A Monte Carlo simulation was conducted to investigate the relative power of four post-MANOVA tests under violations of multivariate homoscedasticity and normality. The four methods for analyzing multivariate group differences following a significant Hotelling's T² were: (1) univariate F; (2) Bonferroni; (3) multiple Bonferroni; and (4) simultaneous F.

Depending on the conditions examined, either the univariate F test or the multiple Bonferroni procedure was shown to be the most powerful for detecting a true difference between two groups.

The following are the major conclusions drawn from the investigation: (1) Power levels of post-MANOVA tests remain constant under violations of multivariate normality, however, they change considerably in the presence of heterogeneity; (2) The univariate F test provides the most liberal power levels and the simultaneous F test provides the most conservative, regardless of sample size, effect size, distribution shape, and degree of violation; (3) As the size of the effect increases, the rate of correct rejections of a false null hypothesis increases; (4) As sample size increases, the rate of correct rejections of a false null hypothesis increases; (5) Regardless of heterogeneity level, power is always larger at larger group size levels; and (6) Within each group size level, power decreases as heterogeneity increases.

Analytical comparisons show simultaneous F tests have the least power, Bonferroni methods to be intermediate, and univariate F tests most powerful under violations of multivariate heterogeneity.