One-Way ANOVA with Unequal Sample Sizes: Assumptions, Methods, and Alternatives

A common question in statistical analysis is whether a one-way ANOVA (Analysis of Variance) can be performed when the groups have different sample sizes.

The answer is yes. Equal sample sizes are not a requirement for a standard one-way ANOVA.

However, unequal group sizes can affect the robustness and interpretation of the analysis, particularly when the group variances are also different. Therefore, researchers should evaluate the assumptions carefully before selecting the appropriate method.

What Is One-Way ANOVA?

One-way ANOVA is used to determine whether the means of three or more independent groups differ significantly.

For example, a researcher might compare:

• Average blood pressure across three treatment groups
• Product scores across four formulations
• Exam scores across different teaching methods
• Customer satisfaction across several regions

The null hypothesis is: H0​:μ1​=μ2​=⋯=μk​

The alternative hypothesis is: HA​:At least one group mean differs

The ANOVA F-statistic compares between-group variability with within-group variability.

Can ANOVA Be Used With Unequal Sample Sizes?

Yes.

Suppose a study has three groups:

A one-way ANOVA can still be performed.

The calculation of the ANOVA statistic naturally accommodates different group sizes.

However, unequal sample sizes become more important when combined with heterogeneous variances.

Why Equal Sample Sizes Are Often Preferred

Equal sample sizes are not mandatory, but balanced designs have several advantages.

1. Statistical Power

A balanced design generally provides good efficiency for detecting differences among group means for a fixed total sample size.

When sample sizes become highly unequal, statistical efficiency can decrease, particularly when the smallest groups contain relatively few observations.

This can make it more difficult to detect a genuine difference.

2. Unequal Variances Become More Problematic

The standard one-way ANOVA assumes that the population variances are approximately equal: σ12​=σ22​=⋯=σk2​

ANOVA can be reasonably robust to moderate departures from this assumption, particularly with balanced sample sizes.

However, when sample sizes and variances are both unequal, the Type I error rate can be affected.

Therefore, unequal sample sizes should prompt greater attention to variance heterogeneity.

Example of Unequal Sample Sizes

Suppose researchers want to compare the effectiveness of three training programs.

The sample sizes are:

• Program A: 20 participants
• Program B: 30 participants
• Program C: 50 participants

The outcome is a performance score.

The unequal sample sizes themselves do not prevent the researcher from using one-way ANOVA.

The researcher should instead examine:

1. Independence
2. Distribution of residuals
3. Homogeneity of variance
4. Presence of influential observations
5. The research design and sampling process

Assumption 1: Independent Observations

The observations should be independent within and between groups.

For example, if each participant contributes one performance score and participants are randomly assigned to independent groups, the independence assumption may be reasonable.

However, repeated measurements from the same participant are not independent and should not be analyzed using a simple one-way ANOVA.

Repeated-measures ANOVA or mixed-effects models may be more appropriate.

Assumption 2: Continuous Outcome Variable

The dependent variable should generally be measured on a continuous scale, although ANOVA can also be used with certain approximately continuous measurements.

The grouping variable should be categorical with at least three groups for the usual one-way ANOVA application.

Assumption 3: Approximately Normal Residuals

The normality assumption applies primarily to the model residuals, rather than requiring every group’s raw data to be perfectly normally distributed.

Researchers can examine:

• Q-Q plots
• Histograms
• Residual plots

Formal normality tests such as the Shapiro-Wilk test can also be used, but they should not be interpreted in isolation.

With sufficiently large samples, ANOVA can often tolerate moderate deviations from normality.

Assumption 4: Homogeneity of Variance

The standard one-way ANOVA assumes that the population variances are approximately equal.

Researchers can examine this using:

• Boxplots
• Residual plots
• Levene’s test
• Brown-Forsythe test
• Bartlett’s test

Levene’s test or the Brown-Forsythe test is often preferable when normality is questionable, because Bartlett’s test is more sensitive to departures from normality.

What If Variances Are Unequal?

This is where the choice of method becomes particularly important.

If the group sizes are unequal and the variances differ substantially, consider Welch’s one-way ANOVA.

Welch’s ANOVA does not require the assumption of equal population variances and is often a better choice than simply applying the standard ANOVA when heteroscedasticity is present.

For example:

Unequal sample sizes        ↓Check variance homogeneity        ↓Equal variances?     ↙       ↘   Yes        No    ↓          ↓Standard     Welch'sANOVA        ANOVA

This is generally a better decision framework than automatically switching to a non-parametric test whenever variances are unequal.

Welch’s ANOVA

Welch’s ANOVA is specifically designed for comparing means when group variances may differ.

It is particularly useful when:

• Sample sizes are unequal.
• Variances are unequal.
• The outcome is approximately continuous.
• The observations are independent.

Therefore, if you have unequal sample sizes and clear heterogeneity of variance, Welch’s ANOVA is often the first alternative to consider.

What About the Kruskal-Wallis Test?

The Kruskal-Wallis test is a non-parametric alternative to one-way ANOVA.

It is useful when the assumptions required for mean-based ANOVA are not appropriate, particularly when the outcome is ordinal or when distributional assumptions are substantially problematic.

However, Kruskal-Wallis should not be viewed simply as “ANOVA when the normality test fails.”

It tests differences in the distributions/ranks of groups and answers a somewhat different question.

For example:

Three or more independent groups              ↓       Continuous outcome              ↓       Examine assumptions              ↓   ┌──────────┴──────────┐   ↓                     ↓Approximately         Strong variancenormal / suitable     heterogeneity   ↓                     ↓Standard ANOVA       Welch ANOVA   │   └──── If distributional assumptions are         substantially unsuitable ────→              Kruskal-Wallis

The choice should depend on the data, study design, and scientific question rather than a single p-value from a normality test.

Post-Hoc Tests With Unequal Sample Sizes

If the overall ANOVA is statistically significant, researchers often need to determine which groups differ.

The choice of post-hoc procedure should match the ANOVA method.

For standard ANOVA with approximately equal variances, Tukey’s HSD is commonly used.

When variances are unequal, Games-Howell is often an appropriate post-hoc method following Welch’s ANOVA.

A useful framework is:

Why Sample Size Imbalance Matters

Consider two studies with the same total sample size.

Study A

This is a balanced design.

Study B

Both studies have 120 observations, but Study B has substantial imbalance.

The smaller group contributes less information to the estimation of its mean and variance.

If the smallest group also has a substantially different variance, the standard ANOVA can become less reliable.

This illustrates why researchers should consider sample-size balance and variance balance together.

A Better Analysis Workflow

Instead of automatically deciding whether to use ANOVA based only on sample size, use a structured workflow.

Step 1: Understand the Study Design

Determine:

• Are observations independent?
• Are groups independent?
• Is the outcome continuous?
• Are there repeated measurements?
• Are there clusters or matched observations?

Study design should be evaluated before statistical testing.

Step 2: Examine the Data

Use:

• Boxplots
• Histograms
• Q-Q plots
• Descriptive statistics
• Group-specific sample sizes

Look for skewness, extreme observations, and substantial variance differences.

Step 3: Evaluate Variance Homogeneity

Use visual assessment together with an appropriate formal test.

Levene’s or Brown-Forsythe tests are commonly useful when normality is uncertain.

Step 4: Select the ANOVA Method

If assumptions are reasonably satisfied:

Standard one-way ANOVA

If variances differ substantially:

Welch’s ANOVA

If the outcome is ordinal or the assumptions for mean-based analysis are seriously unsuitable:

Kruskal-Wallis test

Step 5: Perform Appropriate Post-Hoc Comparisons

Use a post-hoc procedure consistent with the overall test.

For example:

• Standard ANOVA → Tukey HSD
• Welch ANOVA → Games-Howell
• Kruskal-Wallis → Dunn’s test with an appropriate multiple-comparison adjustment

Step 6: Report Effect Size

Statistical significance alone does not tell you how large the group differences are.

Consider reporting:

• Eta squared (η2)
• Partial eta squared (ηp2​)
• Omega squared (ω2)
• Confidence intervals for group differences

Effect-size reporting makes the analysis more informative.

One-Way ANOVA With Unequal Sample Sizes in R

A standard one-way ANOVA can be performed using:

model<-aov(score~group, data=mydata)summary(model)

For checking variance homogeneity:

car::leveneTest(score~group, data=mydata)

If unequal variances are present, Welch’s ANOVA can be performed using:

oneway.test(score~group,data=mydata,var.equal=FALSE)

For a standard ANOVA assuming equal variances:

oneway.test(score~group,data=mydata,var.equal=TRUE)

For a Kruskal-Wallis test:

kruskal.test(score~group, data=mydata)

Common Mistakes

Mistake 1: Assuming Unequal Sample Sizes Automatically Invalidate ANOVA

They do not.

One-way ANOVA can be performed with unequal group sizes.

Mistake 2: Automatically Using Kruskal-Wallis

Unequal variances do not automatically mean that Kruskal-Wallis is the best alternative.

When comparing means with unequal variances, Welch’s ANOVA is often more directly aligned with the research question.

Mistake 3: Relying Only on a Normality Test

A p-value from a normality test should not be the sole basis for choosing between ANOVA and another method.

Use graphical diagnostics, sample sizes, subject-matter knowledge, and residual analysis.

Mistake 4: Ignoring the Smallest Group

A highly imbalanced design can make the analysis particularly sensitive to the behavior of the smallest group.

Mistake 5: Using the Wrong Post-Hoc Test

The post-hoc procedure should correspond to the assumptions of the overall test.

For example, Games-Howell is generally more appropriate than Tukey HSD when substantial heterogeneity of variance is present.

Conclusion

One-way ANOVA can absolutely be performed with unequal sample sizes. Equal group sizes are not a fundamental requirement of the method.

However, substantial imbalance deserves careful attention, particularly when combined with unequal variances. The standard ANOVA is most reliable when observations are independent, residuals are reasonably well behaved, and group variances are approximately homogeneous.

A practical approach is:

1. Examine the study design and independence.
2. Inspect the distributions and outliers.
3. Evaluate variance homogeneity.
4. Use standard ANOVA when its assumptions are reasonably appropriate.
5. Use Welch’s ANOVA when variances are substantially unequal.
6. Consider Kruskal-Wallis when a rank-based, non-parametric approach is more appropriate.
7. Use an appropriate post-hoc test and report effect sizes alongside p-values.

Therefore, unequal sample sizes alone should not prevent you from performing one-way ANOVA. The more important question is whether the chosen method appropriately reflects the variance structure, distribution, and design of your data.

Related: Statistical Analysis » Statistics Methods » Quick Guide » FINNSTATS