When should you perform a meta-analysis — and when should you not?
Software will pool almost anything you give it. The methodological question is whether a single number describing these particular studies means anything at all.
A meta-analysis produces one number from many studies. The number is only interpretable if the studies are estimating something similar enough that a common estimate is a sensible thing to want.
That is a judgement about the studies, not about the statistics, and it has to be made before the model is fitted rather than justified afterwards by an acceptable heterogeneity statistic.
The question to ask first
Not "can these be pooled" but "what would the pooled estimate be an estimate of?"
If the studies used different populations, different versions of the intervention, different comparators, different outcome definitions or wildly different follow-up, the pooled result is an average of things that were never the same quantity. It will have a confidence interval, and the interval will be misleadingly narrow, because precision is a property of the arithmetic rather than of the evidence.
Reasonable grounds to pool
- The studies address the same clinical question in populations you would be willing to treat similarly.
- The intervention and comparator are close enough that a clinician would call them the same decision.
- The outcome is defined comparably, or can be harmonised transparently.
- Follow-up periods are similar, or the difference is handled explicitly.
- Risk of bias is assessed, and the high-risk studies are not silently carrying the result.
Reasonable grounds not to
- Two or three small studies with very different designs. A pooled estimate from three studies is mostly an estimate of the largest one.
- Clinical heterogeneity you would not accept in a single trial. If you would not enrol these populations in one study, pooling them afterwards does not fix it.
- Outcomes measured on scales that cannot be reconciled without assumptions you cannot defend.
- Evidence dominated by one large study at high risk of bias.
Heterogeneity statistics do not answer this
I-squared is the proportion of observed variability attributable to between-study differences rather than chance. It is not a measure of how different the studies are clinically, and it is not a threshold for whether pooling is permitted.
A low I-squared across clinically incompatible studies means those studies are consistently estimating something that has no single meaning. A high I-squared across an otherwise coherent set is a signal to investigate, not automatically to abandon.
This is why the decision belongs in the protocol. Deciding after seeing I-squared is deciding after seeing the answer.
What to do instead
A structured narrative synthesis is a legitimate result, not a failure. Present the studies, the effect estimates, the direction of effect, the sources of variation and what the pattern supports. A review that explains why the evidence cannot be summarised in one number is more useful than one that summarises it anyway.
Where some subsets are coherent and others are not, pool the subsets that are and say why the rest were not pooled.
Frequently asked questions
When should you not perform a meta-analysis?
When the studies differ clinically in ways you would not have accepted within a single trial, when outcomes cannot be reconciled without indefensible assumptions, or when the evidence is dominated by one study at high risk of bias. A structured narrative synthesis is a legitimate result, not a failure.
Does a low I-squared mean it is safe to pool?
No. I-squared measures the proportion of variability not attributable to chance; it says nothing about whether the studies are clinically comparable. A low value across incompatible studies means they consistently estimate something with no single meaning.
How many studies do you need for a meta-analysis?
There is no threshold, but a pooled estimate from two or three studies is mostly an estimate of the largest one, and between-study variance cannot be estimated reliably from so few. Whether pooling is meaningful matters more than the count.