Publication bias: funnel plots, Egger's test and what they can and cannot show
Funnel plot asymmetry has several explanations and publication bias is only one of them. Reading it as proof of missing studies is a common and consequential error.
Publication bias is the tendency for studies with unfavourable or imprecise results to remain unpublished. Where it operates, a meta-analysis of the published literature overstates the effect.
The methods for detecting it are weaker than their routine use implies.
The funnel plot
Effect estimate on the horizontal axis, precision on the vertical. With no bias and no heterogeneity, studies scatter symmetrically around the pooled estimate, widening as precision falls.
Asymmetry is what people look for. The problem is that asymmetry has at least four explanations, and publication bias is one of them.
- Genuine heterogeneity, where smaller studies were conducted in different populations or with more intensive delivery.
- Poorer methodological quality in smaller studies, producing inflated effects.
- Chance, which is substantial with few studies.
- Selective publication.
Calling them all "small-study effects" is more accurate than calling them publication bias, and the distinction changes what the review should conclude.
Egger's test
A regression test for asymmetry. Two constraints matter.
It needs at least ten studies. Below that, power is so low that a non-significant result carries almost no information — and it is precisely in small reviews that the test is most often reported.
It also performs poorly with binary outcomes and rare events, where the standard error is mathematically related to the effect estimate, producing asymmetry with no bias present at all.
Trim and fill
It imputes hypothetical missing studies and recomputes the estimate.
It is a sensitivity analysis, not a correction. The imputed studies do not exist, and the adjusted estimate should be reported as "if the asymmetry were entirely due to missing studies, the estimate would be X" — never as a corrected result.
What to do instead
- Search trial registries and conference abstracts, and note studies registered but never published. This is evidence of unpublished work rather than an inference from a plot.
- Look for outcome reporting bias inside published studies by comparing the protocol to the paper.
- Report asymmetry as small-study effects and say which explanations you can rule out and which you cannot.
- Reflect it in the certainty rating rather than in the point estimate.
Frequently asked questions
What does an asymmetric funnel plot mean?
That smaller studies report different effects than larger ones. Publication bias is only one explanation; genuine heterogeneity, poorer methodological quality in small studies, and chance are others. "Small-study effects" is the accurate description.
How many studies are needed for Egger's test?
At least ten. Below that the test has so little power that a non-significant result carries almost no information — and small reviews are where it is most often reported.
Is trim-and-fill a correction for publication bias?
No. It is a sensitivity analysis. The imputed studies do not exist, and the adjusted estimate should be reported conditionally rather than as a corrected result.