Sample size and statistical power: what the calculation actually needs
A sample size is not a number you find; it is the consequence of decisions you make. Most calculations fail because one of those decisions was never made explicitly.
A sample size calculation converts four decisions into a number. If the number looks wrong, one of the decisions is wrong — the arithmetic almost never is.
The four inputs
- The effect you want to be able to detect. Not the effect you expect, and not the effect a previous study reported.
- The variability of the outcome, or the baseline event rate for a binary one.
- The type I error rate, conventionally 0.05, and whether the test is one- or two-sided.
- The power, conventionally 80 or 90 percent.
The second is an estimate and should be stated as one, with its source. The first is where most calculations go wrong.
Choosing the difference to detect
The number should be the smallest difference that would change practice — the minimal clinically important difference — not the largest one you can justify hoping for.
Powering a study on an optimistic effect produces a small, cheap study that will probably miss a real but moderate benefit, and will then be described as showing no difference. That is a design failure disguised as a negative result.
Where a minimal important difference is not established, say how you chose the value. A reader can disagree with a stated assumption; they cannot evaluate an unstated one.
What inflates the requirement
- Loss to follow-up. Inflate by the attrition you actually expect, not by a token five percent.
- Clustering. Randomising clinics rather than patients multiplies the requirement by the design effect, which depends on cluster size and the intracluster correlation. Ignoring it is a common and serious error.
- Multiple primary outcomes or interim analyses, both of which affect the error rate you are working to.
Post-hoc power is not informative
Calculating power after the fact, using the observed effect, tells you nothing that the confidence interval does not already tell you better. It is a deterministic function of the p-value: a non-significant result always yields low observed power, by construction.
If the question is whether the study could have detected a meaningful effect, answer it with the confidence interval. If the interval excludes every difference that would matter, the study was informative and negative. If it includes them, the study was inconclusive — which is a different and more honest statement.
Report it so it can be checked
State the target difference and its justification, the assumed variability and its source, alpha, power, the statistical test assumed, the attrition allowance, and the resulting number. A reader should be able to reproduce the calculation.
Frequently asked questions
What do you need for a sample size calculation?
The smallest difference worth detecting, the variability of the outcome or the baseline event rate, the type I error rate, and the desired power — plus allowances for attrition and clustering. The difference should be the minimal clinically important one, not the effect you hope to see.
Is post-hoc power analysis useful?
No. Observed power computed from the observed effect is a deterministic function of the p-value, so a non-significant result always yields low power by construction. The confidence interval answers the underlying question better.
What happens if you ignore clustering in a sample size calculation?
The study is underpowered. Randomising clinics rather than patients multiplies the requirement by the design effect, which depends on cluster size and the intracluster correlation, and ignoring it is a common and serious error.