GRADE: what certainty of evidence means and how it is decided
A pooled estimate says what the evidence shows. Certainty says how much the estimate is likely to change when better evidence arrives — a separate and often more useful question.
GRADE rates how confident we can be that an estimate of effect is close to the true effect, for a specific outcome. It is not a rating of the studies, and it is not a rating of the review.
The starting point
Randomised trials start at high certainty. Observational studies start at low. Both can move.
Five reasons to rate down
- Risk of bias. Limitations in the design or conduct of the contributing studies.
- Inconsistency. Effects vary across studies in a way that is not explained.
- Indirectness. The population, intervention, comparator or outcome differs from the question being asked.
- Imprecision. The confidence interval spans values that would lead to different clinical decisions.
- Publication bias. Grounds to suspect that studies with unfavourable results were not published.
Three reasons to rate up
These apply only to observational evidence not already rated down.
- A large magnitude of effect.
- A dose-response gradient.
- Plausible residual confounding that would work against the observed effect.
The two mistakes that matter
The first is rating certainty for the review as a whole. Certainty is per outcome. Mortality and quality of life in the same review routinely differ by two levels, and a single overall rating hides exactly the differences a reader needs.
The second is treating imprecision as a p-value question. The judgement is about whether the interval crosses a threshold that would change a decision, not about whether it crosses the null. A precise interval entirely within a clinically trivial range is not imprecise; a significant result whose interval runs from "barely worth it" to "transformative" is.
Why it is worth doing
A number without a certainty rating invites the reader to treat it as settled. GRADE forces the review to state how much the estimate might move, which is what a clinician deciding whether to act on it actually needs to know.
It also makes disagreement productive: someone who thinks the rating is wrong has to say which domain and why, which is a far more useful argument than a general objection to the conclusion.
Frequently asked questions
What does GRADE actually rate?
How confident we can be that an estimate of effect is close to the true effect, for one specific outcome. It is not a rating of the studies and not a rating of the review.
What are the reasons to rate down certainty in GRADE?
Risk of bias, inconsistency, indirectness, imprecision and publication bias. Observational evidence not already rated down can be rated up for a large effect, a dose-response gradient, or residual confounding that would work against the observed effect.
Is certainty of evidence rated for the whole review?
No, per outcome. Mortality and quality of life in the same review routinely differ by two levels, and a single overall rating hides exactly the differences a reader needs.