How to write a systematic review protocol

A protocol is not paperwork you produce because a journal expects one. It is the document that makes the review interpretable later, because it records what you decided to do before you knew what you would find.

That order matters more than anything else about it. A review whose eligibility criteria were finalised after the authors had seen which studies existed is not a systematic review; it is a defensible-looking selection. Writing the protocol first is what makes the difference checkable.

What the protocol has to decide

Every item below determines something a reader would otherwise have to take on trust.

• The question, stated so that a study can be judged eligible or not without further discussion.
• Eligibility criteria: population, intervention or exposure, comparator, outcomes, study designs, and any language or date limits — with the reasoning for the limits, not just the limits.
• Information sources: which databases, which registries, whether grey literature and reference lists are searched, and when the search will be run.
• The search strategy itself, for at least one database, in full.
• Screening: how many reviewers, working independently or not, and how disagreements are resolved.
• Data extraction: which variables, in what form, and who checks them.
• Risk-of-bias assessment: which tool, applied at which level, by whom.
• Synthesis: whether meta-analysis is planned, under what conditions, which effect measure, which model, and what happens if the studies turn out to be too heterogeneous to pool.
• Any planned subgroup or sensitivity analyses, specified now rather than chosen later.

The parts people skip, and what it costs

Two omissions cause most of the trouble.

The first is the synthesis plan. A protocol that says "a meta-analysis will be performed if appropriate" has not decided anything: it has deferred the decision to a point where the answer is visible, which is the point at which it stops being a methodological judgement.

The second is the plan for heterogeneity. Deciding in advance what degree of clinical or methodological difference would make pooling inappropriate is what stops a pooled estimate from being produced simply because the software would produce one.

Registration

Registering the protocol — PROSPERO is the usual route for reviews of health outcomes — puts a timestamp on the decisions. That is the entire benefit, and it is a substantial one: it converts "we planned this in advance" from a claim into a record.

Registration also surfaces duplicate reviews in progress, which occasionally saves a team from a year of work someone else is already doing.

Changing the protocol

Protocols can change. Reviews take months, and something unanticipated usually appears.

What matters is that the change is reported as a change: what was amended, when, and why. An amendment documented in the review is a normal part of research. An amendment that appears only as a difference between the registered protocol and the published paper is what a reviewer reads as selective reporting.

What a good protocol feels like to write

It should feel slightly uncomfortable, because it forces you to commit to decisions while you can still be wrong about them. That discomfort is the mechanism working. Every decision you defer to the analysis stage is a decision the data will make for you.