Clicky

Medical & Clinical Research Journal

Medical & Clinical Research Journal — Peer-reviewed health analysis, supplement safety reviews, and clinical research summaries.

Systematic Review Protocols: Why a Plan Written First Matters

Guidance such as the Cochrane Handbook expects a systematic review to define its question and methods in a written protocol before the work of selecting studies begins. If a review does not show that plan, or shows one written after the results were known, you cannot easily tell which choices were made in advance and which were made after seeing the data. This article gives a short protocol check card you can apply to any systematic review you read.

Why the plan comes first

The Cochrane Handbook explains that a well-formulated question guides most of the later work: eligibility criteria, the search for studies, data collection, and how results are grouped and presented (Cochrane Handbook, Chapter 2). The Handbook also lists a mandatory expectation for its own reviews: define the objectives, including population, interventions, comparators, and outcomes, in advance.

The same chapter says the review's population, intervention, comparison, and outcome (PICO) are planned at the protocol stage and set which studies are included or excluded. Deciding this before seeing study results limits the room to shape the question around a preferred answer.

What a protocol register does

PROSPERO is an international register of systematic review protocols. An editorial in PLoS Medicine announced it as a service of the Center for Reviews and Dissemination at the University of York and advised investigators to record the protocol for a planned or ongoing review of health care interventions even before screening studies for inclusion (PLoS Medicine editorial). A published description says the register aims to avoid duplicated effort, reduce reporting bias and promote transparency (PubMed record).

Registration is not the only route. A paper surveying options identified PROSPERO, the Research Registry, INPLASY, Open Science Framework Registries, and protocols.io. It also separates prospective registration from publishing a protocol as a manuscript (Where to prospectively register a systematic review).

Protocol check card

Use these six checks on any systematic review. Find the protocol or registry record first, then compare it with the published paper.

  • Question: Is there a clear primary objective naming the intervention, the health problem and the people studied? The Handbook suggests stating this in a single sentence. Look for the population, intervention, comparison, and outcome to be defined, not implied.
  • Eligibility rules: Are the inclusion and exclusion criteria written down before study selection? In the Handbook's framework, these come from the review's PICO. Check whether the paper applies the same criteria the protocol set.
  • Databases: Does the protocol name the sources to be searched? The Handbook notes the question guides searching. As an editorial suggestion, a named list lets you compare the plan with the methods actually reported.
  • Outcomes: Are the outcomes named in advance, and do they matter to the people the review is for? The Handbook expects outcomes to address issues important to consumers, health professionals, and policymakers, and expects potential adverse effects to be considered so summaries are not one-sided.
  • Analysis plan: Does the protocol say how results will be grouped and combined? The Handbook says the PICO for each synthesis is also planned at the protocol stage. It also says authors may build contingencies into the plan for sparse data, such as grouping elements more broadly.
  • Later changes: If the question or methods changed, does the paper say so and explain why? The Handbook says to document changes at the start of the Methods section, and it asks whether results from included studies could have influenced a change.

Changes are not automatically a problem

The Handbook says some refinement of a review question is expected as authors learn what evidence exists. It also warns that data-driven questions can produce false conclusions. What matters is whether authors disclose and justify changes, not whether the plan never moved. If a broad review is split into narrower ones, the Handbook says each component needs its own protocol documenting its eligibility criteria.

Limits of this check

  • A protocol or registry record shows a plan. On its own it does not show that the plan was sound or that it was followed. Comparing the two is the reader's job.
  • This check covers a review's planning, not the quality of the studies it includes. Other parts of the Handbook address risk of bias and certainty of evidence.
  • The Handbook chapter is written mainly for intervention reviews. Other review types may follow different conventions.
  • Registry rules, including who may register and when, can change. Confirm current requirements on the register itself.

Who should seek professional advice?

This article is educational and is not medical advice. If a review's conclusions might affect a treatment decision for you or someone in your care, discuss them with a qualified clinician who can weigh them against your situation. If you plan to conduct a systematic review, consider consulting an experienced review methodologist or an information specialist, and read the current registry guidance before you begin screening. See our Medical Disclaimer.

What you can do next

  1. Look in the paper for a registry number or a link to the protocol.
  2. Open the record, for example on the PROSPERO register, and read the question, eligibility rules, and outcomes.
  3. Compare them with the published paper using the six checks above.
  4. Note any differences and whether the authors explain them.
  5. If no protocol can be found, treat the review's methods with extra caution and look for other evidence.

For how this site handles sources and errors, see our Editorial Standards and Corrections Policy. This article has not been peer reviewed or clinically reviewed.

Sources