The Short Answer
A systematic review or meta-analysis is only as trustworthy as the studies it includes. Two reviews can ask the same question and reach different conclusions simply because they included different sets of studies, using different rules for what counted as “good enough” evidence. Before trusting a headline finding from a systematic review, it helps to know how the researchers decided which studies got in, which were left out, and why.
What “Included Studies” Actually Means
A systematic review is not just a summary of “the research.” It is a structured process: researchers write down, in advance, the exact rules for which studies qualify (called eligibility criteria), then search multiple databases to find every study that might fit, then screen each one against those rules. Only studies that pass screening become “included studies.” A meta-analysis goes one step further by combining the numerical results from those included studies into a single pooled estimate.
This matters because the review's conclusion is a summary of whatever went into it. If the inclusion rules were too narrow, relevant evidence gets left out. If they were too loose, weak or mismatched studies get pooled in, which can skew the result in either direction.
How Included Studies Are Chosen: The Evidence Ladder
Reputable systematic reviews generally follow a structured, multi-step process to build their list of included studies. PRISMA 2020 — the internationally endorsed reporting guideline for systematic reviews, published in the BMJ in 2021 — sets out reporting requirements covering how a review identifies, screens, and reports on the studies it considers.
- Step 1 — Set eligibility criteria before searching. Under the Cochrane Handbook for Systematic Reviews of Interventions, current version, a systematic review begins with an upfront research question and clear agreement on the scope of the review and which studies will be eligible, decided before the evidence is gathered rather than adjusted afterward. This advance planning is described as part of what gives systematic review methodology its structured, transparent, and reproducible character.
- Step 2 — Search broadly, not just one database. The Cochrane Handbook states plainly that a search of MEDLINE alone “is not considered adequate,” because relevant studies are often missed unless the search extends to multiple databases, trial registers, and other sources. A search limited to one convenient database raises the risk of leaving out studies that would have changed the result.
- Step 3 — Screen every study against the same rules. Each study found in the search is checked against the pre-set eligibility criteria, and studies that do not fit are tracked as excluded rather than simply dropped. The Cochrane Handbook devotes a specific section to how reviewers select and document excluded studies, not only included ones.
- Step 4 — Report the selection process, including a flow diagram. PRISMA 2020 includes standardized flow diagrams, used to show how many records were found, how many were screened, and how many ultimately made it into the review. Readers can use this diagram to see the scale of the filtering process, not just the final count of included studies.
When a review skips or blurs any of these steps, or does not clearly report them, readers have less ability to judge whether the conclusion reflects the full body of evidence or a narrower, possibly biased slice of it.
Known vs. Unknown: What Careful Study Selection Can and Cannot Fix
What a careful, well-documented selection process can tell a reader:
- Whether the review searched more than one database, which the Cochrane Handbook identifies as important for reducing the chance of missing eligible studies
- Whether the eligibility criteria were defined before the search, rather than adjusted afterward to fit a preferred result
- How many studies were considered, and how many were excluded, as shown in a flow diagram or similar accounting
- Whether the review searched for unpublished or ongoing studies, since studies with statistically significant results are more likely to get published in the first place — a pattern the Cochrane Handbook describes as a documented source of publication bias
What study selection alone cannot tell a reader:
- Whether the individual included studies were well-designed or free of bias in their own right — that is a separate assessment (often called “risk of bias” appraisal) and is not the same step as deciding whether a study is eligible for inclusion
- Whether the pooled result from a meta-analysis applies to a specific person's situation, since eligibility criteria are set at the level of populations and study designs, not individual patients
- Whether newer studies published after the review's search date might change the picture — a systematic review reflects the evidence available as of a specific search date, not an ongoing real-time picture
A Practical Next-Step Checklist for Reading a Systematic Review
Before treating a systematic review's conclusion as settled, it can help to check for the following:
- Does the review state its eligibility criteria clearly, including the population, intervention or exposure, comparison, and outcomes it required?
- Does it name more than one database or source searched, along with the date range covered?
- Does it include a flow diagram or similar accounting of how many studies were found, screened, excluded, and included?
- Does it disclose whether it searched for unpublished or ongoing studies, and how it handled language restrictions?
- Does it distinguish between how studies were selected and how their quality or risk of bias was separately assessed?
- Is the publication itself recent enough, or does it note a more recent update, given that evidence accumulates over time?
These questions do not require a research background to ask. They simply help a reader tell the difference between a systematic review built on a transparent, well-documented selection process and one that is not.
Limits of This Overview
This article explains general principles of how systematic reviews and meta-analyses select studies for inclusion, based on published, publicly available reporting and methodology guidance. It is not a review of any specific study, product, or medical condition, and it does not replace a full reading of a review's own methods section. Readers evaluating evidence for a specific health decision should discuss it with a qualified healthcare provider, who can weigh the relevant research in the context of individual health history.
Sources and Further Reading
- Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71.
- Lefebvre C, Glanville J, Briscoe S, et al. Chapter 4: Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, et al. (editors). Cochrane Handbook for Systematic Reviews of Interventions, current version. Cochrane.
By MedClinRes.org Clinical Research Team. Last updated September 17, 2026. Read more about our editorial standards and who we are.