How to read a health study: a field guide to evaluating a claim

Every few weeks, a new headline announces that some food, drug, habit, or supplement changes your risk of disease, built from a single study described in a few sentences with none of the context needed to judge whether the finding is solid. This page is not about any one condition or claim. It is a short field guide to the questions that separate a well-supported health claim from a shaky one.

Ask for the absolute number, not just the percentage

The single most useful habit in reading health news is asking one question every time a risk is described as rising or falling by some percentage: a percentage of what?

Here is an illustrative example, not a real study: suppose a headline reads “New research finds eating processed meat daily raises risk of a certain condition by 50%.” That sounds alarming. But a 50% increase could describe a move from 2 cases per 10,000 people to 3 cases per 10,000 people. The relative risk did rise by half; the absolute risk rose by 1 additional case per 10,000 people, or 0.01 percentage points. Both are true at once, but the relative number is the one that makes headlines because it sounds bigger.

The same logic applies to treatments. A drug described as cutting risk “in half” sounds dramatic, but whether it is depends on how common the event was to begin with. Halving a 20% risk down to 10% is large and important; halving a 0.2% risk down to 0.1% is real but changes the outcome for very few people. A well-reported study states both numbers and the actual event counts; a claim that only reports the relative number cannot yet be evaluated.

How the study was built: randomized trial or observational study

The design of a study shapes how much weight its result deserves. In a randomized controlled trial (RCT), participants are assigned by chance to a treatment group or a comparison group, so the two groups start out similar in every way researchers can and cannot measure. Randomization is what lets a trial say a treatment caused a difference in outcomes.

An observational study does not randomize anything. It follows people who have already made their own choices, such as who takes a supplement or who was prescribed a particular drug, and compares outcomes between groups. Observational studies can show two things are associated, meaning they tend to occur together, but they cannot rule out confounding: some other factor, often the underlying health status or habits behind that choice, may be the real driver of the outcome. This is why “associated with” in a study abstract is not the same claim as “causes.”

This distinction has changed real medical practice. Decades of observational data on hormone therapy in postmenopausal women suggested a reduced risk of heart disease. When the Women’s Health Initiative, a large randomized trial, tested the same question directly, it found a different cardiovascular risk pattern, a surprising result that reshaped guidance on hormone therapy. Observational research is not worthless; it can follow far more people for far longer than most RCTs can afford, and answer questions an RCT cannot ethically ask, such as the long-term effects of smoking. But when the two disagree and a well-run RCT is available, the RCT generally carries more weight, because randomization controls for confounding that observational data cannot rule out.

What was actually measured: a surrogate marker or the outcome that matters

Studies often measure a surrogate endpoint, a lab value or imaging measurement believed to predict a real health outcome, rather than the clinical outcome itself. A drug that improves a surrogate marker has not automatically been proven to prevent the event that marker is supposed to predict; that requires a trial that actually measured the event.

This is not a theoretical concern. In the 2000s, a drug developed to raise HDL cholesterol, the “good” cholesterol, succeeded at doing exactly that in trials. But when an outcomes trial tracked actual cardiovascular events and deaths, its development was halted after finding no benefit and a signal of harm, despite the improved cholesterol numbers. Whenever a claim leads with a lab number, ask whether anyone has tested whether changing that number changes what happens to real patients.

Sample size, confidence intervals, and “significant” versus “important”

Small studies produce noisy, unstable estimates; larger studies produce more stable ones. Most studies report a confidence interval, a range meant to capture where the true effect most plausibly lies. A wide confidence interval that crosses the point of “no effect” (a relative risk of 1.0, or a difference of zero) means the result is statistically uncertain, even if the headline number looks impressive.

It also helps to separate two ideas that sound similar but are not: statistical significance and clinical importance. Statistical significance (often a p-value below 0.05) means a result is unlikely to be due to chance alone; it says nothing about how large the effect is. For illustration only: a trial of 50,000 people could find a blood pressure drug lowers systolic pressure by 1 mmHg more than placebo, undetectable to any patient, and still call that statistically significant, because a large enough sample can detect even a trivial true effect. A result can be statistically significant and clinically irrelevant at once.

Who funded it, and does that mean you should ignore it

Industry funding is common in medical research, including in some of the best-designed trials in the literature, and funding source alone is not a sound reason to dismiss a result. It is, however, a reason to look more closely: industry-funded trials have historically been more likely to report favorable results for the funder’s product, through design choices, selective outcome reporting, or publication bias. Two practical checks help: whether the trial was preregistered, meaning its design and primary outcome were published, commonly on ClinicalTrials.gov, before results were known; and whether the finding has been independently replicated. A single industry-funded study is a data point, not a verdict.

Where the strongest answers actually live

Any single study is one data point in a larger body of evidence. The most reliable summaries come from systematic reviews and meta-analyses, which formally identify and pool the relevant studies on a question, weigh their quality, and combine results. Cochrane, an international nonprofit, produces systematic reviews under a published, standardized methodology and is widely regarded as a reference standard for this kind of synthesis. The US Preventive Services Task Force performs a similar function for preventive services in the US, reviewing the evidence on a topic and assigning a letter grade, A through D, or I for insufficient evidence, that reflects the strength of the benefit and the task force’s certainty in it.

The trap to watch for is the single splashy study, often small or observational, that generates headlines before it has been replicated or folded into a systematic review. It may hold up, or it may be one of many early findings that quietly fails to replicate once larger studies are run. A finding consistent across a systematic review, or that matches a major guideline body’s conclusion, has already survived scrutiny a single new study has not yet faced.

A short checklist for the next health headline

• Is the number relative or absolute, and what were the actual event rates?
• Randomized trial or observational study, and does the coverage say which?
• A real clinical outcome, or a surrogate marker believed to predict one?
• How large was it, and did the confidence interval rule out no effect?
• Who funded it, was it preregistered, and has it been replicated?
• One study, or consistent with a systematic review or guideline body?

None of this replaces a conversation with a clinician about a specific decision. It is enough, though, to keep from being swept along by any single week’s headline.