Health reporting frequently describes a study without indicating what kind it was, which is the single most important thing about it.
Preclinical work
Laboratory and animal studies before human testing.
Which establishes plausibility rather than effectiveness.
A large proportion of compounds effective in animals fail in humans, which is why coverage of animal studies as breakthroughs is misleading.
Phase one
Small numbers of participants, primarily assessing safety and dosing.
Which is not designed to demonstrate that a treatment works.
Participants are frequently healthy volunteers rather than patients.
Phase two
Larger, assessing whether the treatment has the intended effect and refining dosing.
Which produces preliminary efficacy evidence.
Many treatments fail here after promising earlier results.
Phase three
Large randomised trials comparing against existing treatment or placebo.
Which is the evidence regulators require for approval.
Randomisation and blinding are what allow effects to be attributed to the treatment rather than to expectation or selection.
Phase four
Post-approval monitoring in wider use.
Which detects rare adverse effects that trials were too small to identify.
Several withdrawals have followed from this stage.
Endpoints
What the trial measures.
Which may be a surrogate marker rather than an outcome that matters to patients.
A drug improving a blood test result may or may not improve survival, and these are different claims.
Publication bias
Positive results are more likely to be published.
Which distorts the overall evidence picture.
Trial registration requirements were introduced specifically to address this.
Reading health coverage
Ask what phase, how many participants, what was measured and against what comparison.
These four answers determine what a study actually established.
Randomisation
Assigning participants to groups by chance.
Which balances known and unknown differences between groups.
It is what distinguishes a trial from an observational study and is the reason trials support causal claims.
Blinding
Concealing which treatment participants received.
Which prevents expectation from affecting both reported outcomes and assessment.
Double blinding conceals it from assessors as well as participants.
Placebo and active comparators
Comparing against no treatment or against existing treatment.
Which answers different questions.
Superiority over placebo does not establish superiority over what is already available.
Statistical significance
Indicates a result is unlikely under the assumption of no effect.
Which is not the same as the effect being large or important.
Effect size and confidence intervals are more informative and are reported less often.
Observational studies
Follow people without assigning treatment.
Which can identify associations and cannot easily establish causation.
Most nutrition and lifestyle headlines derive from observational work, which is worth knowing.
Systematic reviews
Assessments of all available trials on a question rather than of one.
Which is the strongest form of evidence for most treatment questions.
Organisations producing these publish plain-language summaries freely, and they are considerably more reliable than coverage of individual studies.
Registration and protocols
Trials must be registered before starting in most jurisdictions, with the planned analysis specified.
Which prevents changing the outcome measure after seeing results.
Comparing published papers against registered protocols has revealed discrepancies in a meaningful proportion of trials.
Funding and conflicts
Industry-funded trials are more likely to report favourable results, which is documented in meta-research.
Which does not mean they are wrong and does mean funding is relevant context.
Declarations appear in published papers and rarely in coverage of them.
Generalisability
Trial participants differ from typical patients, frequently being younger and healthier.
Which limits how far results transfer.
Under-representation of older adults and of certain groups is a recognised and documented problem.
Reading a health story
Find the study, check the design, the size, the comparison and the outcome measured. The abstract is usually free.
Number needed to treat
How many people must receive a treatment for one to benefit.
Which converts a relative risk reduction into something meaningful.
Preventive treatments frequently have large numbers needed to treat, and this is rarely reported alongside the headline benefit.
Harms
Trials report adverse events alongside benefits.
Which is half of any treatment decision and receives a fraction of the coverage.
Balancing benefit against harm for an individual patient is what clinical judgement actually involves.
A closing observation
The single most useful question to ask about any health claim is what kind of study produced it.
A randomised trial, an observational cohort, an animal study and a laboratory experiment support very different conclusions, and coverage frequently describes all four in the same language.
The distinction is learnable in an afternoon and improves how you read health news permanently.
Guidelines and how they are made
Clinical guidelines are produced by expert panels reviewing all available evidence.
Which is a more reliable guide to practice than any individual study.
Guidelines are published openly, state the strength of evidence behind each recommendation, and are updated as evidence changes.
Talking to a clinician
Bringing a specific question rather than a conclusion generally produces a more useful conversation.
Which most clinicians welcome, whatever the stereotype suggests.