← All articles Research

How to Read a Health Headline

A one-minute skill: who paid, what was measured, and how big the news really is

9 minute read

In 2015, a story raced around the world: a glass of red wine was as good as an hour at the gym. It came from a respectable university and a respectable journal. The study itself used no wine, no people and no gym. It fed rats a purified plant compound, resveratrol, alongside twelve weeks of treadmill training, and those rats ran further.1 The lead researcher later estimated that getting a matching amount from wine would take 100 to 1,000 bottles a day.2 Reading a health headline is a skill, and once you know what to ask, it takes about a minute.

(As for the wine: when researchers corrected for the fact that many "non-drinkers" in the older studies were former drinkers in poor health, the apparent benefit of low-level drinking disappeared.3)

Where the spin begins

It is tempting to blame journalists, but the story often starts earlier. Researchers at Cardiff tracked 462 press releases from 20 leading UK universities, along with the papers behind them and the news stories that followed. Between a third and two-fifths of the press releases already went beyond the research: advice the paper did not give, cause claimed from a mere association, or animal findings applied to people. When a press release exaggerated, 58 to 86 per cent of the news stories carried the same exaggeration. When it did not, only 10 to 18 per cent did.4 So the first habit is simple: look for the study itself. Most news reports name the journal, and the summary, called the abstract, is usually free on PubMed.

Who paid?

Funding is declared near the end of a paper, and it is always worth a glance. In an analysis of 206 studies and reviews of soft drinks, juice and milk, not one of the intervention studies wholly funded by industry reached an unfavourable conclusion, compared with 37 per cent of those with no industry money.5 In the 1960s, the sugar industry quietly paid for an influential review in the New England Journal of Medicine that played down sugar's role in heart disease and pointed at fat; the funding was not disclosed.6

A funder does not make a study wrong, and independent work can be poor. Treat funding as a reason to read more closely, and apply the same test to everyone, including a supplement company reporting on its own capsule. (For how funding shapes the wider evidence, see Beyond the Gold Standard.)

What was actually measured?

This is where most headlines come apart, and there are two traps.

Measuring so many things that something turns up. In 2015, a science journalist ran a real trial to show how easily this is done. He gave a small group a low-carb diet plus daily dark chocolate, then published the result, with chocolate billed as a "weight-loss accelerator". The headlines followed around the world. His secret was 15 people and 18 different measurements. Test that many things in that few people and chance will almost always hand you something that looks significant.7 So ask how many people took part, and whether the headline finding was what the study set out to test.

Measuring a stand-in for what you care about. A blood marker or a scan is a surrogate: a useful signal that is easier to measure than the thing we really want to know. Sometimes it misleads. After a heart attack, irregular heartbeats were known to predict sudden death, and drugs that suppressed them looked like an obvious answer. When a trial finally checked, 63 people died on two of those drugs compared with 26 on placebo, over about ten months.8 Food has its own version: when the full data from Sydney's diet-heart trial were recovered, men who swapped saturated fat for a seed oil had lower cholesterol, and more of them died.9 (See Tipping the Balance.) Markers are valuable, and we use them in clinic every week. But always ask: did people live longer, feel better or stay well?

The number improved. The patient did not.

Relative or absolute?

"Raises your risk by 32 per cent" sounds alarming. The essential question is: 32 per cent of what?

Take bowel cancer and meat. In a UK study of around half a million adults, people eating about 79 g a day of red and processed meat had a 32 per cent higher risk than those eating under 11 g. Cancer Research UK put that in plain numbers: over the seven or so years of the study, 45 in every 10,000 of the lowest eaters were diagnosed, with 14 more among the higher eaters.10,11 Both figures are true. The relative figure tells you the direction and strength of the effect; the absolute figure tells you what it means for one person. It remains a good reason to keep processed meat off the plate, on calm grounds rather than panicked ones.

Headlines made a similar leap in 2015, when processed meat joined tobacco in the World Health Organization's Group 1 list of carcinogens. That group describes how strong the evidence is that something causes cancer. It does not measure how likely cancer is, or say that bacon is as dangerous as cigarettes.12,13

Getting this wrong has real costs. In October 1995, British women were warned that newer contraceptive pills carried twice the risk of blood clots. Later published figures put the risk at around 15 per 100,000 women a year on older pills and 25 on newer ones, against 5 in women taking no pill.14 Use of the newer pills collapsed from 53 per cent to 14 per cent of pill users.15 Many women stopped or switched in alarm, unintended pregnancies rose, and the cost to the NHS of the extra abortions alone was put at around £46 million.16 Whichever way you view that medicine, the women of 1995 deserved both numbers.

Mouse, cell or person?

"Could", "may" and "promising" often mean that nobody has tested the idea in a person yet. Of 101 highly promising laboratory discoveries published in leading journals around 1980, 27 had reached a human trial twenty years later, five had been licensed, and only one was in wide use.17 Even among 76 highly cited animal studies, about a third were later confirmed in human trials, 18 per cent were contradicted, and the rest were never tested.18

Laboratory work has real value, since it shows how things might work, and often explains why a long-used food or herb has the effects people have reported for generations. That is a very different position from a new molecule with no human history. Herbs have been tested in use for centuries; the right one still depends on your constitution, so see a practitioner rather than self-prescribing on the strength of a headline. The test is simply honesty: a study in a dish should be described as a study in a dish.

One study, or the whole body of evidence?

Two researchers once opened a cookbook, chose 50 common ingredients at random and searched for studies linking each with cancer. Eighty per cent had been studied. Single studies reported striking effects in both directions, and when the same foods were examined in pooled analyses, most of those large effects shrank away.19 This does not mean nutrition research is worthless: it means one study is a single voice in a long conversation. It happens with medicine too. Of 45 highly cited clinical studies claiming a treatment worked, later research contradicted seven and found weaker effects for seven more.20

So look for what the story says about earlier work. Does this finding agree with other studies, in different populations, with a mechanism that makes sense? Studying real diets over a lifetime is complex, and the links it uncovers are often the only evidence we can ethically have. The harm from smoking was established in exactly this way, from one careful British study in 1950 and the many that followed.21 When a pattern keeps appearing, by different methods and in different places, take it seriously. When "new research overturns everything", take a breath.

The one-minute headline check

  • Who was studied? People, animals or cells? How many, and are they like you?
  • Who paid? Does the funder gain from the result?
  • What was measured? Something you care about, or a stand-in? One planned question, or a long list?
  • How big is it? If the headline gives a percentage, what are the actual numbers?
  • What kind of study? Did it follow people, or test one thing against another? Does the headline claim more than the design can show?
  • Is it alone? What do other studies, and the reviews that pool them, say?
  • What is it asking of you? Be especially careful if it's selling something, or suggesting you drop something that matters.

Above all, a headline is never a reason to stop a prescribed medicine or to begin a new supplement. It is a reason to ask a good question of someone who knows you.

In short

  • Exaggeration often begins in the press release, so go to the study itself where you can.
  • Check who funded it, how many people took part and whether it measured something that really matters to health.
  • Always ask for the absolute numbers behind a percentage.
  • Animal and cell studies are leads. Treat them as a beginning.
  • Trust a pattern across many studies more than a single surprising result.

At Rootwise, we enjoy these questions. If a headline has left you wondering what it means for your own health, bring it to a consultation and we will look at the evidence, and at you, together.

References

  1. Dolinsky VW et al. Improvements in skeletal muscle strength and cardiac function induced by resveratrol during exercise training contribute to enhanced exercise performance in rats. Journal of Physiology, 2012.
  2. Goh SL. Can drinking wine replace the gym? Global News, 2015.
  3. Stockwell T et al. Do "moderate" drinkers have reduced mortality risk? A systematic review and meta-analysis of alcohol consumption and all-cause mortality. Journal of Studies on Alcohol and Drugs, 2016.
  4. Sumner P et al. The association between exaggeration in health related science news and academic press releases: retrospective observational study. BMJ, 2014.
  5. Lesser LI et al. Relationship between funding source and conclusion among nutrition-related scientific articles. PLoS Medicine, 2007.
  6. Kearns CE, Schmidt LA, Glantz SA. Sugar industry and coronary heart disease research: a historical analysis of internal industry documents. JAMA Internal Medicine, 2016.
  7. Bohannon J. I fooled millions into thinking chocolate helps weight loss. Here's how. io9, 2015.
  8. Echt DS et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo: the Cardiac Arrhythmia Suppression Trial. New England Journal of Medicine, 1991.
  9. Ramsden CE et al. Use of dietary linoleic acid for secondary prevention of coronary heart disease and death: evaluation of recovered data from the Sydney Diet Heart Study. BMJ, 2013.
  10. Bradbury KE, Murphy N, Key TJ. Diet and colorectal cancer in UK Biobank: a prospective study. International Journal of Epidemiology, 2020.
  11. Cancer Research UK. Bacon, sandwich ham and salami: how does processed and red meat cause cancer and how much matters? Cancer News, 2024.
  12. Bouvard V et al. Carcinogenicity of consumption of red and processed meat. The Lancet Oncology, 2015.
  13. World Health Organization. Cancer: carcinogenicity of the consumption of red meat and processed meat (questions and answers, online).
  14. Mayor S. Department of Health changes advice on third generation pills. BMJ, 1999.
  15. Farmer RD et al. Effect of 1995 pill scare on rates of venous thromboembolism among women taking combined oral contraceptives: analysis of general practice research database. BMJ, 2000.
  16. Furedi A. The public health implications of the 1995 'pill scare'. Human Reproduction Update, 1999.
  17. Contopoulos-Ioannidis DG, Ntzani E, Ioannidis JP. Translation of highly promising basic science research into clinical applications. American Journal of Medicine, 2003.
  18. Hackam DG, Redelmeier DA. Translation of research evidence from animals to humans. JAMA, 2006.
  19. Schoenfeld JD, Ioannidis JP. Is everything we eat associated with cancer? A systematic cookbook review. American Journal of Clinical Nutrition, 2013.
  20. Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA, 2005.
  21. Doll R, Hill AB. Smoking and carcinoma of the lung: preliminary report. BMJ, 1950.

This article is for general information and isn't a substitute for personal advice. If you take prescribed medicines, never stop or change them because of something you have read: speak to your prescriber first.

Beyond the headlines

A personal look at your health and your questions, with evidence weighed carefully and applied to you.