The Bad Science framework in outline
Ben Goldacre’s 2008 book Bad Science and his long-running Guardian column codified a popular-reference framework for identifying unreliable medical claims in public discourse. The framework is not specific to academic research; it applies to any clinical claim presented as authoritative, including expert evidence in criminal trials.
The framework’s principal warning signs:
- Retrospective pattern-matching without pre-registration.
- Absence of a control group or comparison population.
- Hypothesis-first reasoning rather than hypothesis-testing.
- Absence of peer review on the methodology.
- Heavy reliance on narrative over data.
- Confident inference from weak or ambiguous evidence.
- Selective reporting of evidence that fits the hypothesis.
Pattern 1: retrospective pattern-matching
The Crown’s case started from a pattern (Letby’s shift-overlap with cluster events) that had been identified retrospectively. The cluster was defined, retrospectively, as the events where Letby was present. The cases were then examined for features consistent with the suspect hypothesis. This is the retrospective-pattern-matching warning sign in its clearest form.
A scientifically-rigorous investigation would have pre-registered the hypothesis before examining the evidence. That did not happen.
Pattern 2: absence of control
No control group of similar-acuity preterm collapses on comparable UK NICUs without Letby present was adduced at trial. Without a control, there is no baseline against which to measure whether the indicted collapses are anomalous for the unit and the patient mix.
The Panel’s subsequent review implicitly supplies the missing control: other UK NICUs with similar acuity mix and staffing conditions produce similar mortality patterns without individual-actor explanation. The Countess of Chester was within range.
Pattern 3: hypothesis-first reasoning
Dr Evans’s causation methodology worked from the suspect hypothesis toward confirming findings. Each case was examined to identify features consistent with deliberate harm; features inconsistent with it were not given equal weight. This is the hypothesis-first warning sign specifically.
A hypothesis-testing methodology would have specified what evidence would refute the hypothesis and searched for it with equal rigour. The Panel’s differential-diagnosis methodology does this.
Pattern 4: absence of peer review
Dr Evans’s methodology was not published in peer-reviewed form. The skin-sign diagnostic framework he applied to air-embolism cases had no peer-reviewed supporting literature beyond the Lee 1989 paper — which Dr Lee himself has publicly said does not support the trial application.
Pattern 5: narrative heaviness
The Crown’s closing speech was heavily narrative: the story of a nurse who harmed babies, connected to the evidence through inferential bridges. Narrative argumentation is not evidence; it is a structure for presenting evidence. Goldacre’s framework warns specifically against treating narrative persuasion as evidential weight.
Pattern 6: confident inference from weak evidence
The Crown’s framing presented ambiguous evidence with high confidence. Skin signs that do not meet Lee 1989 criteria were presented as diagnostic of air embolism. Roche Cobas screening results were presented as forensic proof. Shift- rota overlap was presented as statistical demonstration of guilt. Each is confident inference from weak or ambiguous evidence — the pattern Goldacre’s framework specifically warns against.
Pattern 7: selective reporting
The shift-rota chart’s selection of events, the curated Facebook-search subset, the specific Post-it-note phrases highlighted out of the full-document context — each is an instance of selective reporting. The pattern is that evidence fitting the hypothesis is reported; evidence not fitting is suppressed or de-emphasised.
What this means for conviction safety
The Goldacre Bad Science framework is a popular-reference version of the formal EBM framework. The formal version (Heneghan, Oxford CEBM) and the popular version (Goldacre, Bad Science) reach the same conclusion on the Letby evidence: the methodology fails the criteria. A conviction on methodology that fails the Goldacre test is a conviction on a weak evidential foundation. The Cannings principle applies.