What EBM is
Evidence-based medicine is the formal framework for evaluating clinical evidence: study design, sample size, blinding, confounding, reproducibility, peer-review. It is the framework the NHS uses to decide whether a treatment works, whether a diagnostic test is reliable, whether a clinical claim is supportable. The Oxford Centre for Evidence-Based Medicine (CEBM) is its flagship UK institution.
EBM is not specific to criminal trials, but its criteria apply to any clinical claim — including claims about cause of death, diagnostic interpretation of clinical signs, and the evidential weight of expert opinion.
The EBM hierarchy of evidence
- Systematic reviews and meta-analyses of randomised controlled trials.
- Randomised controlled trials.
- Cohort studies.
- Case-control studies.
- Case series.
- Expert opinion, case reports, anecdote.
The hierarchy is not about what evidence is useful; all levels have their place. It is about what evidence carries what weight. Expert opinion (level 6) carries less weight than controlled studies (levels 1–2). A criminal conviction based on level-6 evidence, without supporting higher-level evidence, is evidentially vulnerable.
Where the Letby trial evidence sits on the hierarchy
The Crown’s causation evidence at trial was:
- Level 6. Dr Evans’s expert opinion on cause of death, based on retrospective unblinded case review.
- Not adduced. No level 1–5 supporting evidence: no peer-reviewed case-control study showing the alleged mechanisms are detectable with specific clinical signs at the claimed specificity; no cohort data showing the alleged mechanisms occur at a population-rate incompatible with the cluster.
The Shoo Lee Panel’s review, conducted to modern EBM standards, is also level-6 evidence. But it has specific methodological features the Crown’s evidence lacked: blinded case assignment, structured differential-diagnosis, cross-specialist consensus. Within the level-6 category, it is substantially more methodologically robust.
The four EBM failures of the Crown’s methodology
1. Retrospective pattern-matching without pre-registration
In EBM terms, retrospective pattern-matching against selected cases without pre-registration of the hypothesis is the lowest-quality form of observational evidence. The Crown’s expert worked from a suspect-identified cluster toward confirming findings, without having pre-specified what evidence would confirm or refute the hypothesis.
2. Absence of control group
Case-control studies require a control group matched on relevant characteristics. A proper EBM analysis of the Letby cluster would compare the indicted cases to a matched set of similar-acuity preterm collapses and deaths on comparable UK NICUs over the same period, to identify what features are specific to the indicted cluster rather than shared with ordinary NICU experience. No such control analysis was adduced.
3. Hypothesis-first reasoning
EBM requires hypothesis-testing, not hypothesis-confirming. The Crown’s expert methodology worked from the hypothesis (Letby as cluster cause) toward confirming findings. A hypothesis-testing EBM methodology would have formally specified what evidence would falsify the hypothesis, and looked for that evidence with equal rigour to the confirming evidence.
4. Non-peer-reviewed methodology
EBM requires peer-reviewed methodology. The Crown’s expert methodology was not published in peer-reviewed form. The Panel’s methodology, by contrast, has peer-reviewed-layer support; the Joint Insulin Report draws on peer-reviewed literature; the paediatric-pathology re-readings are within established peer-review frameworks.
The Heneghan / Goldacre public critique
Heneghan at the Oxford CEBM and Goldacre in his public-engagement work have each applied the EBM framework to the Letby evidence publicly. Neither is a fringe voice; they are the institutional UK EBM community. Their public commentary is that the Crown’s evidence does not meet EBM standards. This is not specialist taste; it is professional EBM judgment.
Why this matters for the CCRC review
The CCRC, in considering the safety of the Letby convictions, can weight the evidence by EBM standards. A conviction on level-6 evidence from a non-peer-reviewed methodology that fails the four EBM tests, where independent methodologically-sound replication (the Panel) reaches the opposite conclusion, is a conviction on evidentially-weak ground. The Cannings principle applies; the EBM framework is the specialist operationalisation of why.