Killer Questions

Why we must be careful when questioning Lucy Letby's conviction

I listened to Radio 4’s File on 4 programme: “Lucy Letby: the killer questions” last night. The carefully thought-out programme presented testimony from experts in different areas, from neonatal medicine to statistics, each of whom asked pertinent questions about the way the trial was conducted and pointed out possible places in which mistakes could have been made.

We also heard from a key expert witness at the trial, Dr Dewi Evans, who had this to say about people questioning the safety of Letby’s conviction. “I’m very, very disappointed with the reaction and the fall out following the verdict. I’m talking about people who have had no access to the clinical notes, who were not present at the trial, who have not seen the witness statements, who have not heard from the parents. And yet there are people out there who are challenging the verdict and I think they have been grossly irresponsible. The disregard that the so-called experts have had for the families is quite frankly disgraceful.”

This is not a view I can fully agree with. As a scientist I seek to welcome people questioning my methods and my results. Added scrutiny should help to strengthen scientific results, and if they are incorrect to help over turn them. This doesn’t mean that having your results questioned or scrutinised is always an easy or a pleasant process. Any scientist who has been through peer review to get an article published in a journal knows how demoralising critical or negative reviews can be and the visceral and defensive reaction that can evoke upon first reading them.

I can understand why a paediatrician expert witness like Dr Evans who has helped police throughout their investigation and acted as the key expert witness for the prosecution is so invested in the outcome of the trial. However, his privileged position does not give him the right to shut down the debate even if he feels unfairly maligned at the hands of his critics.

A variety of independent experts made excellent critiques of the fragility of some of the evidence in the Letby trial. In particular, Professor Jane Hutton, a professor of medical statistics at the University of Warwick raised issues around the way in which statistics were used at the trial.

Hutton highlights the, now infamous, chart which showed that Lucy Letby was on duty each time one of the crimes of which she was accused occurred but that none of the other nursing staff were. On the face of it, it seems like quite a damning bit of evidence. But when you think about it, it’s entirely unsurprising that Letby’s column is the only one full of ticks. Any of the events at which she was not present she would not have been charged with and so they wouldn’t appear on the chart. If you made the a similar chart, this time including only those deaths at which another member of the nursing staff was present, then it’s entirely possible (for example if they were present a deaths at which Letby was not) that their name would be above the only column full of crosses and not Letby’s. Indeed, it later transpired that the table did not include six other deaths which occurred during the period with which Letby was not charged. The jury were not told about these other deaths.

The infamous chart that was shown to the jury indicating that “whenever Lucy Letby was present, Lucy Letby was on present”!

As Hutton puts it in the show, “If you want to find out what went wrong, you need to consider all deaths, not just a subset of them”. She also points out that it’s important to consider how likely the other alternative causes of death were on the struggling Countess of Chester neonatal unit. Undoubtedly the probability of so many deaths in a short period of time would have been quite low even on a unit which was failing. At first glance this might seem to make the alternative explanation of murder seem more likely. However, it’s also worth bearing in mind, when weighing up the evidence, that multiple infant murders are also extremely unlikely. Weighing these two very unusual events against each other is not an easy thing to do.

To treat both sides of the debate equally, there were also some dubious arguments made in the show that don’t really hold mathematical water. Talking about the large amount of circumstantial evidence against Letby (since no-one actually saw her commit a crime) barrister Gudrun Young KC, puts forward an argument about the dangers of adding multiple pieces of weak circumstantial evidence together. The defence lawyer describes the way she often summarises it to a jury as, “You might get eight pieces of evidence, but if … each one of them is really weak … eight times zero is zero. It’s not eight.”

This argument isn’t a sound one from a mathematical point of view. Most mathematicians would caution that lots of small independent pieces of evidence can combine to give you something more substantial, but the image conjured up by the terminology “adding up” is the mathematical metaphor.

Think about it this way instead. Imagine you are given eight chances to win £1000, but the probability on each occasion is only 9%. It’s tempting to dismiss the small probability of each individual bet coming in as overwhelmingly unlikely and to ignore it. In fact the probability of winning at least £1000 is over 50%. You can skip the next paragraph if you don’t want to see the maths behind how you calculate this figure.

(To work this out you need first to think of the probability that you don’t win in a single bet. This is 91% or 0.91 – seemingly overwhelmingly likely. In order for all eight of the independent bets to fail this must happen eight independent times with probability 0.91x0.91x0.91x0.91x0.91x0.91x0.91x0.91=0.9⁸=047. So the probability that you don’t win at all becomes less than half (47%) and the corresponding probability that you win at least once is more than half (53%))

Certainly simply dismissing each piece of evidence as a zero and adding these zeros together isn’t the right way to think about the problem. In fact, Young’s argument just goes to illustrate the point that many in the “unsafe conviction” camp will have made when hearing that the defence did not call an expert witness in statistics - that people who are not well versed in mathematics should be very careful when making arguments which rely on mathematics.

I think it’s important that the competing hypotheses about what happened at the Countess of Chester hospital are aired and fully investigated. In the past, shutting down the whole debate has led to the perpetuation of miscarriages of justice. The infamous case of Sally Clark -  that I wrote about in The Maths of Life and Death – is just such an example.

Clark was convicted for the murder of her two infant sons based on what later transpired to be dubious statistical and medical evidence. Despite clear flaws in the expert testimony and the pathology, her conviction was upheld at appeal and it took four years - until the weight of evidence for her innocence became too great for the conviction to be upheld – for her eventually to be freed in 2003. Despite her overwhelming delight at being reunited with her family, even this reprieve was not enough to make up for the years she had spent wrongfully incarcerated, blamed for killing two of the people she loved most. In March 2007, she was found dead at her home from alcohol poisoning, never having fully recovered from the effects of her wrongful conviction.

To be clear, I am not saying this is what has happened in the case of Letby, but Clark’s case does serve as a stark reminder of the importance of robustly questioning expert testimony. At the same time, those raising doubts should, of course, be considerate of the families who are at the centre of this case. Those families have suffered such terrible losses, unimaginable to those of us who have not been there, and have been forced to carry and relive that trauma for an extended period of time. It is important for everyone involved to acknowledge that suffering and to think carefully about how they make their arguments.