Forensic Science – Whither Quality?

To find the best way forward in the new Forensic Science landscape, we must understand and address the mistakes of the past says Professor Bob Flanagan… 

‘The best practitioners – of whatever art – are usually poorer than the bunglers, because their work takes them longer. They prefer to demonstrate their art in a single piece, while the others turn out six or seven, pleasing most people, since time is of the essence and a lower price all-important’.

Sound familiar? Well the words are those of Nicholas Hilliard (c.1547–1619) Treatise on the Arte of Limning ¹(c. 1600), p. 107. Today caveat emptor (‘buyer, beware’) is something we are all familiar with in everyday life, but we can only take our prescription medicines, eat our purchased food, use public transport or mechanical or electrical devices with confidence because of the massive regulatory framework that underpins modern, Western societies. Once we step into the unknown, be it with food gathered in hedgerows, herbal remedies, or illicit drugs, then ‘buyer, beware’ becomes all too true.

The origins of modern forensic science lay in the chemical detection of poisons such as arsenic and mercury in biological specimens, usually stomach contents or viscera. But from the earliest days there was debate as to what to make of it all. The cases of Madame Lafarge in France (1840), in which the father of forensic toxicology Mathieu Joseph Bonaventure Orfila (1787–1853) became embroiled, and in England that of William Palmer, the ‘Rugeley Poisoner’ (1856), that featured Orfila’s pupil Alfred Swaine Taylor (1806–1880), are just two well-known examples. In the Lafarge case, Orfila maintained that arsenic was present in samples from her husband’s body and conviction for murder duly followed despite failures by local chemists to perform the then new-fangled Marsh test properly. Taylor, on the other hand, faced the difficult task of defending a diagnosis of strychnine poisoning on the basis of clinical features alone, there then being no way of testing for it in the samples available. Shades of the attempted murder of Vladimir Kostov in Paris and the murder of Georgi Markov in London in 1978 – no poison was ever identified, although by a process of exclusion ricin remains the likely suspect.

Nowadays forensic science has ventured into hitherto undreamt of areas such as personal identification, an area that has produced some remarkable results. Most notable of course is that of the identification of the remains of King Richard III found beneath a car park in Leicester. But in criminal investigations, whilst often serving to exclude a suspect, there have been dramatic mistakes. The ordeal of Adam Scott (2012), who was arrested over an attack in Manchester despite never having been there, and who spent five months on remand because his DNA was mixed accidentally with samples from a complainant whilst undergoing analysis at LGC Forensics, is a case in point.

What happened was that samples from the complainant – high vaginal, low vaginal and vulval – were submitted, in duplicate. All six showed sperm and all gave a profile that included the complainant’s boyfriend, but one swab showed not only the boyfriend, but also gave a match with Scott. It was not recognised as unusual that these results were at odds with the finding on the duplicate swab, taken from the same site. Further, another (biased) assumption was made, i.e. that the DNA matching Scott must be from sperm. Luckily police continued to question the match, based on other information, but even that did not prompt any retesting until police paid for a retest. It was only then that re-extraction of DNA from the original swabs was performed and the absence of Scott’s DNA revealed.

Human error was blamed, although ‘the procedures themselves were not adequate, leading to no records being maintained by the technicians and nothing done to mark used trays. These errors were compounded by the failure at LGC to consider the possibility of contamination despite concerns expressed by the investigating officer’, according to regulator Andrew Rennison.² Buyer, beware indeed. But even before the error in the Scott case was discovered, the underlying technical process error had been identified, but was neither properly investigated, nor thought serious.

Things are no better in the land of opportunity. In 2012 a chemist who had worked in a Massachusetts Department of Public Health state laboratory was arrested for allegedly falsifying evidence used in criminal cases, a revelation that has prompted calls for major forensic science reform in the US.³ Annie Dookhan had claimed to have tested about 60,000 samples involved at least 34,000 criminal cases during a 10-year career. Despite her work rate far exceeding that of other employees, her supervisors never suspected that she was ‘dry-labbing’ (simply guessing as to the contents of) submitted packets. On conviction she is now serving a 3–5 year jail sentence, but the ramifications rumble on. Retired judges have had to be recruited to preside over special investigations and the laboratory has been shut down. The whole thing should never have happened, but no-one thought to check her claimed qualifications when she was first employed.

[caption id="attachment_42386" align="alignright" width="320"] Mathieu Orfila, pioneer of forensic toxicology[/caption]

Failures in regulation of course underlie two of our own major criminal disasters, those of nurse Beverly Allitt and GP Harold Shipman. Indeed, the ramifications also rumble on, although the recommendations of the Shipman Inquiry have been so watered-down that some question whether anything has really changed. There are clear parallels with the case of Shipman and that of another likely serial killer, John Bodkin Adams (1899–1983), tried and acquitted for the murder of one patient by administering diamorphine in 1957. In part the acquittal was attributed to the fact that Dr Arthur Henry Douthwaite (c. 1896–1974) was prepared to say that murder had definitely been committed, but changed his mind in the middle of his testimony regarding the exact date.⁴ Douthwaite was Britain’s leading expert on dangerous drugs, and had been instrumental in dissuading the Home Office from banning diamorphine for medical use (and was also much interested in the medical use of cannabis). There remains much speculation as to the role of political influence in the trial, one suspicion being that the medical profession as a whole were alarmed at the prospect of further prosecutions when patients died after being given strong analgesics to relieve suffering, and indeed Douthwaite was in turn passed over for the post of President of the Royal College of Physicians.

This tight-rope is one that many clinicians today face of course. Drugs such as diamorphine, fentanyl, and methadone are potent respiratory depressants, and if patients die unexpectedly then in this post-Shipman era there is a rush to ‘do toxicology’. But this is not as simple as it sounds. Although it is thought best to take blood from a peripheral, usually femoral, vein by needle aspiration after ligating the vein proximally, it is not generally appreciated that this procedure is rarely followed, and indeed may not be possible if there has been trauma, for example. Even if the correct procedure is followed, other changes such as those brought about by prolonged attempts at resuscitation, or by post-mortem decomposition, may serve to alter the composition of the blood sampled from that circulating at the time of death. And then there is the role of the laboratory. Analyses based on a combination of immunoassay and gas chromatography and/or liquid chromatography usually linked to mass spectrometry (single or triple quadrupole, or nowadays increasingly accurate mass) are powerful qualitative tools, but don’t pick up potent drugs such as lithium, and are next-to-useless if inorganic poisons, herbal toxins or pesticides are involved. In these latter cases, suspicion and case history are all important, and the investigation itself becomes a research project in its own right requiring experienced staff with both a well-resourced laboratory and time to do the work. But how can you quality assure experience and common sense, let alone put a value on it?

In difficult cases qualitative identification of a poison and/or circumstantial and clinical evidence alone may be adequate. In that of Alexander Litvinenko, for example, identifying ²¹⁰Po was all that was needed to establish the diagnosis. However, when prescription drugs are suspected then a qualitative analysis is often unhelpful except if non-adherence can be demonstrated, and the focus turns to the ‘level’. And here the fun begins because there is then a conflict, in my opinion, between the demands of regulators for conformance to accreditation schemes based in reality on industrial processes on the one hand, and the ‘real death’ situation on the other. In industry, not only are the analytes known and likely limited in number, but also the matrices can be sampled reliably and reproducibly. This is most definitely not the case in forensic toxicology, where inspection and accreditation of laboratories is thought to ensure quality (it doesn’t, as discussed above) and the rest, the pre- and post-analytical phases, often the most crucial part of any investigation, are left to guidelines or local policies that are neither inspected, nor enforced.

What is the point, for example, of asking for uncertainty data on a benzodiazepine immunoassay at the lowest limit of quantitation (LLoQ) when it is uncertain what is actually being measured in the first place? It is surely enough to demonstrate that the assay is being used in accord with the manufacturer’s guidelines and that if additional sample preparation has been added then such additional procedures are ‘fit for purpose’. As to the inevitable in-house chromatographic assays, there has to be a balance between use of ‘traceable’ (and expensive) reference compounds and extensive (impossible as far as matrix-matching is concerned) assay validation for rarely encountered analytes on the other. Far, far better to have a sensible investigation pursued in collaboration with others such as the pathologist and coroner’s officer or police that gives an idea of the duration and magnitude of any exposure to a drug or other poison. There are so many other variables that affect interpretation of ‘the level’ in post-mortem work that proper identification of what is being measured takes priority, with absolute accuracy of quantification following some way behind.

And what of basics such as duplicates? Somewhere not-so-far-away not-so-long-ago there was what police were convinced was a suspicious death in a substance user. The coroner trusted the analysis to his local hospital laboratory, who duly found and reported the presence of the specific diamorphine metabolite acetylmorphine in urine, but could find no morphine in the blood (claimed limit of sensitivity 0.01 mg/L). The Home Office registered pathologist simply looked at the blood morphine result (‘the level’) and duly reported an alternative cause of death. Police then took the blood sample to their ‘preferred provider’, who reported a blood free (unconjugated) morphine concentration of 0.33 mg/L, clearly pointing to heroin (illicit diamorphine) use as the cause of death there being no other sustainable cause of death. What’s this cried the defence, the heroin supplier, who had been under police surveillance prior to the event in question, having been charged with manslaughter? Who is correct? So the trial judge ordered a second analysis from the police provider, which came in at 0.17 mg/L. Further confusion. It seems that the police provider had not done duplicates on either occasion – anyone who has tried to pipette post-mortem blood knows that it’s not easy to do reliably for starters, although in this instance an error in calculation was also possible. What should have been done was a duplicate analysis (at least, there was plenty of sample), report of the average and range, and a note that the clinical interpretation was the same no matter what the ‘level’. A third laboratory, by then part of the Forensic Science Service, reported 0.13 mg/L, and sent synthetic quality assurance material to the other two; the Coroner’s provider still got it dramatically wrong.

When asked to do a report on the fiasco, which by then had been thrown out by the Court (unjustifiably in my opinion – the cause of death was still heroin poisoning no matter what the ‘level’), I simply pointed to the lack of understanding of clinical toxicology by many pathologists on the one hand, and ‘buyer, beware’ as regards the police decision to go to a private provider on the other. Cheapest is often cheap for a reason, be it simply maximising profit, or marginal costing to grab work (a loss leader in commerce). Yes there are economies of scale, but if the whole thing is simply looked upon as a production line then procedures may be not always be either appropriate, or followed correctly, and sometimes crucial detail gets lost somewhere along the line. Although all involved in forensic investigations must pay attention to costs, financial considerations must not negate the absolute requirement for good science.

I have accumulated many other examples of serious errors by UK forensic toxicology providers over the last 15 or so years, let alone inappropriate interpretation placed on the results. It is not widely appreciated that laboratory results, because they are ‘fact’, often take on a life of their own after they leave the laboratory. Some mistakes have been simply careless (liver chloroform reported at 1 g/kg not 1 mg/kg, ‘neat vodka contains 40 mg/100 mL ethanol’), others more fundamental (misidentification of a clozapine metabolite for quetiapine, a prescription only medicine, in a patient under Mental Health Act detention not prescribed quetiapine – can it really be thought that liquid chromatography with diode array detection alone is good enough for post-mortem work with biological samples these days?). In one instance a 20-fold error in a blood amisulpride concentration was eventually traced to an error by a technician in preparing calibrators, seemingly a clear failure in laboratory practice because the FDA laboratory guidelines are that calibrators and internal quality control (IQC) solutions should be prepared independently by separate analysts, even though the use of independent stocks of certified reference material is almost impossible.⁵ The FDA guidelines don’t mention replicates by the way because the pharmaceutical industry will not tolerate such an extravagance! Whither quality indeed.

The cause of the underlying problem is clear to me at least – over-reliance on ‘test ordering’ and the desire to cut costs at all costs in the belief that an analysis ‘for poisons’ is the same no matter who does it. Is the remedy more regulation? No, regulation clearly fails time and time again, as in the case of Annie Dookhan, who simply falsified the records until in the event she didn’t cover her tracks properly, by which time incalculable damage had been done – $8.5 million of public money in 2013 alone according to one source, and the costs continue to mount.⁶ The reason given by government for closing the Forensic Science Service was that it was ‘losing’ £2 million a month, but what other costs have now emerged? What is needed are systems that combine public service and professional integrity with research and development. Appropriate laboratory regulation and inspection, by inspectors with a background in the specialty being inspected, if such can be found, has a part to play, but cannot be allowed to be the sole player. Personal training and certification, and continuing professional encouragement and development, are also vital if there are not to be further serious failures.

The author:

Prof RJ Flanagan PhD ERT MCSFS CChem FRSC FRCPath HFCMHP

Toxicology Unit

Clinical Biochemistry

King’s College Hospital

 References:

1. Painting in water colour. Hilliard, England’s greatest Renaissance painter, was always broke
2. Rennison A. Report into the circumstances of a complaint received from the Greater Manchester Police on 7 March 2012 regarding DNA evidence provided by LGC Forensics. Available at: https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/118941/dna-contam-report.pdf
3.  Drahl C. Inspector General’s Report Labels Annie Dookhan ‘Sole Bad Actor’ In Massachusetts Crime Lab Scandal. Probe finds major management problems and lack of forensic chemistry training. Chem Eng News 2014; 92, Issue 10: 6
4.  The trial was notable as the first in England & Wales to be decided solely on expert evidence. The principal witness for the defence was Dr John Bishop Harman (1907–1994), a physician at St Thomas’ Hospital, and father of Harriet Harman MP
5. U.S. Department of Health and Human Services. Food and Drug Administration. Center for Drug Evaluation and Research (CDER). Center for Veterinary Medicine (CVM). Guidance for Industry. Bioanalytical Method Validation, May 2001. Available at: http://www.fda.gov/downloads/Drugs/Guidances/ucm070107.pdf
6. http://www.boston.com/news/local/massachusetts/2013/11/22/annie-dookhan-former-state-chemist-who-mishandled-drug-evidence-agrees-plead-guilty/lhg1mwd9U3J8eh4tNBS63N/story.html