Reproducibility’s new rule of three

I’ve been reading a lot lately about the reproducibility crisis in science, and I think I've found a novel solution. It’s an issue that continues to bubble away in our industry, an elephant in the lab, and we really should be more worried about it than we seem to be. One paper suggests that less than a third of scientific results are reproducible, rendering the remaining two-thirds of dubious value.

That really is quite shocking. If you went to the supermarket and bought six tins of beans, and found that only two had any beans inside, you wouldn't shop there again.

There’s an adage in advertising that half of all money spent is wasted, but nobody knows which half. In science, two-thirds may be wasted and nobody knows which is the good and which is the hokum. The acid test of scientific reproducibility is to reproduce it – but it is such a slow process. New research takes an absolute aeon to get published in the first place, what with all the internal assessments, peer review, and everything. To then start all over again is frankly too boring to contemplate.

The way around this is to multiplex from the start, with everything done in parallel. Lab instrument and equipment manufacturers will be overjoyed at the prospect of making multiple sales with each order, and the whole sector will become a national economic powerhouse as a result. Better still, experiments will not simply be doubled but trebled at least, because of simple redundancy requirements.

So every lab, and every experiment, will henceforth be conducted in triplicate. If all three outputs are similar, within tolerances, what we have is good science. If two of them agree, there is at least some confidence in the result. If all three differ significantly, it’s a bad job but at least we know that now rather than years in the future.

Lab instrument and equipment manufacturers will be overjoyed at the prospect of making multiple sales

This tri-science approach might seem expensive, but will actually prove very cost-effective in the medium term.

But there’s a twist, and I’m afraid you might not like it. We're all used to the growing presence of automation in the lab, and the relatively new focus on AI and machine learning. Increasing the use of these, you might think, would be a reliable way to improve reproducibility, but that is only partially true.

The thing is, science remains something of an analogue phenomenon. Bioscientists in particular will be aware that, sometimes, the goo in the Petri dish just behaves oddly. Cell lines might be in a cooperative mood one day, and not the next. These things mean that science will always have fuzzy edges, and that needs humans.

Great, you say. I’m human. I can keep my job. But the human factor remains the variable most likely to generate unreproducible results. One study found that the lab equipment, location, climate and time of year all resulted in less variation in results than did the identity of the people doing the experiment.

It’s a conundrum. Science needs humans but humans make erratic science. This is the clever bit. Rather than employing lots of different scientists and technicians, and purely in the interests of reproducibility, in future all experiments will be performed by the same ones.

How is that even possible? Well, it might take 18 to 20 years before our new clone army is ready for full deployment, but think of the benefits!