Pistoia Alliance survey warns of ‘AI scientific content crisis’

The global, not for profit body said its survey of 170 science professionals revealed that more than one in four respondents either did not know what content their company’s AI or LLM systems used, or only rely on science titles and abstracts.

Additionally, the poll conducted at the Alliance’s recent US conference revealed barely more than one in three said they had plugged internal documents into their firm’s models.

Respondents also highlighted the lack of shared verification standards as an issue, with 50% citing this as the greatest obstacle to adoption.

Neal Dunkinson, senior director at Alliance partner organisation CCC (Copyright Clearance Center), said conference discussions made it clear that many AI models were not drawing on the rage of scientific evidence required for authoritative results.

“Many organisations are still in a learning phase when it comes to both data and governance and, given the stakes for patient safety, that cannot be ignored,” he said.

“Our poll also showed that 38% of respondents say their copyright and licensing policies are unclear or not enforced, meaning many could also be at risk of fines in an already costly drug development process.”

It was necessary, added Dunkinson, that organisations ensured datasets were “AI-ready” and both properly structured and licenced, as well as transparent.

Pistoia president Dr Becky Upton noted that the organisation’s US and European conferences had both highlighted concerns around AI trust, transparency and skills.

She continued: “By working together on common standards, data quality and practical implementation, the life sciences industry will move forward with confidence. The Pistoia Alliance exists to facilitate this collaboration, and we’re excited to carry these discussions into our spring meeting in London.”

The Pistoia Alliance’s spring conference takes place at the Royal Society of Medicine  from13-16 April next year.