AI in Health Care: A Policy Framework for Innovation, Liability, and Patient Autonomy—Part 9
Large language models are good at summarizing information. They are much less reliable at conveying scientific uncertainty.
Public health debates are increasingly being filtered through artificial intelligence. That becomes problematic when the science itself remains unsettled.
Consider the debate over 7‑hydroxymitragynine (7‑OH), a naturally occurring alkaloid in kratom. Ask several popular AI systems about the compound, and you’ll often get remarkably confident descriptions of its potency and risks. One AI-generated summary I recently encountered described 7‑OH as a “gas station opioid” that is many times more potent than morphine and poses serious overdose risks.
Readers encountering such claims could reasonably conclude that the science is settled. However, the scientific literature presents a more complicated picture.
The debate over 7‑OH has intensified as concentrated products have become more widely available. Kratom, a plant native to Southeast Asia, has been used for generations and is increasingly popular in the United States among people seeking pain relief or alternatives to more dangerous prescription opioids and illicit drugs. The controversy centers on concentrated 7‑OH products, which contain high levels of one of kratom’s active compounds. Critics argue that these products pose substantial risks and warrant stricter regulation. Others contend that public health messaging often overstates the evidence and relies on selective interpretations of the literature. Researchers continue to investigate the compound’s pharmacology, abuse potential, and safety profile.
What the Research Actually Shows
The question of potency alone illustrates the challenge.
The frequently repeated claim that 7‑OH is many times more potent than morphine often traces back to older animal and tissue studies. Yet more recent research on human μ‑opioid receptors has produced a more nuanced picture. In a 2021 study, researchers found that 7‑OH activates opioid receptors differently from conventional opioids and may not fit neatly into comparisons based solely on potency.
Other investigators have emphasized the distinct pharmacology of both 7‑OH and mitragynine, the primary active compound in kratom from which 7‑OH is derived, including signaling pathways that differ from those of many conventional opioids. The work of Kruegel and colleagues helped establish that mitragynine and 7‑OH exhibit atypical opioid receptor signaling, raising questions about simple potency comparisons with conventional opioids.
The discussion of overdose risk is similarly more complicated than many AI-generated summaries suggest. Concentrated 7‑OH products raise legitimate safety concerns, yet fatal overdoses involving 7‑OH appear to be uncommon. Reported deaths often involve multiple substances, leaving researchers to debate what role, if any, 7‑OH played.
The existence of these competing interpretations does not establish that concentrated 7‑OH products are harmless. Legitimate concerns remain about dependence, misuse, product quality, marketing practices, and long-term health effects. Researchers are still working through many of these questions.
The scientific literature reflects that ongoing debate. AI-generated summaries often do not.
Why AI Struggles With Scientific Uncertainty
Large language models are trained to synthesize information from vast amounts of text. They excel at condensing complex subjects into concise explanations. They do not reliably distinguish between settled scientific questions and ongoing scientific controversies. When researchers disagree, AI systems frequently default to the interpretation most prevalent in their training data and may understate the extent of scientific disagreement.
In debates over emerging substances, that dynamic can accelerate moral panics. Repeated claims about potency, overdose risk, or abuse potential may acquire an aura of certainty long before the underlying science has reached a consensus.
That tendency can be especially misleading in public health, where evidence evolves, and experts often disagree on how risks should be interpreted. Questions about nutrition, nicotine alternatives, psychedelics, environmental exposures, and emerging therapies often generate competing interpretations of the same evidence.
Scientific progress emerges through debate, reanalysis, and the testing of competing explanations. Researchers challenge assumptions and revise their conclusions. By contrast, AI systems are optimized to provide answers.
That tendency creates a risk that uncertain claims appear certain simply because they are repeated frequently enough in the underlying information ecosystem.
The Wrong Solution: Regulating Scientific Disagreement
Some observers have responded to concerns about AI misinformation by calling for greater regulatory oversight of AI outputs, trusted information frameworks, and government involvement in defining responsible AI behavior. However, public health institutions have their own histories of error, overconfidence, and political influence. Empowering regulators to referee scientific controversies risks replacing one source of false certainty with another.
Artificial intelligence holds tremendous promise for health care and scientific research. These tools can help clinicians navigate the medical literature, assist researchers in identifying patterns, and help the public access information that would otherwise remain buried in journals and databases.
Yet many public health controversies already involve institutions presenting uncertain evidence as settled fact. AI systems trained on that information ecosystem may inherit the tendency to understate uncertainty and project confidence beyond what the evidence warrants.
Competition among models, transparency about uncertainty, open debate, and continued scrutiny of confident claims offer a healthier path forward. That scrutiny should apply whether those claims originate with advocacy groups, regulators, journalists, or AI systems.
The debate over 7‑OH will continue as new evidence emerges. Public health discussions are healthier when uncertainty remains visible. Policymakers considering restrictions or bans should remember that today’s controversy may become tomorrow’s accepted therapy. Many substances now used in medicine were once dismissed, feared, or poorly understood.
AI can summarize a scientific controversy, but it cannot resolve one. When the evidence remains contested, confidence is not the same thing as certainty—and a chatbot’s answer is not a substitute for scientific debate.
To read other parts of this blog series, go here.