LLM Bias Downgraded Triage for Ill Patients
Stigmatizing language in clinical notes caused large language models to inaccurately deprioritize acutely ill patients.
Updated on Sept. 19, 2026 in Artificial Intelligence
Live Poll
Do you trust artificial intelligence to make fair triage decisions in emergency medical settings?
Researchers found that large language models consistently downgraded triage prioritization for patients when clinical inputs included stigmatizing language. The study, which evaluated three open-weight models, showed significant harmful reprioritization when identifying patients with psychiatric histories or frequent emergency room visits.
Why it matters
The findings reveal critical safety risks in deploying AI for clinical triage, as models may inadvertently rely on social stigma rather than medical urgency. This bias threatens to delay life-saving care for the most vulnerable patients.
The study performed 221,556 comparisons across three open-weight models, Gemma, Qwen, and DeepSeek, testing 18 different input formulations. Psychiatric history alone caused prioritization shifts of up to 9.5 percent.
The players
Gemma
This is an open-weight model family developed by Google.
Qwen
This is a large language model series developed by Alibaba Cloud.
DeepSeek
This is a series of open-weight models released by the DeepSeek research organization.
The details
Researchers used Emergency Severity Index-matched patient pairs to observe how models reacted when clinical notes included specific demographic or social attributes. While stigma-related language prompted significant bias, the study found no consistent harmful shifts related to patient race, language, or insurance status.
Timeline
September 19, 2026: The study was published.
The Big Picture
This study updates the current understanding of algorithmic safety by providing empirical evidence of failure modes in LLM triage systems. It highlights the urgent need to integrate social bias mitigation into clinical AI validation workflows, following the precedents set by the Nature Digital Medicine clinical AI safety guidelines.
Patients may face significant delays if their clinical records contain stigmatizing labels that influence automated triage systems. Clinicians should be aware that these models might prioritize individuals based on social history rather than the six-hour clinical deterioration window.
The takeaway
Healthcare providers should exercise caution when using LLMs for triage and prioritize human oversight to prevent bias against marginalized groups. Developing more robust, bias-aware datasets is essential to ensure AI tools accurately evaluate patient urgency regardless of social history.
Further reading
For more on the development of ethical systems, visit our Artificial Intelligence section.
More information
Review the full findings in the peer-reviewed study on LLM triage bias published in Nature.
Source note: This article includes information reported by Nature.
Live Poll
Do you trust artificial intelligence to make fair triage decisions in emergency medical settings?
