Biases of large language models in diagnosing Cushing’s syndrome

Abstract

The diagnosis of endogenous Cushing’s syndrome (CS) can be complicated and often delayed, given its low incidence (estimated globally at 1.8 cases to 4.5 cases per million people per year) and its clinical features that mimic far more prevalent metabolic disorders, such as central obesity, hypertension, and glucose intolerance. In clinical practice, physicians rely on cognitive heuristics that are prone to error, contributing to diagnostic delays (on average around 34 months pass from symptom onset to diagnosis of CS). Large language models and machine learning algorithms could be potential decision-support tools for screening and differential diagnosis of CS. However, these systems are at risk of inheriting and even amplifying existing cognitive biases and data-driven distortions embedded in their training data. Machine learning models designed for CS could be vulnerable to methodological flaws, notably spectrum bias and the exclusion of clinically relevant demographic variables, demanding attention from the endocrine and medical informatics communities. This paper examines how cognitive and algorithmic biases intersect in diagnostic models for CS, highlighting parallels between human diagnostic heuristics (e.g., anchoring, availability, and framing) and data-driven distortions (e.g., spectrum and measurement bias) in artificial intelligence.

Keywords: Cushing’s syndrome; Diagnostic bias; Large language models; Spectrum bias; Algorithmic fairness

Core Tip: The diagnosis of Cushing’s syndrome remains challenging due to its rarity and its resemblance to common metabolic disorders. Large language models and other artificial intelligence-capable systems are potential diagnostic tools for early detection and differential diagnosis; nevertheless, they are likely to strengthen both human cognitive and training data biases. Large language models are susceptible to biases in the textual data they are trained on, reflecting human cognitive biases, while traditional machine learning models are susceptible to biases in structured data, leading to spectrum and measurement bias. Spectrum bias, exclusion of demographic variables, and heterogeneity of the data undermine diagnostic validity and justice. These heuristics are similar to clinicians’ mental shortcuts - anchoring, availability, and framing - and share the same diagnostic bias. Transparency, data diversity, and clinically relevant predictors are required to build unbiased, interpretable artificial intelligence solutions for endocrine diagnosis.