Accuracy and reproducibility of ChatGPT responses to parent and patient inquiries on attention-deficit/hyperactivity disorder

Table 1 Grading criteria for accuracy of ChatGPT responses

Grading category	Definition
Comprehensive/correct	The response is accurate, clinically appropriate, and sufficiently comprehensive such that no additional explanation would be required in routine clinical practice
Incomplete/partially correct	The response is generally accurate but lacks relevant clinical details, developmental nuance, or contextual clarification
Mixed/potentially misleading	The response contains both accurate and inaccurate elements or statements that could lead to misunderstanding in a clinical or developmental context.
Inaccurate/irrelevant	The response is incorrect or unrelated to the question asked

Responses were independently rated by two blinded child and adolescent psychiatrists using the above accuracy framework. Ratings were assigned based on clinical appropriateness, completeness, and developmental contextualization. Disagreements were resolved by consensus.

Table 2 Distribution of accuracy ratings across attention-deficit/hyperactivity disorder question categories (n = 88), n (%)

Question category	Comprehensive/correct	Incomplete/partially correct	Mixed/misleading
Basic knowledge (n = 30)	20 (66.7)	7 (23.3)	3 (10.0)
Diagnosis and assessment (n = 22)	13 (59.1)	6 (27.3)	3 (13.6)
Treatment and medication (n = 21)	10 (47.6)	7 (33.3)	4 (19.1)
Long-term outcomes and psychosocial impact (n = 15)	8 (53.3)	4 (26.7)	3 (20.0)
Total (n = 88)	52 (59.1)	24 (27.3)	12 (13.6)

Percentages were calculated within each question category. No responses were rated as inaccurate/irrelevant.

Table 3 Examples of ChatGPT responses rated as incomplete or potentially misleading, with expert commentary

Parent/patient question	ChatGPT response summary	Reason for rating	Expert clinical commentary
Can ADHD medications cause addiction?	Indicates that stimulant medications may lead to dependence if misused, while generally being safe when prescribed	The response oversimplifies addiction risk and does not clearly differentiate therapeutic use from substance misuse	Clinically, stimulant treatment for ADHD is not equivalent to substance use disorder. When appropriately prescribed and monitored, evidence suggests a low risk of addiction, and this distinction should be explicitly communicated to prevent unnecessary parental anxiety
Will my child outgrow ADHD?	Suggests that symptoms often improve with age and that some individuals may no longer meet diagnostic criteria in adulthood	The response inadequately addresses symptom persistence and functional impairment over time	Although symptom severity may decrease, ADHD frequently persists into adulthood, often with ongoing academic, occupational, or emotional difficulties. Families should be informed about the developmental trajectory rather than reassured by a simplistic “outgrowing” narrative
Does ADHD medication affect brain development?	States that there is no clear evidence of harmful effects on brain development	The answer lacks reference to current neurodevelopmental and longitudinal research	Contemporary neuroimaging and longitudinal studies suggest possible normalization of certain neural networks with treatment, while uncertainties remain. A balanced discussion of current evidence and ongoing research is essential in developmental psychiatry
How long should my child take ADHD medication?	Notes that treatment duration varies depending on individual needs	The response provides vague guidance and omits principles of treatment planning	In clinical practice, medication use involves regular monitoring, periodic reassessment, and shared decision-making with families, rather than indefinite or open-ended treatment. This nuance is critical for parental understanding
Does ADHD cause emotional problems later in life?	Associates ADHD with emotional regulation difficulties	The framing risks a deterministic interpretation of long-term outcomes	Emotional difficulties are influenced by multiple factors, including comorbidities, environmental supports, and early intervention. Overly deterministic statements may contribute to stigma or parental pessimism
Are stimulant medications safe for long-term use?	States that stimulants are commonly used long term and are considered safe	The response does not emphasize monitoring or potential side effects	Long-term stimulant use requires systematic monitoring of growth, cardiovascular parameters, sleep, and emotional functioning. Omitting these elements may give families a false sense of unconditional safety

This table presents representative examples of ChatGPT responses rated as incomplete/partially correct or mixed/potentially misleading, along with expert clinical commentary highlighting missing developmental context and potential patient-safety implications. Response summaries were condensed for clarity. ADHD: Attention-deficit/hyperactivity disorder.