Feasibility of human-in-the-loop multimodal generative artificial intelligence chatbot for school-based adolescent mental health support

Table 1 Characteristics of the study sample (n = 62), stratified by intervention group (n = 30) and waitlist control group (n = 32), mean ± SD or n (%)

Characteristic	Control (n = 32)	Intervention (n = 30)	Overall (n = 62)	P value
Age (year)	13.81 ± 0.59	13.9 ± 0.31	13.86 ± 0.47	0.464
Gender				0.362
Male	12 (37.5)	8 (26.7)	20 (32.3)
Female	20 (62.5)	22 (73.3)	42 (67.7)
Depression symptoms
CES-D	25.53 ± 5.47	34.27 ± 7.42	29.76 ± 7.79	< 0.001a
PHQ-9	9.88 ± 3.32	14.8 ± 4.06	12.26 ± 4.43	< 0.001a
Anxiety symptoms
GAD-7	6.91 ± 2.72	11.9 ± 4.28	9.32 ± 4.33	< 0.001a
Psychological inflexibility
AAQ-II	22.50 ± 6.87	30.37 ± 7.50	26.31 ± 8.15	< 0.001a
Ruminative thinking
RRS	49.03 ± 10.37	58.63 ± 12.40	53.68 ± 12.29	0.002a
Emotion regulation
ERI	41.63 ± 10.50	41.43 ± 11.51	41.53 ± 10.91	0.946
Life orientation
LOT-R	11.13 ± 2.25	10.00 ± 2.74	10.58 ± 2.55	0.082
Working alliance (WAI-SR)		3.56 ± 0.86
Bond subscale		3.96 ± 0.92
Task subscale		3.06 ± 0.87
Goal subscale		3.71 ± 1.09
Engagement
Number of messages		325.60 ± 129.71
Total use time (second)		6080.04 ± 2557.09

^aP < 0.01.

A working alliance and engagement were assessed only in the intervention group. CES-D: Center for Epidemiological Studies Depression Scale; PHQ-9: Patient Health Questionnaire-9; GAD-7: Generalized Anxiety Disorder-7; AAQ-II: Acceptance and Action Questionnaire-II; RRS: Rumination Response Scale; ERI: Emotion Regulation Inventory; LOT-R: Life Orientation Test-Revised; WAI-SR: Working Alliance Inventory-Short Revised.

Table 2 Comparisons of mental health outcomes between groups across time points

Measure and group	Baseline, mean ± SD	Post-intervention, mean ± SD	Δmean ± SD	n	Between-group LMM1			ANCOVA2
					F	Cohen’s d3, 95%CI	P value	Adjusted mean difference, 95%CI	P value
CES-D
Intervention	34.27 ± 7.42	31.40 ± 9.08	-2.87 ± 7.55	30	2.40	-0.40, -0.90 to 0.11	0.127	-0.96, -5.96 to 4.04	0.703
Control	25.53 ± 5.47	25.94 ± 9.53	0.41 ± 8.86	32
GAD-7
Intervention	11.90 ± 4.28	10.43 ± 4.78	-1.47 ± 4.08	30	4.81	-0.56, -1.07 to -0.05	0.032a	-0.52, -2.94 to 1.90	0.668
Control	6.91 ± 2.72	7.66 ± 4.15	0.75 ± 3.86	32
PHQ-9
Intervention	14.80 ± 4.06	12.90 ± 5.58	-1.90 ± 4.31	30	1.54	-0.32, -0.82 to 0.18	0.220	-0.34, -3.46 to 2.78	0.828
Control	9.88 ± 3.32	9.56 ± 5.66	-0.31 ± 5.50	32
AAQ-II
Intervention	30.37 ± 7.50	29.93 ± 8.82	-0.43 ± 6.98	30	0.52	-0.18, -0.68 to 0.32	0.473	1.11, -3.00 to 5.22	0.592
Control	22.50 ± 6.87	23.38 ± 7.82	0.88 ± 7.75	32
RRS
Intervention	58.63 ± 12.40	54.10 ± 12.51	-4.53 ± 11.30	30	2.85	-0.44, -0.95 to 0.06	0.097	-1.22, -6.63 to 4.19	0.654
Control	49.03 ± 10.37	49.06 ± 11.44	0.03 ± 9.34	32
ERI
Intervention	41.43 ± 11.51	41.17 ± 11.62	-0.27 ± 8.70	30	0.02	0.04, -0.46 to 0.54	0.879	-0.03, -4.69 to 4.63	0.990
Control	41.63 ± 10.50	40.97 ± 10.30	-0.66 ± 11.30	32
LOT-R
Intervention	10.00 ± 2.74	11.07 ± 3.21	1.07 ± 3.51	30	0.93	0.25, -0.25 to 0.75	0.338	0.14, -1.37 to 1.65	0.854
Control	11.13 ± 2.25	11.38 ± 2.76	0.25 ± 2.90	32
LOT-R subscale
Optimism
Intervention	5.07 ± 1.51	6.30 ± 2.22	1.23 ± 2.22	30	5.13	0.59, 0.08 to 1.09	0.027a	0.71, -0.44 to 1.87	0.219
Control	6.13 ± 1.56	6.03 ± 2.10	-0.09 ± 2.31	32
Pessimism
Intervention	4.93 ± 2.08	4.77 ± 2.19	-0.17 ± 2.00	30	0.69	-0.20, -0.70 to 0.29	0.409	-0.56, -1.59 to 0.46	0.277
Control	5.00 ± 2.21	5.34 ± 1.96	0.34 ± 2.88	32

¹Mixed-effects models estimated the group × time interaction across two groups and two time points, controlling for age and gender.

²Analysis of covariance with the postintervention score as the dependent variable, adjusting for the corresponding baseline score, age, and sex; the adjusted mean difference is intervention-control.

³Between-group Cohen’s d was calculated as the post-intervention score difference between intervention and control groups divided by the pooled standard deviation. Within-group Cohen’s d was calculated as the difference between baseline and post-intervention scores divided by the pooled standard deviation. Thresholds of Cohen’s d equal to 0.20, 0.50, and 0.80 represent small, medium, and large effect sizes, respectively.

^aP < 0.01.

95%CI: 95% confidence interval; CES-D: Center for Epidemiological Studies Depression Scale; PHQ-9: Patient Health Questionnaire-9; GAD-7: Generalized Anxiety Disorder-7; AAQ-II: Acceptance and Action Questionnaire-II; RRS: Rumination Response Scale; ERI: Emotion Regulation Inventory; LOT-R: Life Orientation Test-Revised.