Retrospective analysis of nursing interventions using artificial intelligence multimodal emotion recognition for perioperative depression in head and neck tumors

Abstract

BACKGROUND

Perioperative depression is prevalent among head and neck tumor patients, while traditional emotional assessment methods lag and fail to meet clinical nursing needs.

AIM

To investigate artificial intelligence (AI) multimodal emotion recognition's effects on enhancing individualized nursing care for depression patients during perioperative head and neck tumor surgeries, focusing on emotional fluctuations, nursing response efficiency, and postoperative recovery outcomes.

METHODS

Sixty patients with perioperative depression associated with head and neck tumors admitted to our hospital between January 2023 and December 2025 were selected and divided into two groups: The conventional care group (27 patients.) and the AI individualized care group (33 patients). We compared the baseline data, perioperative emotional indicators, postoperative outcomes, emotional recognition, and nursing responses between the two groups while simultaneously evaluating the recognition performance of the multimodal model.

RESULTS

There were no significant baseline differences between the groups. During observation, the Hamilton Depression Rating Scale (HAMD) score, emotional score, Emotional Mood Index (EMI), and proportion of positive emotions showed dynamic changes with significant group-time interaction. The AI-individualized care group demonstrated lower HAMD and EMI scores, higher comprehensive emotional scores, and a greater proportion of positive emotions, postoperatively. Additionally, hospital stay length was reduced, complications decreased, and Physical Component Summary, Mental Component Summary, and Emotional Recovery Score improved significantly. The AI group outperformed the conventional nursing group in recognizing negative emotions, intervention matching, and emotional improvement within 24 hours. The time for initial recognition and nursing responses was much shorter. The area under the curve, sensitivity, and specificity of the multimodal fusion model were 0.894, 86.364%, and 83.871%, respectively, indicating its superiority over the single-modal model.

CONCLUSION

AI-informed individualized nursing can facilitate earlier identification of abnormal emotions in patients with head and neck tumors perioperatively, enhance nursing efficiency, and positively influence perioperative emotions and recovery.

Key Words: Head and neck tumors; Perioperative depression; Artificial intelligence; Multimodal emotion recognition; Personalized care; Quality of life

Core Tip: Artificial intelligence multimodal emotion recognition has good diagnostic efficacy. Applying it to targeted nursing for head and neck tumor patients can promptly capture perioperative depressive mood, streamline nursing response, and improve emotional status and postoperative rehabilitation.

INTRODUCTION

Patients diagnosed with head and neck tumors often experience significant physical and psychological stress during the perioperative period. The psychological burden associated with the diagnosis, coupled with the effects of surgery and subsequent treatments, can adversely affect patients' swallowing, vocalization, eating functions, and appearance. These changes may induce or exacerbate persistent negative emotions, such as anxiety and depression[1]. For patients with preexisting depressive symptoms who undergo surgery, the perioperative stress response can further intensify emotional fluctuations, potentially undermining treatment compliance and adversely affecting postoperative recovery and overall rehabilitation[2]. Consequently, timely identification of and intervention for abnormal emotional states during the perioperative period have emerged as critical issues in the nursing management of head and neck tumors.

Currently, the identification of perioperative depressive states in clinical practice depends primarily on traditional methods, including scale assessments, observations by nursing staff, and patients’ chief complaints. Although these methods possess certain operational and practical utilities, they exhibit notable deficiencies in the continuous and dynamic monitoring of emotional fluctuations. During the early postoperative period, patients frequently experience pain, fatigue, limited communication, or reduced willingness to express themselves, which hampers their ability to convey their true feelings in a timely and comprehensive manner. Consequently, reliance solely on manual identification often results in delays or omissions[3]. With the rapid advancement of artificial intelligence (AI) technology, emotion recognition is progressively transitioning from a singular information source to a multi-modal fusion approach, in which facial expressions, vocal characteristics, and textual cues can reveal patients' emotional fluctuations from various perspectives. A thorough analysis of multisource information is expected to enhance the sensitivity, timeliness, and reliability of recognizing abnormal emotional states[4]. If this technology is further incorporated into the perioperative care process, it may offer novel technical support for early screening, dynamic assessment, and stratified intervention for patients at an elevated risk of depression[5].

This study focused on patients with depression during the perioperative period for head and neck tumors. It compared the differences between conventional and individualized care plans informed by AI multimodal emotion recognition in terms of emotional improvement, nursing response efficiency, and postoperative recovery. This study aimed to provide a reference for optimizing perioperative care models for head and neck tumors.

MATERIALS AND METHODS

General data

This study retrospectively selected 60 patients with perioperative depression resulting from head and neck tumors who were admitted to our hospital between January 2023 and December 2025. The inclusion criteria were as follows: (1) A confirmed diagnosis of head and neck squamous cell carcinoma via pathological examination; (2) Fulfillment of the relevant diagnostic criteria for perioperative depression, with a Hamilton Depression Rating Scale (HAMD)-17 score of ≥ 17 points; (3) Age ranging from 18 years to 75 years; and (4) Completeness of clinical data. The exclusion criteria included: (1) Presence of other severe mental disorders; (2) Significant cognitive impairment; (3) Receipt of systematic antidepressant treatment or psychotherapy within three months prior to surgery; (4) Substantial restrictions affecting facial or voice data collection; and (5) Absence of critical research data. Based on the different perioperative nursing methods employed, patients were categorized into two groups: Conventional nursing (27 patients) and AI individualized nursing (33 patients).

AI multimodal emotion recognition system and early warning mechanism

System composition: In this study, we employed an AI multimodal emotion recognition system to facilitate the assessment of the emotional states of patients. The system comprises of three main components: Bedside video collection, voice collection, and electronic diary recording. The video equipment captures patient facial expression data, whereas the voice equipment gathers vocal features during patient interactions. Additionally, the electronic diary module recorded patients’ subjective emotional experiences. The system autonomously collects information at predetermined intervals and functions in a non-contact manner.

Core algorithm workflow: The system initially pre-processed the collected video, audio, and text data. It subsequently extracts indicators, including changes in expression, phonetic features, and emotional tendencies from the text. A comprehensive analysis of this multisource information was conducted to generate an overall emotional score for each patient. In addition, the system outputs parameters such as the Emotional Mood Index (EMI) and Emotional Recovery Score (ERS) to represent the patient's emotional fluctuations and recovery trends.

Early warning trigger logic: This system implements hierarchical early warnings based on a comprehensive assessment of emotional states. When mild abnormalities are detected, the information is relayed to the nurse responsible for performing the initial intervention. If the risk level is assessed as relatively high, both the psychological specialist nurse and attending physician are simultaneously alerted to conduct further evaluation. A multidisciplinary collaborative approach should be initiated for individuals exhibiting persistent emotional fluctuations or inadequate recovery. The backend of the system records the timing of early warnings, response duration, and intervention details, thereby facilitating subsequent tracking and analysis.

Standardized nursing intervention plan

Prior to the research implementation, the relevant nursing staff underwent standardized training to become proficient in system usage, emotion recognition, and essential communication strategies. The routine care group received standard perioperative care, including health education, general psychological counseling, pain management, and rehabilitation guidance. In contrast, the AI-individualized care group incorporated dynamic monitoring through a multimodal system and employed stratified care based on early warning results. A designated nurse was responsible for early intervention in cases of mild abnormalities. For patients at higher risk, psychological specialist nurses participated in further assessment and management, employing intervention strategies, such as family support and optimization of pain management. Multidisciplinary team discussions were organized as required. The entire intervention phase spanned from three days before the operation to 14 days after the operation.

Observation indicators

The observation indicators for this study included baseline patient data, perioperative emotion-related metrics, postoperative outcomes, and the recognition efficacy of the system. The baseline data primarily consisted of age, sex, tumor location, tumor-node-metastasis (TNM) stage, surgical method, and American Society of Anesthesiologists (ASA) grade. The emotion-related metrics included the HAMD-17 score, comprehensive emotion score, proportion of positive emotions, EMI. In this study, a HAMD-17 score of ≥ 17 was adopted as the criterion for identifying high-risk perioperative depression and served as the reference standard for receiver operating characteristic (ROC) analysis. The postoperative outcomes focused on the length of hospital stay, incidence of complications, and quality of life score at one month postoperatively, including clinically diagnosed adverse events such as wound infection, pharyngeal fistula, pulmonary infection, and postoperative hemorrhage. The system’s recognition efficiency was assessed using sensitivity, specificity, area under the curve (AUC), and the Youden index.

Statistical analysis

Statistical analyses were performed using SPSS version 26.0 and Python version 3.8. Following the normality assessment of the measurement data, values conforming to a normal distribution were reported as mean ± SD, and appropriate parametric tests were employed for intergroup comparisons. In contrast, data that did not conform to a normal distribution were presented as medians and quartiles using non-parametric tests. Categorical data were expressed as n (%), and comparisons between groups were conducted using the χ² test or Fisher’s exact test. Repeated-measures data collected during the perioperative period were analyzed using repeated-measures analysis of variance, whereas correlation analyses were performed using either the Pearson or Spearman method. The recognition efficiency of the model was assessed using ROC curves. Statistical significance was set less than 0.05.

RESULTS

Comparison of baseline data and preoperative emotional parameters between the two groups of patients

No statistically significant differences were observed between the two groups in age, sex, smoking history, primary tumor site, ASA classification, TNM stage, surgical method, preoperative HAMD score, comprehensive emotional score, baseline EMI, proportion of positive emotions, or facial modal quality score (P > 0.05) (Table 1).

Table 1 Comparison of baseline data and preoperative emotional parameters between the two groups of patients, n (%).

Project	Conventional care group (n = 27)	Artificial intelligence personalized care group (n = 33)	t/χ2 value	P value
Age (years)	58.48 ± 9.73	57.94 ± 10.18	0.208	0.836
Male	17 (62.96)	22 (66.67)	0.090	0.765
History of smoking	16 (59.26)	21 (63.64)	0.120	0.729
Oral cancer	10 (37.04)	12 (36.36)	0.003	0.957
Laryngeal cancer	8 (29.63)	10 (30.30)	0.003	0.955
Oropharyngeal/hypopharyngeal cancer	9 (33.33)	11 (33.33)	0.000	1.000
ASA class II	20 (74.07)	25 (75.76)	0.022	0.881
TNM stage III-IV	21 (77.78)	27 (81.82)	0.152	0.697
Total laryngectomy	8 (29.63)	10 (30.30)	0.003	0.955
HAMD score (points)	24.93 ± 4.11	24.12 ± 4.36	0.734	0.466
Comprehensive Emotional Score (point)	37.84 ± 10.22	38.67 ± 9.85	-0.319	0.751
Baseline EMI	0.39 ± 0.11	0.37 ± 0.10	0.737	0.464
Baseline proportion of positive emotions (%)	28.10 ± 9.23	29.22 ± 8.95	-0.476	0.636
Facial modal quality score	0.85 ± 0.07	0.86 ± 0.07	-0.551	0.584

ASA: American Society of Anesthesiologists; TNM: Tumor-node-metastasis; HAMD: Hamilton Depression Rating Scale; EMI: Emotional Mood Index.

Correlation analysis between AI multimodal emotion parameters and HAMD score

Correlation analysis revealed a moderate negative correlation between the HAMD score and the comprehensive emotional score, proportion of positive emotions, and ERS, as well as a moderate positive correlation with EMI (P < 0.05). No significant correlation was found between the facial modal quality score and HAMD score (P > 0.05) (Table 2).

Table 2 Correlation analysis of artificial intelligence multimodal emotion parameters and Hamilton Depression Rating Scale score.

Variable	R value	P value
Comprehensive emotional score (point)	-0.624	< 0.001
Proportion of positive emotions	-0.592	< 0.001
EMI	0.671	< 0.001
ERS	-0.561	< 0.001
Facial modal quality score	-0.208	0.112

EMI: Emotional Mood Index; ERS: Emotional Recovery Score.

Comparison of perioperative emotional improvement between the two groups of patients

Repeated-measures analysis of variance revealed significant time effects on the HAMD score, comprehensive emotional score, EMI, and proportion of positive emotions (all P < 0.05). Intergroup comparisons indicated that the main effects of the comprehensive emotional score and proportion of positive emotions were statistically significant (P < 0.05), whereas the main effects of the HAMD score and EMI did not reach statistical significance (P > 0.05). Additionally, the group and time interaction effects for the HAMD score, comprehensive emotional score, EMI, and proportion of positive emotions were all statistically significant (all P < 0.05) (Table 3).

Table 3 Comparison of perioperative emotional indicators between the two groups of patients, mean ± SD.

Indicator	Time point	Conventional care group (n = 27)	Artificial intelligence personalized care group (n = 33)	F inter group/time/interaction value	P inter group/time/interaction value
HAMD score (points)	Before surgery 1 day	24.93 ± 4.11	24.12 ± 4.36	0.133/69.956/3.784	0.716/< 0.001/0.026
	After surgery 3 days	22.70 ± 3.86	20.49 ± 3.52a
	After surgery 7 days	20.48 ± 3.51	16.88 ± 3.13a
Comprehensive emotional score (point)	Before surgery 1 day	37.84 ± 10.22	38.67 ± 9.85	4.024/101.392/7.057	0.047/< 0.001/0.001
	After surgery 3 days	42.59 ± 8.94	48.88 ± 8.21a
	After surgery 7 days	48.74 ± 8.31	57.46 ± 7.68a
EMI	Before surgery 1 day	0.39 ± 0.11	0.37 ± 0.10	0.832/74.273/5.907	0.364/< 0.001/0.004
	After surgery 3 days	0.35 ± 0.10	0.29 ± 0.08a
	After surgery 7 days	0.30 ± 0.08	0.22 ± 0.071
Proportion of positive emotions (%)	Before surgery 1 day	28.10 ± 9.23	29.22 ± 8.95	5.385/95.170/8.509	0.022/< 0.001/< 0.001
	After surgery 3 days	32.67 ± 8.50	39.39 ± 8.27a
	After surgery 7 days	38.07 ± 7.96	47.97 ± 7.51a

^aP < 0.05 between the routine care group vs artificial intelligencepersonalized care group at the same time point.

HAMD: Hamilton Depression Rating Scale; EMI: Emotional Mood Index.

Comparison of postoperative clinical outcomes between the two groups of patients

Compared to the conventional care group, the AI-personalized care group experienced a reduced postoperative hospital stay, a lower rate of complications, an elevated ERS, and higher 36-Item Short Form Health Survey Physical Component Summary and (PCS) Mental Component Summary (MCS) scores one month post-surgery. These differences were statistically significant (P < 0.05) (Table 4).

Table 4 Comparison of postoperative clinical outcomes between the two groups of patients, n (%)/mean ± SD.

Indicator	Conventional care group (n = 27)	Artificial intelligence personalized care group (n = 33)	t/χ2 value	P value
Postoperative hospital stay (days)	14.48 ± 4.13	11.76 ± 3.36	2.814	0.007
Incidence of complications	11 (40.74)	5 (15.15)	4.972	0.026
SF-36 PCS score	56.41 ± 8.26	63.52 ± 7.84	-3.412	0.001
SF-36 MCS score	57.70 ± 7.92	66.27 ± 7.20	-4.385	< 0.001
ERS (minutes/day)	1.51 ± 0.44	2.39 ± 0.52	-6.981	< 0.001

SF-36: 36-Item Short Form Health Survey; MCS: Mental Component Summary; ERS: Emotional Recovery Score.

Comparison of negative emotion recognition and nursing response efficiency between the two groups

The AI-personalized nursing group exhibited a significantly higher recognition rate for negative emotions than the conventional nursing group. Additionally, the time required for the initial recognition and nursing responses was markedly reduced. Furthermore, both the intervention matching and emotion improvement rates within 24 hours showed significant increases (all P < 0.05) (Table 5).

Table 5 Comparison of negative emotion recognition and nursing response efficiency between the two groups, n (%).

Indicator	Conventional care group (n = 27)	Artificial intelligence personalized care group (n = 33)	t/χ2 value	P value
Negative emotion recognition rate	18 (66.67)	31 (93.94)	7.377	0.007
First identification time (hours)	18.70 ± 6.32	6.82 ± 2.95	9.607	< 0.001
Nursing response time (minutes)	42.19 ± 11.73	17.67 ± 5.42	10.707	< 0.001
Intervention matching rate	16 (59.26)	30 (90.91)	8.315	0.004
Emotional improvement rate within 24 hours	14 (51.85)	27 (81.82)	6.163	0.013

The recognition efficiency of AI multimodal models for high-risk states of perioperative depression

ROC curve analysis indicated that the multimodal fusion model exhibited the highest recognition efficacy for high-risk states of perioperative depression, achieving an AUC of 0.894, with sensitivity and specificity values of 86.364% and 83.871%, respectively. The Youden index was calculated to be 0.702, all of which surpassed the performance of single-modal models based on faces, speech, and text (Table 6 and Figure 1).

Figure 1 Receiver operating characteristic curves of recognition performance of artificial intelligence multimodal models and single-modal models. AUC: Area under the curve.

Table 6 Comparison of recognition performance between artificial intelligence multimodal models and single-modal models.

Model	AUC (95%CI)	Sensitivity (%)	Specificity (%)	Youden index
Facial modality	0.782 (0.663-0.901)	72.727	74.194	0.469
Speech modality	0.741 (0.616-0.867)	68.182	70.968	0.391
Text modality	0.756 (0.634-0.878)	70.455	72.581	0.430
Multimodal fusion	0.894 (0.812-0.975)	86.364	83.871	0.702

AUC: Area under the curve; CI: Confidence interval.

DISCUSSION

Emotional problems are relatively common during the perioperative period in patients with head and neck tumors[6]. Their formation is often closely related to multiple factors such as psychological shock caused by disease diagnosis, surgical trauma stress, and changes in swallowing, vocalization, eating functions, and appearance after surgery[7]. These factors not only increase the psychological burden on patients, but may also continuously affect their self-perception, social interaction, and disease coping styles. For patients who already have a depressive state before surgery, the superimposition of multiple stress factors is more likely to lead to persistently low mood, decreased willingness to communicate, and weakened cooperation with treatment[8]. Therefore, in the perioperative care of head and neck tumors, in addition to paying attention to the stability of vital signs, surgical recovery, and prevention of complications, it is necessary to attach importance to the dynamic assessment and timely intervention of the patient’s psychological state to promote overall rehabilitation.

The results of this study showed that there were no statistically significant differences in the general information and preoperative emotional indicators between the two groups of patients before the intervention and that they had good comparability[9]. On this basis, after intervention with different nursing methods, the AI individualized nursing group showed more significant improvements in indicators such as the HAMD score, comprehensive emotional score, EMI, and the proportion of positive emotions[10]. At the same time, it also demonstrated a better trend in outcome indicators such as hospital stay, complication rate and quality of life. These results suggest that after incorporating AI multimodal emotion recognition technology into the perioperative nursing process, its potential benefits may not be limited to enhancing the ability to recognize abnormal emotions[11]. It may also further influence perioperative recovery by promoting early intervention and optimizing the nursing response process. This result is reasonable from the clinical practice perspective. In the traditional nursing model, the judgment of patients’ emotional states relies mainly on chief complaints, scale evaluations, and observations by the nursing staff. However, during the perioperative period, patients often have difficulty conveying their feelings in a timely and sufficient manner due to pain, fatigue, limited communication, or atypical emotional expressions, thereby affecting the early identification of emotional abnormalities. In contrast, multimodal emotion recognition systems can continuously integrate facial expressions, voice features, and text information to monitor patients’ emotional changes more continuously, providing earlier abnormal alerts for caregivers and thus creating conditions for timely assessment and intervention[12].

This study also found that the HAMD score was consistent with the comprehensive emotional score, proportion of positive emotions, and changing direction of ERS, but opposite to EMI, indicating that the emotion-related parameters output by the system corresponded well with the results of the clinical scale[13,14]. Meanwhile, the multimodal fusion model outperformed the single-modal model in terms of AUC, sensitivity, and specificity, suggesting that the integration of multi-source information can improve the stability and accuracy of the recognition results to a certain extent[15,16]. From the perspective of postoperative recovery, the hospital stay in the AI group was shortened, the incidence of complications decreased, and the PCS and MCS scores increased, suggesting that timely emotional intervention may help improve the overall recovery status of patients[17,18]. If patients remain in a negative emotional state for a long time during the perioperative period, they often experience problems, such as poor sleep quality, reduced communication, and insufficient cooperation in rehabilitation training. Early identification and targeted treatments may alleviate these adverse effects to a certain extent[19,20].

In addition, the AI group outperformed the conventional nursing group in terms of the recognition rate of negative emotions, time to first recognition, response time to nursing, matching rate of intervention, and improvement rate of emotions within 24 hours. This indicates that the system not only helps to detect abnormalities but also promotes the transformation of the nursing process from a passive response to active management. This model has application value for patients with rapid perioperative emotional changes and significant risk fluctuations. The reason for this might lie in the fact that AI systems can continuously integrate multi-source information and conduct dynamic monitoring, thereby detecting anomalies earlier and providing support for timely assessment and intervention by caregivers. To some extent, this compensates for the deficiency of traditional nursing, which mainly relies on chief complaints and manual observation, and is especially suitable for patients[21,22] with limited expression or atypical emotional manifestations in the early postoperative period. However, it should also be noted that in this study, the AI system mainly played the role of auxiliary recognition[23,24], risk warning, and process support and could not replace the communication, assessment, and intervention of the nursing staff[25]. Improvement in patient outcomes ultimately depends on whether subsequent care measures are timely, continuous, and effective[26]. Therefore, AI technology is more suitable as an auxiliary tool for perioperative care to enhance recognition efficiency and optimize the care process.

This study has several limitations that should be considered when interpreting the findings. First, it was a single-center retrospective study, and patients were grouped according to the nursing model they received in routine clinical practice rather than a randomized allocation. Although the baseline characteristics and preoperative emotional indicators were comparable between the groups, potential selection bias and residual confounding factors could not be completely excluded. Differences in nursing resource allocation, nursing team experience, patient compliance, family support, and other unmeasured variables may have influenced clinical outcomes. Therefore, the associations observed in this study should be interpreted cautiously and causal relationships cannot be established. Second, the sample size was relatively small, with only 60 patients. Because this study was designed as a retrospective exploratory analysis based on available clinical cases, a prospective sample size calculation was not performed before data collection. This limited sample size may have reduced the statistical power and restricted further subgroup analyses, such as the association between specific postoperative complications and emotional recovery. Future studies with larger cohorts and multicenter designs are required to confirm the robustness and generalizability of these findings. Third, although postoperative complications were evaluated as an important clinical outcome, this study mainly analyzed the overall incidence of complications. Different types of complications, including wound infection, pharyngeal fistula, pulmonary infection, and bleeding, may have different relationships with psychological status and recovery. Due to the sample size limitation, a detailed classification analysis was not performed. Future studies should investigate whether AI-assisted nursing has differential effects on specific postoperative complications. Fourth, although the AI multimodal emotion recognition system showed good performance in identifying perioperative depressive risk, several technical limitations remain. Facial swelling, tracheotomy, speech impairment, postoperative pain, and fatigue may affect the quality of facial and speech signal acquisitions. Although facial modality quality was assessed in this study, the influence of missing data, low-quality signals, and modality-specific degradation on model performance was not systematically evaluated. Future algorithm optimization should incorporate more robust quality control strategies and evaluate the model stability under different clinical conditions. Fifth, this study focused mainly on clinical outcomes and nursing response efficiency; implementation-related factors were not comprehensively evaluated. However, the effects of AI-assisted nursing on nurses’ workload, system usability, nursing compliance, workflow integration, and patient acceptance remain unclear. Since AI technology is intended to support, rather than replace, nursing professionals, future studies should incorporate human–computer interaction evaluations and real-world implementation assessments.

Future research should focus on conducting prospective multicenter randomized controlled studies with larger sample sizes to validate the effectiveness of AI-assisted individualized nursing. Additionally, future investigations should optimize the multimodal algorithms, improve transparency of the model architecture, and explore adaptive models that can better handle postoperative changes in facial, speech, and behavioral signals. Long-term follow-up should also be incorporated to evaluate the sustained effects on psychological recovery, quality of life, and healthcare utilization. Furthermore, cost-effectiveness analysis and evaluation of clinical workflow integration are important to determine the feasibility of applying AI-assisted emotional monitoring in routine perioperative nursing practice.

CONCLUSION

In conclusion, the individualized nursing model based on AI multimodal emotion recognition can detect emotional abnormalities earlier in patients with depression during the perioperative period of head and neck tumors and is conducive to improving the efficiency of nursing response, improving the perioperative emotional state, and promoting postoperative recovery. This model provides new ideas for perioperative psychological care and refined management of head and neck tumors and has certain clinical application prospects. This is worthy of further optimization and promotion in subsequent research and practice.