Accuracy of artificial intelligence in prediction of osteoporotic fractures in comparison with dual-energy X-ray absorptiometry and the Fracture Risk Assessment Tool: A systematic review

Abstract

BACKGROUND

Osteoporotic fractures, whether due to postmenopausal or senile causes, impose a significant financial burden on developing countries and diminish quality of life. Recent advancements in artificial intelligence (AI) algorithms have demonstrated immense potential in predicting osteoporotic fractures.

AIM

To assess and compare the efficacy of AI models against dual-energy X-ray absorptiometry (DXA) and the Fracture Risk Assessment Tool (FRAX) in predicting fragility fractures.

METHODS

We conducted a literature search in English using electronic databases, including PubMed, Web of Science, and Scopus, for studies published until May 2024. The keywords employed were fragility fractures, osteoporosis, AI, deep learning, machine learning, and convolutional neural network. The inclusion criteria for selecting publications were based on studies involving patients with proximal femur and vertebral column fractures due to osteoporosis, utilizing AI algorithms, and analyzing the site of fracture and accuracy for predicting fracture risk using SPSS version 29 (Chicago, IL, United States).

RESULTS

We identified 156 publications for analysis. After applying our inclusion criteria, 24489 patients were analyzed from 13 studies. The mean area under the receiver operating characteristic curve was 0.925 ± 0.69. The mean sensitivity was 68.3% ± 15.3%, specificity was 85.5% ± 13.4%, and positive predictive value was 86.5% ± 6.3%. DXA showed a sensitivity of 37.0% and 74.0%, while FRAX demonstrated a sensitivity of 45.7% and 84.7%. The P value for sensitivity between DXA and AI was < 0.0001, while for FRAX it was < 0.0001 and 0.2.

CONCLUSION

This review found that AI is a valuable tool to analyze and identify patients who will suffer from fragility fractures before they occur, demonstrating superiority over DXA and FRAX. Further studies are necessary to be conducted across various centers with diverse population groups, larger datasets, and a longer duration of follow-up to enhance the predictive performance of the AI models before their universal application.

Key Words: Artificial intelligence; Osteoporosis; Prediction; Fragility fractures

Core Tip: Fragility fractures due to osteoporosis are a tremendous economic burden and cause pronounced morbidity and mortality. Early diagnosis allows the initiation of preventive measures to reduce the incidence of fractures. Dual-energy X-ray absorptiometry (DXA) and the Fracture Risk Assessment Tool (FRAX) are two inaccurate modalities used to predict fractures. The emergence of artificial intelligence (AI) has changed this scenario completely. We compared AI, DXA, and FRAX and found that AI models are 99% accurate in predicting an impending fracture compared with DXA and FRAX, which are about 70%. More studies on AI should be performed before made universally available.

INTRODUCTION

Osteoporosis is a disease associated with the aging process and is characterized by bone loss that increases susceptibility to fractures from minor falls. Osteoporotic fractures are prevalent worldwide, with reported prevalence rates of osteoporosis and osteopenia ranging from 19% to 41%[1,2], and the hospital-based prevalence exceeds 55% in Saudi Arabia[3]. If osteoporosis is not diagnosed early and treated appropriately, patients sustain fragility fractures. Fractures of the femur and spine lead to repeated hospital admissions and prolonged morbidity, which affects quality of life and imposes economic burdens. The economic impact of osteoporotic fractures is projected to rise significantly in the coming decades. The European Union spent €37 billion in 2013 on the care of fragility fractures, with an estimated increase of 25% by 2025[4]. In Saudi Arabia, direct and indirect costs for femur fracture care due to osteoporosis are expected to reach $9.4 billion by 2025[5], while costs in China may reach $25.43 billion by 2050[6].

With advancements in diagnostic modalities and treatment improvements, the general population is living longer, which may further increase the risk of osteoporosis and fractures. In the past two decades, fracture risk assessment tools have not evolved significantly, and the shortcomings of dual-energy X-ray absorptiometry (DXA) and the Fracture Risk Assessment Tool (FRAX) have not been adequately addressed[7]. DXA, which was developed for assessing fracture risk in postmenopausal females[8], remains the gold standard for diagnosing osteoporosis and monitoring treatment. However, the accuracy of DXA predicting fractures is less than 65%[9].

FRAX has emerged as an important risk calculator but lacks the inclusion of other critical factors that can help predict fracture risk prediction accuracy. Although FRAX is used globally, concerns have been raised that its thresholds may not be universally applicable and should be calibrated to specific regional populations. Intervention thresholds based on FRAX scores aim to prevent major and hip osteoporotic fractures, with claims that FRAX significantly improves fracture risk prediction. Quantitative CT has shown greater accuracy in creating a three-dimensional geometry of selected bone areas, but its cost, radiation exposure, and time requirements have limited its routine use[10,11].

Recently, the rapid progress of artificial intelligence (AI) in image recognition and medical diagnostics has raised hopes for more accurate evaluations and improved performance in predicting various diseases, including fragility fractures[12]. The accuracy of image analysis and recognition tools exceeds 95%, largely due to the development of convolutional neural networks (CNN), which are designed for feature extraction, classification, and segmentation[13]. Given the increasing financial burden of fragility fractures, it is essential to explore alternative tools that can accurately predict fracture risk. This review assessed whether AI can enhance accuracy in predicting fracture risk in patients with osteoporosis compared with DXA and FRAX.

MATERIALS AND METHODS

This review evaluated the accuracy of AI in predicting fragility fractures in comparison with DXA and FRAX. A literature search was conducted for English language studies using electronic databases, including PubMed, Web of Science, and Scopus, for publications until May 2024. The keywords searched included “fragility fracture,” “osteoporosis,” “AI,” “deep learning,” “machine learning,” and “convolutional neural network.” AI encompasses a range of algorithms, including machine learning (ML), deep learning (DL), and neural networks. DL is a subset of ML based on deep neural networks, while ML is a subset of AI. CNNs are among the most popular neural networks, excelling in image processing.

The inclusion criteria for publication selection focused on studies involving patients of either sex with proximal femur and vertebral column fractures due to osteoporosis and low-energy trauma, utilizing AI methods and algorithms such as ML, DL, and CNN. Studies that developed and/or validated models for fracture prediction were included, while conference abstracts and letters to the editor were excluded. References that compared the prediction models of DXA/FRAX and AI were added to the analysis. Studies that failed to meet the inclusion criteria were excluded from the analysis. Studies excluded were those published in languages other than English. Figure 1 provides details of the literature search conducted until May 2024.

Figure 1 PRISMA flowchart of systematic review.

The following data was extracted for the final analysis: Number of patients; sex; age; AI algorithm used; site of fracture; and accuracy for predicting fracture risk. The area under the receiver operating characteristic curve (AUC) represents model accuracy, with an AUC of 1.0 indicating a 100% probability of correct fracture prediction. The average sensitivity and specificity of the models were compared to those of DXA/FRAX. Data were analyzed using SPSS, version 29.0 (Chicago, IL, United States). Results are expressed as mean and standard deviation. Statistically significant differences between groups were determined using a Student’s t-test, with a P value of < 0.05 considered statistically significant.

RESULTS

We identified 156 publications that met our inclusion criteria, with 92 studies excluded from analysis for not fulfilling the criteria. The remaining 64 studies were analyzed in depth, and a further 51 references were removed due to duplicate data. Thirteen studies involving 24489 patients were included in the final analysis (Table 1)[14-26]. The majority of studies focused on males and females with fragility fractures of the proximal femur and vertebral column. The average AUC was 0.925 ± 0.69, indicating a high probability of accurate fracture prediction. The mean sensitivity was 68.3% ± 15.3%, specificity was 85.5% ± 13.4%, and the positive predictive value (PPV) was 86.5% ± 6.3%. In comparison, DXA showed a sensitivity of 37.0% and 74.0%, while FRAX demonstrated sensitivity of 45.7% and 84.7%. The P value for sensitivity between DXA and AI was < 0.0001, while for FRAX it was < 0.0001.

Table 1 Description of analyzed studies.

Ref.	AI	Number of patients	Sex	Site of fracture	Area under the curve
Dong et al[14], 2023	DL	1790	Male	Spine	0.95
Shimizu et al[15], 2022	ML	7000	Male/female	Hip	0.75
Kong et al[16], 2022	CNN	1595	NR	Hip/spine	0.97
Dong et al[17], 2022	DL	4461	Male	Spine	0.99
Ulivieri et al[18], 2021	ANN	172	Female	Spine	0.87
de Vries et al[19], 2021	ANN DeepSurv model	7578	Male/female	Spine and hip	NR
Villamor et al[20], 2020	Neural networks	137	Female	Hip	NR
Beyaz et al[21], 2020	CNN	65	Male/Female	Hip	0.98
Muehlematter et al[22], 2019	ML	58	Female	Spine	0.97
Ferizi et al[23], 2019	ML	32	Female	Hip	0.92
Kruse et al[24], 2019	ML	6606	Male/female	Hip and spine	0.91
Ho-Le et al[25], 2017	ANN	1167	Female	Hip	0.94
Tseng et al[26], 2013	ANN	434	Male/Female	Hip	0.87

AI: Artificial intelligence; ANN: Artificial neural network; CNN: Convolutional neural network; DL: Deep learning; ML: Machine learning; NR: Not reported.

DISCUSSION

This analysis indicated that the prediction of osteoporotic hip and spinal fractures using various AI algorithms had an accuracy of 92%, with AUC values ranging between 0.74-0.99), which surpassed the prediction accuracy of 65%-70% for DXA/bone mineral density and FRAX. Our analysis clearly demonstrated that different AI models can serve as valuable tools, assisting physicians in taking necessary corrective actions before a fracture occurs during osteoporosis management[27-30]. Despite the efficacy of DXA and FRAX to accurately predict an impending fragility fracture, finite element analysis and quantitative CT-based models are more accurate[29,30]. However, the cost, radiation dose, and time required did not allow these models to be universally accepted.

Depending on the prediction models of DXA and FRAX, the number of fractures was reduced. But it is expected that the global cost of managing hip fractures will rise to $131.5 billion annually by 2050[31,32], with estimates for Saudi Arabia reaching SR35 billion. In addition to the financial burden, social costs increase, with 1-year mortality rates after hip fractures reaching up to 30%[33-36]. Furthermore, the quality of life for surviving patients declines significantly. Therefore, it is imperative to address both the economic and social impacts and to develop alternative methodologies with technological support to accurately predict osteoporosis-related fractures before they occur.

Studies have shown that AI algorithms represent one such technology that can help physicians accurately determine fracture risk. Lex et al[37] analyzed 714939 hip fractures for validating and testing ML models and reported a sensitivity of 89.3% and specificity of 87.5% in diagnosing hip fractures. Similarly, Dong et al[14] achieved a sensitivity of 54.5% and 47.8%, specificity of 99.7% and 99.6%, and a PPV of 89.8% and 94.8% after prioritizing PPV. Other analyses of AI models reported an accuracy of an AI prediction model for hip fracture risk prediction as high as 90%[38]. Kong et al[16] compared a CNN-based prediction algorithm (DeepSurv) with FRAX in 1595 patients followed for 40.7 months. They found that DeepSurv outperformed both FRAX and Cox proportional hazard models in predicting fragility fractures. The development of fracture prediction algorithms necessitates large datasets with proper elucidation for correct classification, localization, and segmentation. For better comparisons between normal and abnormal bone quality, data from patients with fractures should be compared with those of healthy young individuals under 35 years of age.

The fracture parameters utilized by AI models focus on bone toughness, initial toughness, and unstable toughness in elastic crack models as well as early crack propagation leading to full-fledged fractures. Neural models predict fracture toughness and crack growth paths through a learning process. Additionally, neural networks utilize parameters such as microdefects that contribute to fractures. ML models capture the nonlinear relationships among high-dimensional data fed from images in fracture prediction. Trained neural network models have demonstrated robust potential and have achieved a correlation coefficient of ≥ 0.99[39].

In recent years, AI-based algorithms have been integrated into assessment and practice, including GoogLeNet (Inception Net), DeepSurv, Random Survival Forests, TWIST system, Light Gradient-Boosting Machine, PelviXNet, Feature Pyramid Network, Support Vector Machine with radial basis function, and XGBoost, all employing artificial neural networks, ML, DL, CNN, and neural networks.

Algorithms of ML, particularly those within the supervised learning subset, have outperformed other algorithms in analyses, including linear and logistic regression, which is applied to identify patterns for estimating fracture risk[40]. AUCs and precision-recall curves demonstrate the performance of AI algorithms. In our analysis, different models exhibited varying AUC levels, sensitivity, and specificity, indicating that some models outperform others due to the application of different unsupervised and semi-supervised learning techniques. Additional issues related to the comprehensive application of these models still need to be addressed to further enhance the algorithms that can outclass the present models like DXA and FRAX.

Our analysis had notable strengths. First, we compiled and compared the most recent data related to AI models against DXA and FRAX. Second, all AI algorithms were evaluated in this review. However, the study had limitations, including the analysis of only 24489 patients, which may limit the generalizability of our conclusions. Additionally, potential biases, overfitting issues, and concerns regarding the generalizability of different AI models and algorithms for a common goal of fracture prediction remain[41].

CONCLUSION

This review found that AI models are effective tools for analyzing and predicting fragility fractures before they occur and that AI outperformed DXA and FRAX. Further studies are needed across various centers with diverse population groups, larger datasets, and extended follow-up durations to enhance the accuracy before applying the AI algorithms universally.

ACKNOWLEDGEMENTS

All the authors sincerely thank Mr. Muhammed Ajmal Khan, Lecturer at Imam AbdulRahman Bin Faisal University, Dammam as without his help in collection of the publications this study could not have been successfully completed.