Artificial intelligence in ophthalmology and visual sciences: Current implications and future directions

Abstract

Since its inception in 1959, artificial intelligence (AI) has evolved at an unprecedented rate and has revolutionized the world of medicine. Ophthalmology, being an image-driven field of medicine, is well-suited for the implementation of AI. Machine learning (ML) and deep learning (DL) models are being utilized for screening of vision threatening ocular conditions of the eye. These models have proven to be accurate and reliable for diagnosing anterior and posterior segment diseases, screening large populations, and even predicting the natural course of various ocular morbidities. With the increase in population and global burden of managing irreversible blindness, AI offers a unique solution when implemented in clinical practice. In this review, we discuss what are AI, ML, and DL, their uses, future direction for AI, and its limitations in ophthalmology.

Key Words: Artificial intelligence; Ophthalmology; Retina; Machine learning; Eye care

Core Tip: Machine learning and artificial intelligence have evolved rapidly in recent years. Powerful machines and futuristic algorithms are bringing many possibilities towards the utilization of artificial intelligence in medical sciences. Ophthalmology is versatile in its adapting to newer and novel technologies earlier than other fields. Machine learning techniques assist clinicians and researchers in the detection and diagnosis of diseases as well as quantification of different disease biomarkers from ocular images. Interestingly, recent innovations like auto-machine learning has made it possible for clinicians, with little knowledge in computing and mathematics, to partake in creating, modifying, and training models tailored to their area of interest.

INTRODUCTION

Artificial intelligence (AI) refers to the ability of a machine to think independently. In 1956, it was first described by John McCarthy at his workshop in Darthmouth which is now considered as the birthplace of AI[1]. Later in 1959, Arthur Samuel defined machine learning (ML) as the ability of a machine to learn and improve with experience without being explicitly programmed[2,3].

The two major subfields of AI used in medicine are ML and deep learning (DL). ML derives information based on manually selected features and classifiers from already labeled data which is presented to the machine as a training dataset. This approach can be used with small datasets and requires comparatively shorter training time. In contrast, DL implements the use of artificial neural network (ANN) which is a complex system consisting of several layers of artificial neurons mimicking the neural network of human brain and its pattern recognition abilities. When input is provided to a DL algorithm, it is propagated through the multiple layers of the ANN and pattern recognition is performed by the DL algorithm itself without manual feature selection. Figure 1 illustrates the principle difference between ML and DL. The DL algorithms are fed large volumes of data containing both negative and positive examples (for instance, images of the healthy and diseased retina) for training. The algorithm autonomously trains itself and learns to recognize the differences between the two types of data, thus classifying it into positive and negative categories. The deep neural network (DNN) is a more efficient subtype of ANN in which the pattern recognition ability of the algorithm improves with the volume of training dataset. The larger the input data volume, the better the performance of the DNN at the given task. Another type of ANN is convolutional neural network (CNN) that has found its application in ophthalmology owing to its image recognition and classification ability. Although DL requires substantially larger training data and high computational power, the recent advances in technology and availability of graphics processing units have made its application in medicine and research more convenient[1-3].

Figure 1 Outline of principle difference between machine learning and deep learning.

For diagnosis and record-keeping, modern ophthalmology is dependent on imaging and large volumes of visual data are generated in the form of color fundus photographs (CFP) and scans from optical coherence tomography (OCT), OCT angiography, corneal topography, and other diagnostic procedures. The multimodal imaging approach allows the clinicians to view relevant structures in greater detail and provides them with useful information for decision-making in routine practice. The accurate processing of this large data volume can be cumbersome if efficient data processing methods are not accessible; however, its availability offers an optimal platform to bridge AI with ophthalmology as it is essential to analyze massive volumes of data for making data-driven decisions in the training of DL algorithms[4]. This review aims to summarize the applications of AI in ophthalmology, its limitations, and potential paths forward.

SCREENING FOR OCULAR DISEASES

With the increase in population, the burden of managing ocular disease has also increased. The need for regular follow-ups to timely detect and treat ocular adversities in the patients at risk can be challenging for the clinician as well as the patient. Diabetic retinopathy (DR), age-related macular degeneration (ARMD), and glaucoma are the leading causes of irreversible blindness worldwide. It has been estimated that 288 million people will suffer from ARMD while 600 million people will be affected by DR by the year 2040. The care of these disorders requires frequent follow-ups as the optimal time of treatment is at the early stage of the disease to prevent profound visual loss[5]. AI can play a huge role in the screening of ocular diseases in large populations where the care of an optometrist or ophthalmologist is not accessible to the masses. The screening of DR with the help of CFPs by utilizing AI has been well documented[6]. A summary of applications of AI in detection of various retinal diseases is given in Table 1. Moreover, studies have shown that DL can determine refractive errors from CFPs, which puts into perspective of how useful AI can be in extracting details from fundus images that are, otherwise, not discernable to human graders[7].

Table 1 Summary of artificial intelligence applications in detection of suspected diabetic retinopathy, age related macular degeneration, and glaucoma.

Ref.	Imaging modality	AI algorithm	Dataset for training	Dataset for validation	AUC	Sensitivity (%)	Specificity (%)
Diabetic retinopathy
Abràmoff et al[9], 2016	CFP	AlexNet and VGGNet	10000 to 1250000 images	Messidor-2: 1748	0.980	96.8	87
Gulshan et al[29], 2016	CFP	Inception-V3	128175 images	EyePACS-1: 8788 Messidor-2: 1745	0.9910.990	97.596.1	93.493.9
Ting et al[6], 2017	CFP	VGG -19	76370 images	SiDRP: 71896 images	0.936	90.5	91.6
				Guangdong: 15798	0.949	98.7	81.6
				SIMES: 3052	0.889	97.1	82
				SINDI: 4512	0.917	99.3	73.3
				SCES: 1936	0.919	100	76.3
				BES: 1052	0.929	94.4	88.5
				AFEDS: 1968	0.98	98.8	86.5
				RVEEH: 2302	0.983	98.9	92.2
				MEXICAN: 1172	0.95	91.8	84.8
				CUHK: 1254	0.948	99.3	83.1
				HKU: 7706	0.964	100	81.3
Abràmoff et al[30], 2018	CFP	AlexNet and VGGNet	10000 to 1250000 images	819 patients	N/A	87.2	90.7
Li et al[10], 2018	CFP	Inception V3	58790 images	8000 images for referable DR	0.989	97	91.4
Ruamviboonsuk et al[31], 2019	CFP	Inception V4	1665151 images	25326 images	0.987	96.8	95.6
Son et al[11], 2020	CFP	Custom CNN	95350 images	Two data sets: IDRiD: 144 images &	0.957 to 0.980	88.9-92.6	94.0- 100
				e-ophtha: 434 images	0.947 to 0.965	89.2-93.6	91.4 - 97.1
Age related macular degeneration
Ting et al[6], 2017	CFP	VGG-19	72610 images	35948 images	0.932	93.20	88.70
Lee et al[13], 2017	OCT scans - Spectralis	Modified VGG 16	80839 images	20163 images	0.974	92.64	93.69
Zapata et al[14], 2020	CFP	CNN 1 image type selection	53396	20% of training datasets	0.979	97.7	92.4
		CNN 1 CFP quality selection	150075		0.989	98.3	96.6
		CNN 1 OD/OS	30119		0.947	96.9	81.8
		AMDNET	8832		0.936	90.2	82.5
		Modified RESNET 50 (23) Referable GON	3776		0.863	76.8	83.8
Glaucoma suspect
Ting et al[6], 2017	CFP	VGG-19	125189 images	71896 images	0.942	96.40	93.20
Li et al[18], 2018	CFP		31745 images	8000 images	0.986	95.6	92

CFP: Color fundus photographs; AUC: Area under curve; GON: Glaucomatous optic neuropathy; OCT: Optical coherence tomography; DR: Diabetic retinopathy; SiDRP: Singapore integrated diabetic retinopathy screening program; CNN: Convolutional neural network; VGG-19: Visual geometry group; CUHK: Chinese University Hong Kong; HKU: Hong Kong University; N/A: Not available; RVEEH: Royal Victorian Eye and Ear Hospital; SCES: Singapore Chinese Eye Study; SIMES: Singapore Malay Eye Study; SINDI: Singapore Indian Eye Study; BES: Beijing Eye Study; IDRiD: Indian Diabetic Retinopathy Image Dataset; AFEDS: African American Eye Disease Study.

DIABETIC RETINOPATHY

DR is the most prevalent cause of irreversible blindness in adults. The progressive nature of the disease requires vigilant monitoring of the retina over time to initiate treatment as soon as possible. The early treatment of DR is the key to avoid visual impairment in the working-age groups which experience visual impairment or blindness by the ocular complications of diabetes mellitus (DM). Therefore, yearly follow-ups are required for the clinical examination of the eye in patients with DM, which presents a challenge to the ophthalmic community particularly in countries where medical services are not easily available to people. Moreover, about half of the patients fail to stick to their follow-up regimen[1,5]. To screen a large group of people and to keep efficient and timely follow-ups, AI can help reduce the burden by providing the convenience of quick analysis of large datasets[8].

The use of DL in ophthalmology has seen a rapid increase after its successful application for screening DR was reported in multiple papers in 2016[2]. Abràmoff et al[9] conducted a study in 2016 using a validation dataset of 1748 images and a DL algorithm to detect referable DR from CFPs. Their algorithm achieved an accuracy of 98% with a sensitivity of 96.8% and specificity of 87% in detecting vision threatening referable DR. In another study, Ting et al[6] trained an algorithm with a total of 494661 CFPs obtained from a population of ten various ethnic origin groups for detecting referable DR, ARMD, and glaucoma. In the validity dataset, the area under receiver operating curve (AUC), sensitivity, and specificity for referable DR were 0.936, 90.5%, and 91.6%; for vision-threatening DR were 0.958, 100%, and 91.1%; for possible glaucoma were 0.942, 96.4%, and 87.2%; and for ARMD were 0.931, 93.2%, and 88.7% respectively[6]. In 2018, inception V3, a DL algorithm, was trained by Li et al[10] with 58790 CFPs for the detection of DR. The model had an AUC of 0.989 with a sensitivity of 97% and specificity of 91.4%. It was also reported that 77.3% of false negatives were due to the undetected intraretinal microvascular abnormalities[10]. Son et al[11] developed a DL algorithm based on 103262 macula centered retinal photographs to detect hemorrhages, hard exudates, cotton-wool spots, macular hole, myelinated nerve fiber layer, chorioretinal atrophy, retinal nerve fiber layer (RNFL) defect, vascular abnormalities, glaucomatous disc change, and nonglaucomatous disc change. They reported that the DL accurately and reliably detected multiple abnormalities of the retina and recommended that DL could be used as a screening tool for routine clinical practice[11]. The success of DL in detecting vision threatening DR shows that screening for DR can be carried out by utilizing AI in clinical practice, particularly, in the areas where direct access to an eye care provider is not available.

AGE-RELATED MACULAR DEGENERATION

ARMD is a progressive disease of the retina and is one of the major causes of irreversible blindness in developed countries. The early stage of the disease can stay quiescent for several years without causing any further visual deterioration; however, it can rapidly progress to advanced geographic atrophy (GA) or CNV. The development of CNV can cause profound visual loss if not treated at the earliest, which makes the observation of the at-risk population indispensable[1]. For the screening of ARMD, Venhuizen et al[12] trained an algorithm on 3256 OCT scans to identify five stages of ARMD: No ARMD, early ARMD, intermediate ARMD, advanced GA, and advanced CNV. On a test dataset of 384 OCT scans, the algorithm had a sensitivity of 98.2%, specificity of 91.2%, and AUC of 0.980, thus performing fairly well at the given task[12]. Lee et al[13] used 80839 OCT scans for the training of a DL model and 20163 scans as the validation dataset to detect ARMD and achieved an AUC of 0.974 with a sensitivity of 92.64% and specificity of 93.69%[13]. In 2020, Zapata et al[14] used 8832 CFPs as a training dataset to classify the images into three stages of ARMD as early ARMD, intermediate ARMD, and advanced ARMD. Their model achieved an AUC of 0.936 with a 90.2% sensitivity and 82.5% specificity[14].

Another potential use of AI is to predict visual acuity outcomes and disease progression. The visual acuity outcomes of patients being treated with anti-vascular endothelial growth factor treatment (Anti-VEGF) are rather erratic. If AI could help the clinician decide which patients will have good functional response post-therapy, it would reduce the burden of extra treatment. One such study was conducted by Schmidt-Erfurth et al[15] in which they trained a DL model over one year to predict visual acuity outcomes after Anti-VEGF therapy. The model was able to predict with a 71% accuracy[1].

As the course of ARMD progression is unpredictable in most cases, some studies have addressed this matter by applying AI in an attempt to predict the development of CNV. Schmidt-Erfurth et al[16] trained a model on 495 patients with intermediate ARMD in one eye and CNV in the fellow eye for 24 mo. The model was able to predict CNV development with an accuracy of 68% and development of GA with an accuracy of 80%[1]. Likewise, the progression of GA, its speed, and its course have also been investigated with the help of AI. Niu et al[17] reported a successful model trained for 2.25 years on 38 eyes for predicting GA progression. The model accurately projected the future direction of GA development. The major biomarkers that governed this prediction by the model were thinning of outer retinal layers and reticular pseudodrusen[17]. These studies show the benefits that implementation of AI in clinical practice can help in screening, management, and future prediction of disease progression. However, for the introduction of AI in routine practice, more research work is crucial in future by training the algorithms on larger datasets and studying their use in clinical practice.

GLAUCOMA

Glaucoma is a neurodegenerative disease that leads to irreversible loss of vision and is the second most prevalent cause of global blindness. The patient remains asymptomatic in the early stages of most types of glaucoma and only a comprehensive eye examination may detect the pathology. The diagnosis of glaucoma consists of optic nerve examination, visual field assessment (VFA), corneal thickness profile, anterior chamber assessment, and RNFL analysis. Owing to the lack of eye care professionals in developing countries and the limited availability of adjunct imaging devices, the need of an AI model for screening the disease efficiently is inevitable[5]. Previous studies have focused on the diagnosis of glaucoma by implementation of AI with the c-d ratio of the optic disc, neuroretinal rim width, and ISNT rule; however, the diagnosis of glaucoma without VFA remains incomplete[1].

In 2018, Li et al[18] developed a DL system by training it on 48116 CFPs to detect referable glaucomatous optic neuropathy. Their algorithm achieved an AUC of 0.986, sensitivity of 95.6%, and specificity of 92%. The false-negative results obtained were due to high myopia, DR, and ARMD while the false positives were attributed to physiological cupping of the optic disc by the authors[18]. In another study, to diagnose glaucoma from VFA and RNFL thickness, Kim et al[19] trained and compared various ML approaches. They found that the random forest model gave the most accurate result with an AUC of 0.979, sensitivity of 0.983, and specificity of 0.975 while distinguishing between healthy and glaucomatous eyes[19]. A DL model was implemented on the macular RNFL thickness and ganglion cell complex layer thickness to diagnose open angle glaucoma by Asaoka et al[20]. The DL model had an AUC of 93.7%, whereas the AUC decreased to 82% and 67.4% with random forest and support vector model, respectively[20].

ANTERIOR SEGMENT DISORDERS

The growing use of imaging for anterior segment disease management and diagnosis has facilitated the application of AI in this area. Recent studies have shown that AI algorithms can successfully differentiate between keratoconic and normal eyes from the corneal topography scans. Reportedly, KeratoDirect, a CNN integrated algorithm, was trained on 3000 scans containing 50% healthy scans and 50% scans from keratoconic eyes. When tested on a final set of 200 eyes, it distinguished between the normal and ectatic eyes with a 99.3% success rate[2]. By using corneal SS-OCT scans of 3156 eyes, Yousefi et al[21] developed and trained an unsupervised algorithm that distinguished between normal and keratoconic corneas with a specificity of 97.4% and sensitivity of 96.3%. Moreover, the algorithm included a small number of normal eyes in the category of mild keratoconus which, according to the authors, represented form fruste keratoconus and needed further evaluation[21].

Owing to the development of Ocular Response Analyser and Corvis ST for the assessment of corneal biomechanics, it has become possible to evaluate the corneal ectatic disorders in greater detail. The development of Corvis, which used Scheimpflug camera with non-contact air-puff tonometer to evaluate the central 8mm horizontal cornea at a rate of 140 images per 33 ms, has yielded new parameters to study corneal ectasia. With the implementation of AI, Ambrosio et al[22] combined these parameters with corneal topographical data leading to the development of Tomographic and Biomechanical Index (TBI). TBI has not only detected the mild forms of corneal ectasia but it has also been suggested that TBI provided data about the susceptibility of the cornea to developing ectasia. It can play an important role in the pre-operative assessment for laser vision correction to rule out patients that might be at risk of developing postoperative complications[23,24]. AI has also been implemented in the grading of nuclear sclerosis. Recent studies have shown improvement in the grading of nuclear sclerosis from cross-sectional slit-lamp images of the lens with CNN as compared to its previous attempts[25].

FUTURE DIRECTIONS

The digital revolution has changed the pace of medicine globally. New treatments are being discovered and new investigative technologies are being introduced; meanwhile, the patients are growing older and co-morbidities are increasing. AI has been successfully integrated in the field of radiology and dermatology to make the decision-making process easier for clinicians. It has also been applied to screen people who cannot reach eye care services.

DR, ARMD, and glaucoma are the leading causes of blindness worldwide. These pathologies result in irreversible blindness which can be prevented if they are timely detected and treated. In rural areas and developing countries, there is a lack of eye care professionals and facilities. In future, the utilization of AI based screening strategies coupled with telemedicine can make it possible to screen the populations at risk in a time and cost- efficient manner. For the screening of DR, an FDA approved hybrid algorithm, IDx-DR, is currently in use to detect referable and non-referable cases of DR[26]. Moreover, clinicians and researchers are working on training AI models by using larger datasets to enhance the already available models. Currently, an improved TBI model is in progress by training the model with bigger dataset[24].

BARRIERS

The process by which a DL algorithm learns pattern recognition from the training data remains largely unknown, which is often termed as the “black box”, consequently making it harder for the researchers to understand how the algorithm reaches its final decision. Moreover, the process of troubleshooting and debugging becomes inexplicable unless the researcher becomes familiar with the ANN.

Although the multimodal approach in ophthalmology has helped in attaining large datasets of digital images, the easy access to the data of patients can pose ethical challenges. Furthermore, the digital data may also be subject to cyberattacks[1]. Despite the recent advances that AI has made in ophthalmology, most of the successful ML models have not been validated or used in actual clinical practices where the machine models, cameras, and image quality vary from each other. Therefore, further studies need to be done by testing these models in real-world settings[27]. Lastly, another limitation of AI is the use of two-dimensional images for the training of DL algorithms, which makes the detection of space-occupying and three-dimensional lesions impractical. In future, the inclusion of stereoscopic images in training and validation datasets might address this challenge[28].

CONCLUSION

AI has revolutionized the world of medicine and ophthalmology in recent years. The success of DL in detecting ophthalmic pathologies in recent years is well proven; however, its implementation in routine practice is rare. Future research is crucial to address the challenges and limitations of AI in order to make it a part of daily practice in eye clinics.