Review of “LOCSIII-based artificial intelligence program for automatic cataract grading”

Review of “LOCSIII-based artificial intelligence program for automatic cataract grading”

ASCRS News: EyeWorld Journal Club
December 2022

by Tirth J. Shah, MD,* Zachary Q. Mortensen, MD,* Michael D. Abramoff, MD, and Thomas A. Oetting, MD
*Co-first authors 

Thomas A. Oetting, MD
Thomas A. Oetting, MD
Associate Residency Program Director
Department of Ophthalmology and Visual Sciences
University of Iowa
Iowa City, Iowa

The rapid rise of artificial intelligence (AI) has revolutionized the field of medicine. As an image-centric specialty, ophthalmology stands at the frontier of AI applications. The implementation of AI has already provided large-scale early screening and detection of eye diseases, improved health disparities through better access, and decreased cost, in both the U.S. and low-income countries. Comparatively, the development of AI for evaluation of the lens is still in its infancy.1,2,3 Lu et al. set out to establish and validate an AI-assisted automatic cataract grading program based on the Lens Opacities Classification System III (LOCSIII).4

Methods

The authors prospectively analyzed an internal dataset of slit lamp photographs of the anterior segment of cataract-affected eyes taken between 2018 and 2020 at the Fudan University Department of Ophthalmology. The group also obtained an external dataset of slit lamp photographs from March 2018 to August 2019 from the Pujiang Eye Study in Shanghai. All photographs were taken under pharmacologic mydriasis. The internal dataset was used to train, validate, and test, while the external dataset was used for testing.

Tirth J. Shah, MD
Tirth J. Shah, MD Ophthalmology Resident
University of Iowa
Iowa City, Iowa

Each patient received slit beam imaging for nuclear cataract, diffuse illumination imaging for cortical cataracts, and retroillumination for posterior subcapsular cataracts. Eyes with small pupils, blurred region of interest, or any corneal disease that interfered with lens observation were excluded. The images were graded based on LOCSIII. Nuclear cataracts were graded on a scale from 1.0 to 6.0 with respect to nuclear color. Cortical and posterior subcapsular cataracts were graded on a scale from 1.0 to 5.0. The images were graded by at least two experienced ophthalmologists, and the reference standard was the average of the two experts. A grade of 3.0 or greater was regarded as “moderate to severe” requiring referral. Advanced deep learning algorithms, including Faster R-CNN and ResNet, were applied to identify the capture modes, annotate regions of interest, and grade the cataracts. Interobserver repeatability among the various ophthalmologists was assessed with intraclass correlation coefficients (ICC) using a mixed-effects model.

Zachary Q. Mortensen, MD
Zachary Q. Mortensen, MD Ophthalmology Resident
University of Iowa

Results

A total of 847 slit beam, 326 diffuse illuminated, and 155 retroilluminated photographs were taken from the internal dataset, and 192 slit beam, 92 diffuse illuminated, and 71 retroilluminated photographs were taken from the external dataset. Interobserver reproducibility of cataract evaluations yielded ICC values of 0.958, 0.718, 0.787, 0.733, 0.835, and 0.780 for nuclear-internal, nuclear-external, cortical-internal, cortical-external, posterior subcapsular-internal, and posterior subcapsular-external cataracts, respectively. The authors found 99.4% of the internal photographs and 100% of the external photographs had an absolute predictive error of ≤1.0 for grading nuclear cataracts. The accuracy of grading a nuclear cataract was found to be 93.6% and 92.7% for the external dataset, respectively. For grading cortical cataracts, the authors found 75% of the internal dataset and 93.5% of the external dataset had an absolute prediction error of ≤1.0 The accuracy of grading a cortical cataract was found to be 84.4% and 80.4% for the internal and external dataset, respectively. In contrast, the grading of posterior subcapsular cataracts was found to be inconsistent with no clinical value.

University of Iowa residents and faculty; top row: Arnulfo Reyes, MD, Brandon Baksh, MD, Patrick Donegan, MD, Cheryl Wang, MD, Joanna Silverman, MD, Chad Lewis, MD, Samuel Tadros, MD, Bilal Ahmed, MD, Matthew Meyer, MD; middle row: Caroline Yu, MD, Sean Rodriguez, MD, Mahsaw Motlagh, MD, Aaron Dotson, MD; bottom row: Program Director Pavlina Kemp, MD, Tirth Shah, MD, Michael Abramoff, MD, Zachary Mortensen, MD Source: University of Iowa
University of Iowa residents and faculty; top row: Arnulfo Reyes, MD, Brandon Baksh, MD, Patrick Donegan, MD, Cheryl Wang, MD, Joanna Silverman, MD, Chad Lewis, MD, Samuel Tadros, MD, Bilal Ahmed, MD, Matthew Meyer, MD; middle row: Caroline Yu, MD, Sean Rodriguez, MD, Mahsaw Motlagh, MD, Aaron Dotson, MD; bottom row: Program Director Pavlina Kemp, MD, Tirth Shah, MD, Michael Abramoff, MD, Zachary Mortensen, MD
Source: University of Iowa

Discussion

The implementation of AI has already provided large-scale early screening and detection of eye disease, improved health disparities through better access, and decreased cost, in both the U.S. and low-income countries. Comparatively, the development of AI for evaluation of the lens is still in its infancy.

In the field of ophthalmology, AI offers a potential solution to address deficiencies of cataract evaluation, particularly in developing countries. This study shows that adequate performance can be achieved with relatively small training sets in grading and providing appropriate referral for nuclear cataracts. However, limitations remain, especially with regard to bias, before algorithms like this can be used for research and patient care. The AI was evaluated on a monoracial sample that was very similar to that of the training set, which leads to performance overestimation. In addition, it may be helpful to know how the images were captured and whether this can be generalizable to another setting, where there may be more variability in ophthalmic examination instruments and slit lamp digital cameras.

While LOCSIII is effective for standardizing and grading cataracts, it may be challenging to apply in the clinical setting because indications for surgical intervention rely largely on patient visual function. However, such a system of automated cataract grading may have potential for utility in research, such as monitoring cataract progression in large scale drug trials.

Overall, we commend the authors on their impressive work. We hope their work adds to the growing AI literature and contributes to furthering research and addressing healthcare disparities. 

Lens Opacities Classification System III-based artificial intelligence program for automatic cataract grading

Lu Q, et al. J Cataract Refract Surg. 2022;48:528–534

  • Purpose: To establish and validate an artificial intelligence (AI)-assisted automatic cataract grading program based on the Lens Opacities Classification System III (LOCSIII).
  • Setting: Eye and Ear, Nose, and Throat (EENT) Hospital, Fudan University, Shanghai, China.
  • Design: AI training
  • Methods: Advanced deep-learning algorithms, including Faster R-CNN and ResNet, were applied to the localization and analysis of the region of interest. An internal dataset from the EENT Hospital of Fudan University and an external dataset from the Pujiang Eye Study were used for AI training, validation, and testing. The datasets were automatically labeled on the AI platform in terms of the capture mode and cataract grading based on the LOCSIII system.
  • Results: The AI program showed reliable capture mode recognition, grading, and referral capability for nuclear and cortical cataract grading. In the internal and external datasets, 99.4% and 100% of automatic nuclear grading, respectively, had an absolute prediction error of ≤1.0, with a satisfactory referral capability (area under the curve [AUC]: 0.983 for the internal dataset; 0.977 for the external dataset). 75.0% (internal dataset) and 93.5% (external dataset) of the automatic cortical grades had an absolute prediction error of ≤1.0, with AUCs of 0.855 and 0.795 for referral, respectively. Good consistency was observed between automatic and manual grading when both nuclear and cortical cataracts were evaluated. However, automatic grading of posterior subcapsular cataracts was impractical.
  • Conclusions: The AI program proposed in this study shows robust grading and diagnostic performance for both nuclear and cortical cataracts, based on LOCSIII.

References

  1. Abràmoff MD, et al. Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci. 2016;57:5200–5206.
  2. Pead E, et al. Automated detection of age-related macular degeneration in color fundus photography: a systematic review. Surv Ophthalmol. 2019;64:498–511.
  3. Devalla SK, et al. Glaucoma management in the era of artificial intelligence. Br J Ophthalmol. 2020;104:301–311.
  4. Lu Q, et al. Lens Opacities Classification System III-based artificial intelligence program for automatic cataract grading. J Cataract Refract Surg. 2022;48:528–534. 

Contact 

Mortensen: zachary-mortensen@uiowa.edu
Oetting: thomas-oetting@uiowa.edu
Shah: tirth-shah@uiowa.edu

Tags: , ,