Application of the Belle.AI Comparative Image Reference System for Describing Chronic Ear Infections in Pediatric Patients From Low-Resource Communities

Timely access to primary care physicians for diagnosing and treating otitis media (infection or inflammation of the middle ear) in pediatric patients is often limited in rural, low-resource, and skin-of-color communities, particularly in remote or at-home settings.

This lack of access increases the risk of undiagnosed and untreated pediatric otitis media, potentially leading to severe and permanent complications, including hearing loss, deafness, tinnitus, impaired language development, cognitive deficits, and delayed educational development. A growing body of literature supports the accuracy (as measured by sensitivity and specificity of the model) of automated image analysis of middle ear conditions with machine learning-based algorithms. This proposed study will evaluate the effectiveness of a smartphone-based software that describes the physical nature of otoscopy images, powered by artificial intelligence (AI) and computer vision, in support of healthcare providers diagnosing acute otitis media and otitis media with effusion.

This pilot study will prospectively examine the value of Belle Otoscopy AI in describing otoscopy images in pediatric patients in low resourced communities who present with ear discomfort. It will also examine the utility of Belle.ai's companion software for healthcare providers caring for patients using commercially low-cost otoscope for at home image capture.

This project is funded in whole or in part with Federal funds from the Advanced Research Projects Agency for Health (ARPA-H), Department of Health and Human Services, under Contract No. 75N91023C00045.

A significant volume of research has been published over the past several years about diagnosing ear conditions through the AI methodology of deep learning on otoscopic images and supporting the accuracy of this analysis. Since 2020, studies have primarily trained convolutional neural networks to detect the tympanic membrane, with a strong focus on otitis media.

At-home ear infection detection can reduce healthcare costs by minimizing unnecessary visits to physicians and emergency rooms, lowering antibiotic prescriptions, and enabling more timely treatment for pediatric patients living in areas or communities where accessing healthcare is difficult.

A 2021 study published in JAMA Otolaryngology-Head \& Neck Surgery evaluated the feasibility of patient use of low-cost digital videoscopes and smartphones for examining the ear and oropharynx. Participants were instructed to capture images and videos of their ear canals and oropharynx using digital videoscopes and their smartphones under real-time guidance over a telehealth platform. Among the images obtained using the digital otoscope ear examination, 95% were considered acceptable by the healthcare clinicians and 91% were considered acceptable by a blinded reviewer. The mean time required to acquire images for both ears was 114 seconds (95% CI, 84-145 seconds). Twenty-one participants (91%) were willing to pay for a digital otoscope for telehealth use.(https://pubmed.ncbi.nlm.nih.gov/33475683/).

With this study the investigators plan to build on this framework by evaluating the ability of Belle Otoscopy AI's to describe ear physical conditions to providers and evaluate its performance in real-world telemedicine settings. This pilot will provide an opportunity to further explore the use of advanced tools, such as Belle Otoscopy AI and its potential benefit in an under-resourced telemedicine setting. This study would represent one of the first investigations into the prospective, real-time use of an AI tool within such a setting.

Belle Otoscopy AI's convolutional neural network image referencing technology utilizes deep learning algorithms to analyze an uploaded clinical image and then compare its geometric pattern characteristics to Belle's database of images to provide description of the image's physical characteristics and differential references. For example, the software identifies ear features such as purulence (pus), redness and swelling.

The images and analysis from the Belle Otoscopy AI app can be remotely viewed by healthcare professionals via the bellePro app or the Belle.ai web platform. After the AI powered image comparison is complete, the bellePro (not Belle Otoscopy AI) displays references of similar looking images. The AI also provides a description of the visual elements observed (e.g., redness, swelling, presence of pus) that prompted the AI model to predict an ear anomaly, adding an explainable AI component that aids the healthcare professional in understanding the rationale behind the AI's description.

Data collected from this study will be used for a report and research paper publication, including the possibility of developing additional studies. Also, data from this study will be preserved for any future possible submission to regulatory bodies for review.

The study will address three primary research questions:

1. Does Belle Otoscopy AI accurately describe ear physical features? 1a. How accurate is Belle Otoscopy AI on photos taken during the same visit by clinicians and parents of pediatric patients? 2. Are parents able to capture photos using an off-the-shelf otoscope that meet the AI's requirements, and can they successfully process the images through our product to obtain a result? 3. Is the proposed clinical workflow -- including use by parents at home in coordination with a remote healthcare provider -- effective? 3a. After successful photo capture and analysis by Belle Otoscopy AI, is there a smooth transition to consultation with a physician, should an ear infection be identified? 3b. For the intended use where the product is provided to parents on the day the child is sick, without requiring an in- clinic visit, will the proposed mechanism for getting the hardware and software into the hands of the patient work?

The following describes the workflow:

The parent brings the sick child into clinic with suspected ear infection. The parent asked for consent to participate in a clinical trial. Those who consent continue to the next step. Belle Otoscopy AI is downloaded onto the parent's phone, and they are given a low-cost, off-the-shelf (OTS) otoscope. The parent is trained to use the software and the hardware. The parent takes one photo of the middle ear using the OTS otoscope and processes it through Belle Otoscopy AI. The result is recorded. Prior to using Belle Otoscopy AI, the site clinician uses standard of care method to diagnose the presence of an ear infection. The clinician's preliminary diagnosis is recorded. The site clinician takes photo of the middle ear using the OTS otoscope and processes it through Belle Otoscopy AI. The clinician makes a final diagnosis and the result is recorded. The patient's treatment plan is determined based on the standard of care method, and the patient is treated according to the plan (e.g., prescribed antibiotics). Each patient will receive a subsequent appointment to follow up in person with the provider. At the halfway point between the initial and subsequent appointment, the parent will take a new photo of the middle ear using the at-home OTS otoscope and will submit the image through Belle app. The provider will review the image, and if the condition is determined to have resolved, the upcoming appointment will be canceled. For the purposes of this study, a panel of board-certified otolaryngologists will review all medically relevant information for each study participant to establish the gold standard. The investigators then compare the gold standard to determine: a.) did Belle Ear AI increase the accuracy of the clinician's final diagnosis; b.) did Belle Ear AI facilitate the cancellation of unneeded follow-up appointments; and c.) did Belle Ear AI as used by clinicians detect Acute Otitis Media and Otitis Media with Effusion with sufficient specificity and sensitivity in-clinic to be a diagnostic tool in-clinic.

Participants in this study will download Belle Otoscopy AI onto their personal smartphones and receive a low-cost otoscope to bring home. They will be encouraged to capture images of their middle ear condition at designated time points. These images will be analyzed for condition and severity and provided to the treating healthcare providers. As is standard at the clinic, all patients within this study will be scheduled for a two-week follow-up clinical visit, but some of these scheduled visits may be cancelled at the discretion of the healthcare provider's after viewing the Belle.ai images and analysis.

In total, one image of the presenting condition will be captured by the parent of the patient using the Belle Otoscopy AI tool. All images will be analyzed and compared to the image reference database, with the images along with analytic results will be provided to the treating healthcare provider. Participating healthcare providers will use this information to make a clinical determination if an in-person follow-up visit is required to provide ongoing care. Finally, all images, analytic results, and differential diagnoses will be reviewed by the medical review committee to assess the accuracy of the Belle Otoscopy AI tool in predicting the likelihood of middle ear infection.

When the parent of a pediatric patient initiates the clinic on-site clinical flow, they register the child and then are triaged by front-line clinical staff. At this stage the patient's primary complaint is recorded and entered into the Electronic Medical Record (EMR) system. Parents of patients identified as potentially presenting with a qualifying middle ear infection condition will be approached for recruitment into the trial. A dedicated study coordinator within the clinic setting will be flagged whenever a qualifying patient is identified within EMR system.

The study coordinator will meet with the parent to briefly explain that the patient may have an opportunity to participate in a clinical trial and will ask if the patient and their parent are interested in knowing more. If they express interest, the coordinator will provide further details about the trial.

The study coordinator will then explain the study's focus, design, and what is required for the patient to participate. Participation is entirely voluntary and parents of the minor patients will be asked to sign an IRB-approved consent form before any study procedures begin. The consent form briefly explains the Belle.ai AI technology, and how it may impact the care they will receive. The consent document describes the purpose of the study, details the potential risks and benefits, provides information on data security and privacy, and describes the compensation being provided for participation. In addition, the study coordinator will describe the actions the patient is asked to undertake, which include requests for them to provide two images of their middle ear condition, one approximately 7 days after treatment at the clinic and one on or around day 14 after enrollment. If the parent is unable or unwilling to capture these images they will not be allowed to enroll into the study and will be returned to the waiting area.

If the parent agrees to the at-home image capture at the time of the initial in-clinic visit setting and provides their written consent, their child will be officially enrolled in the study. The study coordinator will then have the parent download Belle Otoscopy AI onto their smartphone. A scannable QR-code will be provided that will automatically download, install, and verify that the application was correctly set up. The study coordinator will enter a pre-assigned Study ID code number into the app which will serve as the patient's study ID outside of the clinic setting. In addition, the coordinator will collect the parent's phone number, which will be stored along with their ID number in the clinic's database. These code numbers will be linked to the patient's EMR record; however, no ide

• Agreement between diagnoses [Time frame: Through study completion, an average of 14 days]
• Cost-Impact Analysis [Time frame: Through study completion, an average of 14 days]