Application of the Belle.AI Dermatological Image Reference System for Patient Diagnosis in an Active Clinical Setting

Access to dermatologists is often limited, leading to around 60% of skin, hair, and nail issues being treated by non-specialists. This study will evaluate the effectiveness of an AI dermatology decision support tool in assisting primary care providers (PCPs) with the diagnosis of skin conditions. AI-based image analysis has been shown to enhance diagnostic accuracy, particularly for non-dermatologists. Previous studies have primarily focused on dermatologists, but AI could be more beneficial for PCPs, as it has been shown to improve their diagnostic accuracy and agreement with dermatologists.

Globally, about 1.9 billion people suffer from skin diseases annually, with 1 in 3 Americans seeking dermatological care from non-specialists. Skin-related issues make up a significant proportion of visits to general practitioners and emergency departments. AI has proven effective in diagnosing skin conditions such as melanoma and other inflammatory diseases, and studies indicate that AI tools can enhance diagnostic accuracy, particularly for non-dermatologists.

The Belle AI tool, which will be used in this study, employs a convolutional neural network trained on over 500,000 images to identify over 2,000 skin conditions. It provides image match scores to help physicians identify conditions and offers a protocol for second opinions from board-certified dermatologists. The study aims to assess the tool's utility in real-time clinical settings, with potential improvements in triage accuracy, referral quality, and cost savings.

This study is supported by the Advanced Research Projects Agency for Health (ARPA-H) and will be one of the first to prospectively examine AI's impact on dermatology decision support in primary care.

The study aims to evaluate the accuracy and utility of the Belle AI dermatological reference system in a real-world clinical environment, in partnership with Urban Health Plan (UHP). Key endpoints include assessing diagnostic utility and accuracy compared to a final diagnosis from a dermatological review committee, as well as gathering feedback from primary care providers and physician extenders on their experiences with the AI. The sponsor will also analyze the cost implications of the system's use to demonstrate its value in frontline medicine.

Participants will use a smartphone app to capture images of their skin conditions, which will then be analyzed by the AI. Participants will receive financial incentives for submitting images after their initial visit. A follow-up appointment will be scheduled two weeks after the initial consultation, though some visits may be canceled based on the AI analysis.

Participants will be included if the participants present with a primary dermatological complaint and can provide informed consent. Exclusions apply to those unable to comply with procedures or pediatric participants with genital conditions for privacy reasons.

Upon entering UHP, participants register and are triaged. Those with qualifying dermatological conditions will be approached for recruitment by a study coordinator, who will explain the study and obtain consent. Participants will download the Belle Image Capture App to their smartphones, where a Study ID code linked to their EMR will be created, ensuring privacy.

During the initial appointment, providers will examine the patient, document their history, and diagnose the condition. The BellePro Physician App will be used to capture images and generate a differential diagnosis, which the provider will review before making a final diagnosis. Participants will be scheduled for a follow-up visit, and the study coordinator will notify providers of any received images captured through the app.

Beginning seven days after enrollment, push notifications will prompt participants to submit images using the app. The coordinator will follow up with participants who do not respond, aiming for a clear image within a specified timeframe. Upon receipt, the provider will reassess the diagnosis using updated AI analysis. Decisions regarding follow-up appointments will be communicated by the coordinator.

If a follow-up appointment is deemed unnecessary, participants will still be asked to submit images on Day 14. The coordinator will follow up similarly to ensure compliance.

The study spans from Day 1 (initial clinic visit) to Day 14-18, when final images or follow-up appointments will occur. Case notes will be updated continuously in eClinicalWorks, determining whether cases are resolved or require ongoing care.

Primary care providers at UHP will undergo onboarding, including an electronic intake survey and training on the BellePro Physician App via group video chat. Providers will be trained on the app's use, and their feedback will be collected in an exit survey to evaluate their experiences and willingness to continue using the AI system.

Given the complexity of dermatological diagnoses, a review committee of senior board-certified dermatologists will confirm diagnoses from the study. The review process involves three phases: initial image assessment, review of redacted medical records, and consideration of AI analysis results. The committee's consensus will determine the final diagnosis, which will be documented for analysis. Cases lacking a unanimous decision will be excluded from the study's final evaluation. Reviews will occur once enrollment is complete.

• Agreement between diagnoses [Time frame: Through study completion, an average of 14 days]
• Provider-Rated Usefulness [Time frame: Through study completion, an average of 14 days]
• Cost-Impact Analysis [Time frame: Through study completion, an average of 14 days]