Testing an AI Tool to Help Primary Care Clinicians With Specialty Consultation Questions
For patients and families
In plain language
An automatic summary of structured registry data. It is an orientation aid, not a substitute for the official protocol or a physician assessment.
- What is being studied
- The protocol lists: SAGE (AI clinical decision support), Standard eConsult.
- Who it may be relevant to
- Registry conditions: Decision Support Systems, Clinical, Referral and Consultation. Basic parameters: from 18 years · All.
- What needs checking
- Age, condition and sex are only basic indicators. Prior treatment, laboratory values and other mandatory requirements appear in the eligibility criteria below.
- Where it takes place
- United States
- Next step
- Save the trial, show it to the treating physician, and confirm current recruitment with the study center. Costs, documents and travel →
Unsure about the terms? Read our patient guide →
Official title
Physician Usability and Output Quality Evaluation of a Retrieval-Augmented Language Model System for Specialty Medical Consultation: A Human-Computer Interaction Study
Overview
The goal of this study is to test an artificial intelligence (AI) tool called SAGE. SAGE helps primary care doctors with questions that often need a specialist. Primary care doctors are the doctors people usually see first. SAGE reviews a case and suggests what a specialist might advise. The main questions it aims to answer are: * Do doctors make sound, timely care decisions when they use SAGE? * Do those decisions match what a specialist would advise? Researchers will compare the decisions doctors make with and without SAGE. Doctors in the study will: * Review made-up patient cases (these are not real patients) * Make a decision for each case, their usual way and with SAGE
Interventions
- Other SAGE (AI clinical decision support)
SAGE is an artificial intelligence (AI) based clinical decision support tool. It generates specialty-informed recommendations for a primary care consultation question. Physicians review the SAGE output while making a management decision for the synthetic case. - Other Standard eConsult
A standard electronic consultation (eConsult) format, without SAGE. Physicians review this information while making a management decision for the synthetic case.
Primary outcome measures
- Clinical actionability of the clinician's decision [Time frame: Post-session independent review, through study completion (up to approximately 2 months)]
- Concordance of the clinician's decision with the specialist reference answer [Time frame: Post-session independent review, through study completion (up to approximately 2 months)]
Secondary outcome measures (3)
- System usability (System Usability Scale, SUS) [Time frame: End of study session (Day 1, approximately 60 minutes)]
- Order acceptance in the SAGE condition [Time frame: During study session (Day 1)]
- Time to complete case review [Time frame: During study session (Day 1)]
Eligibility criteria
Inclusion criteria
- Physician practicing in primary care internal medicine and/or family medicine
- Attending physician, or resident physician at postgraduate year 2 (PGY-2) or higher
- Stanford-affiliated (faculty, staff, or trainee)
- Able to complete a single, approximately 60-minute remote (Zoom) session conducted in English
Exclusion criteria
- Not currently practicing in primary care internal medicine or family medicine
- Member of the study team or otherwise involved in the development of SAGE
- Unable to complete the remote study session
Criteria are shown verbatim from the registry (in English). Final eligibility is always assessed by the study center.
Healthy volunteers: Yes
Study design
- Allocation
- N/A
- Model
- Single group
- Masking
- Open label
- Primary purpose
- Health services research
Study locations
United States · 1 center
- Stanford University — Stanford
Identifiers
NCT: NCT07706920 · 85430