Clinical Language Evaluation With AI for Residents
For patients and families
In plain language
An automatic summary of structured registry data. It is an orientation aid, not a substitute for the official protocol or a physician assessment.
- What is being studied
- The protocol lists: educational LLM-based feedback tool.
- Who it may be relevant to
- Registry conditions: Patient Communication. Basic parameters: 18 years — 50 years · All.
- What needs checking
- Age, condition and sex are only basic indicators. Prior treatment, laboratory values and other mandatory requirements appear in the eligibility criteria below.
- Where it takes place
- United States
- Next step
- Save the trial, show it to the treating physician, and confirm current recruitment with the study center. Costs, documents and travel →
Unsure about the terms? Read our patient guide →
Official title
Clinical Language Evaluation With AI for Residents (CLEAR2) - A Pilot Randomized Controlled Trial
Overview
The purpose of this study is to refine and test existing enterprise-grade large language model (LLM) based on generative artificial intelligence (AI), to assess the feasibility and acceptability of LLM-based feedback, to assess the ability of LLM-based feedback to improve residents' communications,to explore the ability of standardized patients to assess residents' communication and to explore the ability of residents to self-assess their communication complexity
Interventions
- Behavioral educational LLM-based feedback tool
Participants will have their verbal communications with standardized patients (SP) regarding 3 different scenarios recorded, transcribed, and analyzed in real-time by the large language model (LLM) and will receive feedback as suggestions and alternative scripts. These will be reviewed by residents between SP scenarios
Primary outcome measures
- Readability discernment as assessed by a survey [Time frame: end of intervention ( 1 hour after baseline)]
- Quality discernment as assessed by a survey [Time frame: end of intervention ( 1 hour after baseline)]
- Correctness of recommendations as assessed by a survey [Time frame: end of intervention ( 1 hour after baseline)]
- Applicability of recommendations as assessed by a survey [Time frame: end of intervention ( 1 hour after baseline)]
- Perceived readability of resident-standardized patient (SP) interactions as assessed by a survey: schooling level [Time frame: end of intervention ( 1 hour after baseline)]
- confidence in communication ability [Time frame: end of intervention ( 1 hour after baseline)]
- usefulness of the LLM [Time frame: end of intervention ( 1 hour after baseline)]
- acceptability of future use [Time frame: end of intervention ( 1 hour after baseline)]
Secondary outcome measures (5)
- Survey feedback on the LLM interface [Time frame: end of intervention ( 1 hour after baseline)]
- readability grade level of resident-SP transcripts as assessed by the Flesch-Kincaid Grade Level (FKGL) readability tool [Time frame: end of intervention ( 1 hour after baseline)]
- Quality based on Ensuring Quality Information for Patients (EQIP) score of resident-SP transcripts [Time frame: end of intervention ( 1 hour after baseline)]
- Perceived readability of SP-resident interactions as assessed by a standardized survey [Time frame: end of intervention ( 1 hour after baseline)]
- confidence in communication ability [Time frame: end of intervention ( 1 hour after baseline)]
Eligibility criteria
Inclusion criteria
- McGovern Medical School (MMS) general surgery residents
- postgraduate year (PGY) 1-5
Criteria are shown verbatim from the registry (in English). Final eligibility is always assessed by the study center.
Healthy volunteers: Yes
Study design
- Allocation
- Randomized
- Model
- Parallel assignment
- Masking
- Open label
- Primary purpose
- Other
Study locations
United States · 1 center
- The University of Texas Health Science Center at Houston — Houston
Identifiers
NCT: NCT07222644 · HSC-MS-25-0920