Use of Artificial Intelligence to Assess Trainee Communication Compared to Human Assessment
For patients and families
In plain language
An automatic summary of structured registry data. It is an orientation aid, not a substitute for the official protocol or a physician assessment.
- What is being studied
- The protocol lists: Communication Assessment through AI and Human.
- Who it may be relevant to
- Registry conditions: Communication Skills. Basic parameters: 18 years — 99 years · All.
- What needs checking
- Age, condition and sex are only basic indicators. Prior treatment, laboratory values and other mandatory requirements appear in the eligibility criteria below.
- Where it takes place
- United States
- Next step
- Save the trial, show it to the treating physician, and confirm current recruitment with the study center. Costs, documents and travel →
Unsure about the terms? Read our patient guide →
Official title
Artificial Intelligence Assessment of Trainee Communication Compared to Human Assessment: A Prospective Cohort Study
Overview
This study will evaluate whether a web-based artificial intelligence platform (AI) (Clinical Mind AI \[CMAI\] Stanford, CA), can assess communication skills in anesthesiology trainees, including residents and fellows, in the setting of disclosing medical errors. All participants will participate in an AI-generated simulation by using the platform remotely, and CMAI will assess trainee performance immediately after the simulation.
Detailed description
The goal of this study is to evaluate the accuracy of CMAI platform to assess participant performance following a voice-based AI simulation designed to help trainees practice disclosing medical errors. The platform will feature a custom clinical case, created using CMAI's patient creation tool, involving a discussion with the parent of a child who suffered a dental injury that occurred during intubation. The platform will provide an audio-based, simulated encounter with the parent enabling the participant to interact with, and then assess trainee communication performance, followed by the delivery of questionnaires to the trainee to determine usability and satisfaction. A human evaluator will also assess the trainees' performance using the same scales, and the investigators will compare the AI performance evaluation to the human evaluation. This study will allow the investigators to determine:
1. Reliability of the performance assessments of the CMAI platform compared to human raters 2. Usability for the CMAI audio voice model for simulated patient encounters 3. Satisfaction related to an innovative, educational technique
By evaluating these domains, we aim to determine the educational value of using simulated voice communication for training in emotionally complex, clinical scenarios.
Interventions
- Other Communication Assessment through AI and Human
Participants will remotely engage in an AI-generated simulation that presents an voice-based conversation with a simulated parent of a pediatric patient. The scenario involves discussing a clinical scenario concerning their child. Following the simulation, the participant's communication skills will be assessed using an artificial intelligence (AI) system trained to evaluate the key aspects of communication using standardized rating scales. In addition, two human evaluators will assess the parti
Primary outcome measures
- Reliability of AI Conversational Performance Assessment [Time frame: Immediately after the simulation]
Secondary outcome measures (2)
- Usability of CMAI Simulation Platform [Time frame: immediatley after the simulation]
- Satisfaction of CMAI Simulation [Time frame: immediately after the simulation]
Eligibility criteria
Inclusion criteria
- Participant's must be 18 years of age or older
- Graduate medical trainess such as residents and fellows at all levels of training excluding PGY1 trainees at Department of Anesthesiology, Perioperative, and Pain Medicine at Stanford University
- Able to speak and understand English
- Willing and able to provide consent to participate in research
- Able to participate in Artificial Intelligence communication simulation
Exclusion criteria
- Non-English speaker
- unable to provide consent to participate in research
- individual with no access to the necessary technology (internet, computer/smart device) required for participating in the digital, voice-based interaction with the AI platform
Criteria are shown verbatim from the registry (in English). Final eligibility is always assessed by the study center.
Healthy volunteers: Yes
Study design
- Allocation
- N/A
- Model
- Single group
- Masking
- Open label
- Primary purpose
- Supportive care
Study locations
United States · 1 center
- Lucile Packard Children's Hospital Stanford — Palo Alto
Identifiers
NCT: NCT07107880 · 81684