K-ORCA: Testing a Decision Support Tool and Group Process for Selecting Interventions

Official title

Refining and Pilot Testing a Decision Support Intervention to Facilitate Adoption of Evidence-Based Programs to Improve Parent and Child Mental Health

Overview

This proposal responds to NIMH Objective 4.2.c to develop "decision-support tools and technologies that increase the effectiveness and continuous improvement of mental health interventions" by leveraging the Family First Prevention Services Act (FFPSA) policy opportunity. First, a web-based platform to host (a) a decision-support tool and (b) automated facilitation for group decisions with the tool will be developed with state partners' feedback. Next, decision makers leading their states' FFPSA quality improvement efforts will be engaged to pilot a decision-support intervention comprised of the tool and live or automated facilitation, and to evaluate the implementation quality of evidence-based programs adopted with the decision-support intervention.

Detailed description

Over 268,000 children entered foster care in the United States during fiscal year 2019 despite the existence of Maltevidence-based programs (EBPs) that can successfully prevent child maltreatment. Evidence-informed decision-making (EIDM) can lead to the adoption of EBPs that are likely to be successfully implemented and result in the intended impacts. Despite the existence of EBPs to prevent child maltreatment, EBPs are underutilized in child welfare. Proposed study activities in this career development award respond to NIMH Objective 4.2.c by developing, refining, and piloting an implementation strategy to increase EIDM when decision makers are adopting EBPs to improve mental health and child welfare services. Activities leverage an unprecedented federal policy opportunity to prevent child maltreatment and test implementation decision support strategies in the real world: The Family First Prevention Services Act (FFPSA). FFPSA aims to prevent child maltreatment and foster care entries by providing federal funding to states for implementing EBPs that support child and parent mental health, reduce parental substance misuse, and improve parenting skills. This project engages decision makers from four states implementing FFPSA to extend the candidate's prior work developing an innovative EIDM tool to support EBP adoption: Optimizing Responses through Collaborative Assessments (ORCA). ORCA is based in multi-criteria decision analysis, which provides a structured approach to reach quality, evidence-informed group decisions. Given that group decisions with tools such as ORCA benefit from facilitation, this study will develop and test two facilitation modalities to accompany ORCA: automated (ORCA-A) and live (ORCA-L). In addition, a web-based platform to host the ORCA tool and the ORCA-A facilitation strategy will be developed with decision makers' input (Aim 1). The impact of ORCA-A and ORCA-L on quality of decision experiences, processes, and outcomes then will be tested (Aim 2). Implementation process quality for each EBP adopted with ORCA-A and ORCA-L will be monitored (Aim 3). To enhance her strong foundation in the implementation, decision, and systems sciences, the candidate, Dr. Gracelyn Cruden, will leverage these research activities and training activities to expand her skillset to include: leading implementation studies in service systems, employing a continuum of methods for engaging community members, leveraging interdisciplinary approaches for eliciting members' preferences, conducting advanced simulation model testing, and using advanced statistics for multilevel, longitudinal studies. Cruden will be mentored by an outstanding team led by Primary Mentor Dr. Lisa Saldana, who is accompanied by Mentors Dr. Jason Chapman, Dr. Lindsey Zimmerman, Dr. R. Christopher Sheldrick, Dr. Jonathan Purtle, and Consultants Dr. David Vanness, and Ms. Clare Anderson. Upon completion of these activities, Cruden will be equipped to lead an interdisciplinary research team that will support quality mental health and child welfare services by increasing decision makers' use of EIDM.

Interventions

• Behavioral Optimizing Responses with Collaborative Assessments-Automated (ORCA-A)
  Participants will receive automated facilitation in the ORCA platform. This will entail prompts for group discussion based on group model building scripts and decision hygiene approaches proposed by Kahneman, Sibony, and Sunstein (2021). Participants will have the option of using these prompts to discuss ORCA results in group decision discussions. Optimizing Responses with Collaborative Assessments (ORCA) is a technical decision support tool based in multi-criteria decision analysis. Decision m
• Behavioral Optimizing Responses with Collaborative Assessments-Live (ORCA-L)
  A facilitator will guide group decision discussions using group model building scripts and decision hygiene approaches proposed by Kahneman, Sibony, and Sunstein (2021). Facilitation will be either in-person or virtual, but occur "live" as in during real-time. Optimizing Responses with Collaborative Assessments (ORCA) is a technical decision support tool based in multi-criteria decision analysis. Decision makers rate and compare evidence-based programs or other practices with ORCA, then priorit

Primary outcome measures

• Mean changes in decision assumptions, decision goals Pre and Immediately Post-intervention [Time frame: Baseline; Immediately Post-intervention]
• Mean changes in decision experience quality pre-post intervention [Time frame: Baseline; Immediately Post-intervention]
• Mean changes in decision commitment pre-post intervention [Time frame: Baseline; Immediately Post-intervention]
• Mean changes in perceived potential community health impact pre-post intervention [Time frame: Baseline; Immediately Post-intervention; 12-month follow-up]
• Descriptive changes in ordinal intervention rankings assisted (with ORCA) compared to unassisted (no ORCA) [Time frame: Baseline; Immediately Post-intervention]
• Qualitatively described rationale for program adoption from pre-post intervention [Time frame: Baseline; Immediately Post-intervention]
• Descriptive, mean group-level changes in perceived feasibility of intervention Pre-post intervention [Time frame: Baseline; Immediately Post-intervention; 12-month post-intervention]
• Descriptive, mean group-level changes in perceived acceptability of intervention for implementation [Time frame: Baseline; Immediately Post-intervention; 12-month post-intervention]
• Descriptive, mean group-level changes in perceived appropriateness of intervention for implementation [Time frame: Baseline; Immediately Post-intervention; 12-month post-intervention]
• Descriptive differences in implementation process fidelity (activities completed) for interventions adopted with ORCA [Time frame: Immediately Post-intervention; bi-weekly through 12-month post-intervention]

Secondary outcome measures (4)

• Qualitative changes in decision reasoning pre- post-intervention [Time frame: Baseline; Immediately Post-intervention]
• Qualitative changes in decision goals pre- post-intervention [Time frame: Baseline; Immediately Post-intervention]
• Qualitative changes in decision consensus pre- post-intervention [Time frame: Baseline; Immediately Post-intervention]
• Time to adoption or re-adoption of interventions with ORCA [Time frame: Baseline to adoption of an intervention for implementation, assessed for up to 24 months]

Eligibility criteria

Inclusion criteria

• Non-institutionalized
• Adults (18 years or older).
• English-Speaking

Aim 1:

• Involved in initial state decision-making related to Family First Prevention Services Act.
• Willing to participate in two data collection occasions.

Aim 2:

• Non-institutionalized
• Adults (18 years or older). English-Speaking.
• Involved in ongoing state decision-making related to Family First Prevention Services Act.
• Willing to participate in three measurement occasions.

Aim 3:

-Willing to participate in multiple (bi-weekly up to two years) measurement occasions.

Exclusion criteria

-Not involved in or potentially influencing child welfare intervention decisions

Criteria are shown verbatim from the registry (in English). Final eligibility is always assessed by the study center.

Healthy volunteers: Yes

Study design

Allocation

Randomized

Model

Parallel assignment

Masking

Open label

Primary purpose

Health services research

Study locations

United States · 1 center

• Chestnut Health Systems — Eugene

Publications

• Alley ZM, Chapman JE, Schaper H, Saldana L. The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs. Implement Sci. 2023 Jul 21;18(1):30. doi: 10.1186/s13012-023-01285-0. PMID 37480144
• Oetzel JG, Wallerstein N, Duran B, Sanchez-Youngman S, Nguyen T, Woo K, Wang J, Schulz A, Keawe'aimoku Kaholokula J, Israel B, Alegria M. Impact of Participatory Health Research: A Test of the Community-Based Participatory Research Conceptual Model. Biomed Res Int. 2018 Apr 24;2018:7281405. doi: 10.1155/2018/7281405. eCollection 2018. PMID 29854784
• Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, Boynton MH, Halko H. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017 Aug 29;12(1):108. doi: 10.1186/s13012-017-0635-3. PMID 28851459
• Cruden G, Frerichs L, Powell BJ, Lanier P, Brown CH, Lich KH. Developing a Multi-Criteria Decision Analysis Tool to Support the Adoption of Evidence-Based Child Maltreatment Prevention Programs. Prev Sci. 2020 Nov;21(8):1059-1064. doi: 10.1007/s11121-020-01174-8. Epub 2020 Oct 11. PMID 33040271

Identifiers

NCT: NCT06375551 · 1171-0623 · K01MH128761

Primary sources (government registries)

• ClinicalTrials.gov (US NIH)

View this study on ClinicalTrials.gov ↗