A Cluster-randomized Control Trial of PAX Good Behavior Game in Swedish Elementary Schools

Design

A cluster-randomized controlled trial will be conducted randomizing approximately 28 schools (clusters) with a total of 84 teachers/classrooms into either receiving training and supervision in PAX GBG or receiving equally much training and supervision for implementing one of the control interventions.

Teachers in both arms will be implementing and using the intervention they were randomized to during the first year and then start implementing also the other intervention, while continuing using the first. When the baseline measure occurs at the end of grade 2 and the initial intervention is implemented for pupils during grade 3, the pupils in these classes will only receive one intervention since they move up to middle school where none of the methods are used. The teacher will instead start over with new pupils in grade 1 and use both methods with them. The sub-group of pupils that receives only one intervention will thus preserve the randomized controlled design also for long-term follow-ups.

Procedure

Ideally, half of the classrooms will be enrolled to start the intervention/control intervention during the spring semester and half during the autumn semester. The purpose of this is to control for the occasional effect of the point in time during the school year, and calendar year, when the intervention is introduced.

Credible control interventions requiring the same amount of training and supervision will be used, to control for unspecific effects of implementing a new method. It also makes it possible to promote the study to schools and teachers as a way to implement and compare several useful methods, although for different purposes, which helps to avoid too high expectation or placebo effects for PAX and to minimize possible disappointment and nocebo effects related to having to wait one year before starting with PAX. The school will have the possibility to choose one of the two control interventions - a math application, 'Count On Me!', or Collegial learning community. This serves two purposes; (1) the possibility to choose will further reduce the risk of nocebo or being dissatisfied if not randomized to PAX and (2) it will enable more schools to participate if one of the control interventions would be seen as impractical to implement. Since neither fo the control interventions are deemed to have the same specific effects as PAX, at least not to the same extent, they will both act as controls for unspecific effects.

A total of seven measures will be administered in relation to pupils, teachers and the classroom during the two years, with the baseline (Measurement 0) at the end of the semester before the implementation starts (Semester 0) and the primary endpoint (Measurement 4) one year afterwards. To reduce the burden for teachers and pupils, the full range of outcome measures will be conducted only on two occasions (Measurement 0 and Measurement 4) and otherwise a reduced set of measures will be used for pupils and teachers (For measures on classroom level, the full set of measures will always be applied). The reduced set of self-rating measures for teachers will include work related stress (primary outcome), classroom management, general stress, sleep problems, work ability and observed negative/positive effects of the method being implemented. For pupils, the reduced set will contain questions on well-being (primary measure). Teachers will also measure the well-being for each pupil at each measurement point.

Following time frames will apply to the measurement points:

Measurement 0 (M0) - semester before the first implementation semester, a 4-weeks administration window ending two weeks before the end of the semester.

Measurement 1 (M1) - the first implementation semester, a 6-weeks administration window starting two weeks after the start of the semester (and preferably two weeks after the completion of the intervention training).

Measurement 2 (M2) - the first implementation semester, a 4-weeks administration window ending two weeks before the end of the semester).

Measurement 3 (M3) - the second implementation semester, a 4-weeks administration window starting two weeks after the start of the semester).

Measurement 4 (M4) - primary endpoint, the second implementation semester, a 4-weeks administration window ending two weeks before the end of the semester.

Measurement 5 (M5) - the third implementation semester, a 4-weeks administration window ending two weeks before the end of the semester.

Measurement 6 (M6) - the fourth implementation semester, a 4-weeks administration window ending two weeks before the end of the semester.

Participants and recruitment

Schools, teachers, classrooms and other school staff

The cluster-randomization will be performed on the school-level, and will in turn decide which teachers and pupils that are included. The schools will be recruited via an established network of PAX GBG trainers in Sweden and the research groups other connections with school-related networks and the recruitment will be country wide. The schools will be recruited from at least two different Regions and at least four different municipalities (kommuner) to increase the generalizability of the results. In total, around 28 schools will be included, depending on the number of classrooms participating from each school.

A total of 84 classes, with an expected total of up to 100 teachers (as some classes have more than one teacher) will be included in the project.

Other school staff is recruited only to participate in the qualitative evaluation of expectations and experiences of the interventions. The selection of these will be pragmatic and theoretically lead to reaching a saturation of qualitative data, with a maximum of four extra staff from each school.

Pupils and their participation in interventions and data collection

In total, 2200 pupils will be included, with approximately 26 pupils included from each classroom. All the pupils in a class related to the included teachers/classrooms will participate in the intervention since those are all implemented in the regular operation/teaching at the school and focused on the teachers' way of teaching and managing the class and the classroom and is not possible to exclude specific pupils from. Pupils can however be included in the data collection in three different ways:

* Data from pupils to parents/custodians having signed the informed consent will have the data from their self-ratings, from the teachers ratings of them, and from registries connected to an ID-number, which in turn can be connected to their social security number (personnummer). * Data on pupils whose parents have not filled out the consent will still be collected, but without any ID-number or other identifier, which makes it impossible to connect it to the specific pupil and hence make the data anonymous. Only the interventions the pupil has been randomized to, and at which time-point the data was collected, is connected to this data. * Parents/custodians and the pupil himself/herself will be informed that the pupils do not need to fill out the self-ratings if they inform the teacher or researchers about their wish. Pupils will be informed of this before each self-rating.

Randomization

To ensure that the socioeconomic status (SES) of the schools do not differ too much, each school will be classified as being low, middle, or high SES and then, when there are enough schools in a cohort, randomized consecutively according to one of three corresponding block-randomized lists with block sizes varying randomly between 2, 4 and 6. The randomization will be handled by an independent third party, the Karolinska Trial Alliance, to ensure the researchers are not able to assess the lists or otherwise predict randomization results.

Interventions

The implementation procedure will be similar for each intervention, with smaller adjustments to fit specific needs of each method. Each starts with brief initial written information about the interventions, followed by one to two days of training/education for teachers. This is then followed by regular supervision face-to-face or via video/telephone depending on practical aspects and possible restrictions due to the Covid-19 pandemic. Supervision always includes some form of observation and monitoring of the use of the method by the supervisor, as a source to further inform the supervision. A total of six supervision sessions are planned: 4 sessions during the first implementation semester (held during the 1st to 4rth quarter of the semester respectively, starting with 2 individual sessions, followed by 1 group session and then by 1 individual session) and 2 sessions during the second implementation semester (held during the 1st and 2nd half of the semester respectively, 1 group session followed by 1 individual session). Teachers in all groups will also be able to receive briefer supervision on demand or additional supervision sessions initiated by the supervisor if deemed necessary.

PAX Good Behavior Game

PAX Good Behavior Game (GBG) is based on its precursor programme Good Behavior Game (GBG) that trains teachers to use principles of social learning in order to maximize the childrens' task-oriented and prosocial behaviors and minimize the occurrence of disruptive and off-task behaviors. PAX GBG is packaged into ten specific tools that the teachers are trained to introduce in their classroom management practice. The program introduces verbal and visual cues that facilitate prosocial and attentive behaviors One of the core elements of the program is the PAX-game, meaning that the children - while focusing on ordinary school tasks like math - get to play a game in teams where each team tries to make as few classroom rules infractions as possible during a certain period of time. The teams that succeed in achieving predetermined goals get a reward (often a fun and desirable activity) that serves as a reinforcer of prosocial and task-oriented behaviors.

Supervision is performed individually with each teacher (4 sessions) and in group (2 sessions) and includes that the PAX-trainer observes a lecture given by the teacher before each supervision session.

Count On Me! (Control condition)

Count On Me! is an adaptive application teaching fluency in addition and subtraction facts (i.e., math facts), equality, compose and decompose numbers and a few more basic mathematical skills taught in the first grades of schooling. Count on me! is set in adventure game where the child has an avatar that travels thru different worlds/platforms by using their mathematical skills. The child starts of in the home of the avatar, in a house like environment, where the child discovers a magical map that brings the child into a different world called Numberia. During the adventures in Numberia the child learns that it has certain skills, called the mathemagical powers, that will be used to liberate Numberia from the evil Prince Claw that has captured all the beautiful things (e.g., the birds that used to sing in the forest).

Count on me! was developed within a behavior analytic framework using a nonlinear programming process originally developed by Tiemann and Markle and later slightly revised by other researchers while creating Headsprout Early Reading and Headsprout Reading Comprehension. Instructional design elements from Tiemann and Markle, explicit instruction and the Morningside model of generative instruction were used to build the exercises. The math exercises contain at least one of three recurrent phases used in all training: the modeling, guide, and fluency phase. Modeling of new skills involves animations and hand-drawn examples along with audial instructions and requires frequent untimed student responding. Guided practice of newly modeled skills helps students achieve high levels of accuracy without timing. Finally, fluency building is aimed at developing proficiency with the skill at hand using timing.

• [Pupil-focused] Child Health Utilities - 9 dimensions (CHU9D) [Срок оценки: At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.]
• [Pupil-focused] KIDSCREEN-10 [Срок оценки: At baseline, then after 6, 12 and 24 months. See Detailed Description section for the details of the measurement points.]
• [Pupil-focused, administered by the teacher] Strengths and Difficulties Questionnaire (SDQ) [the 5-items dysregulation profile] [Срок оценки: Up to 24 months. See Detailed Description section for the details of the measurement points.]
• [Pupil-focused, administered by the teacher] Strengths and Difficulties Questionnaire (SDQ) [the 15-items dysregulation profile] [Срок оценки: At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.]
• [Pupil-focused] School absence [Срок оценки: A total measure for comparison between the randomization groups 12 months after baseline, and for comparison to a matched control group 24 months after baseline.]
• [Pupil-focused] The parents' need to stay home from work in order to take care of a sick child [Срок оценки: A total measure for comparison between the randomization groups 12 months after baseline, and for comparison to a matched control group 24 months after baseline.]
• [Pupil-focused] Number of primary health contacts [Срок оценки: A total measure for comparison between the randomization groups 12 months after baseline, and for comparison to a matched control group 24 months after baseline.]
• [Pupil-focused] Neurodevelopmental diagnoses [Срок оценки: A total measure for comparison between the randomization groups 12 months after baseline, and for comparison to a matched control group 24 months after baseline.]
• [Pupil-focused] Heidelberger-Rechen Test 4 dimensions (HRT-4) [Срок оценки: At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.]
• [Pupil-focused] LegiLexi [Срок оценки: At baseline, then after 12 and 24 months. See Detailed Description section for the details of the measurement points.]
• [Teacher-focused] Teacher Stress Inventory (TSI) (a 12 items subscale). [Срок оценки: Up to 24 months. See Detailed Description section for the details of the measurement points.]
• [Teacher-focused] The 7-item Stress subscale from the Depression, Anxiety ans Stress Scale (DASS) [Срок оценки: Up to 24 months. See Detailed Description section for the details of the measurement points.]