COMPASS Year 3 and 4 School Recruitment and Retention

COMPASS year 3 and 4 school recruitment and retention (pdf)

Table of contents

Acknowledgements
Introduction
COMPASS Sample in Year 3 (2014-2015)
COMPASS Sample in Year 4 (2015-2016)
Discussion
References

Acknowledgements

Authors

Rachel Reel, BSc (School of Public Health and Health Systems, University of Waterloo, Waterloo, ON),
Chad Bredin, BA (School of Public Health and Health Systems, University of Waterloo, Waterloo, ON)
Katelyn Battista, MMath (School of Public Health and Health Systems, University of Waterloo, Waterloo, ON)
Scott T. Leatherdale, PhD (School of Public Health and Health Systems, University of Waterloo, Waterloo, ON)

Report funded by

The COMPASS study was supported by a bridge grant from the Canadian Institutes of Health Research (CIHR) Institute of Nutrition, Metabolism and Diabetes (INMD) through the “Obesity – Interventions to Prevent or Treat” priority funding awards (OOP-110788; grant awarded to S. Leatherdale) and an operating grant from the Canadian Institutes of Health Research (CIHR) Institute of Population and Public Health (IPPH) (MOP-114875; grant awarded to S. Leatherdale).

Suggested citation

Reel R, Bredin C, Battista K, Leatherdale ST. COMPASS Year 3 and 4 School Recruitment and Retention. COMPASS Technical Report Series. (2017); volume 4 (1): Waterloo, Ontario: University of Waterloo. 

Contact

COMPASS research team
University of Waterloo
200 University Ave West
Waterloo, ON
Canada
N2L 3G1
uwaterloo.ca/compass-system

Introduction

COMPASS is a longitudinal study (started in 2012-2013) designed to follow a cohort of grade 9 to 12 students attending a convenience sample of secondary schools for four years to understand how changes in school environment characteristics (policies, programs, built environment) are associated with changes in youth health behaviours [1]. COMPASS originated to provide school stakeholders with the evidence to guide and evaluate school-based interventions related to obesity, healthy eating, tobacco use, alcohol and marijuana use, physical activity, sedentary behaviour, school connectedness, bullying, and academic achievement. COMPASS was designed to facilitate multiple large-scale school-based data collections and uses in-class whole-school sampling data collection methods consistent with previous research [2-5]. COMPASS also facilitates knowledge transfer and exchange by annually providing each participating school with a school-specific feedback report that highlights the school-specific prevalence for each outcome, comparisons to provincial and national norms or guidelines, and provides evidence- based suggestions for school-based interventions (programs and/or policies) designed to address the outcomes covered in the feedback report (refer to: www.compass.uwaterloo.ca)
As previously reported [6] there were 43 participating schools from across Ontario (ON) that participated during Year 1 (Y1), 2012-2013, of COMPASS. Since the target number of 90 schools was not reached due to a number of factors (including labour strife and competing research), additional recruitment continued in Year 2 (Y2), 2013-2014, which culminated in the addition of 46 new participating schools. Of this supplemental sample, there were 36 schools included from Ontario, and 10 schools included from Alberta (AB) [7,8]. The addition of Alberta schools allowed the COMPASS researchers to reach their target numbers and provide the additional value of having a provincial comparison to the Ontario schools. With the remaining two years of the study ahead, there was a total of 89 participating schools from which to collect longitudinal data; 79 schools in Ontario, and 10 schools in Alberta.
This technical report will provide details on school recruitment and retention during Year 3 (Y3), 2014-2015, and Year 4 (Y4), 2015-2016, of the COMPASS study. No additional school boards/divisions were recruited nor did any withdraw during this time. Figure 1 shows the school sample size over the first 4 years of COMPASS overall and by province, including annual drop out and replenishment totals.

Figure 1: COMPASS school sample for Years 1 to 4 (2012/13 to 2015/16)

COMPASS Sample in Year 3 (2014-2015)

As shown in Figure 1, 87 schools participated in COMPASS during Y3. Of the participating schools, 41 had participated in two previous data collections (Y1 schools), 45 had participated in one previous data collection (Y2 schools), and 1 participated for the first time (Y3 school). Of this sample, there were 78 Ontario schools and 9 Alberta schools. The sample changed slightly in Y3 as a result of some recruitment and attrition in the sample, which will be explored below.

Recruitment in Y3

All 89 Y2 schools were approached via phone or email in September 2014 in order to select a COMPASS survey date for the upcoming school year (Y3). There was 1 additional school in Ontario that was added to the sample. As a result of administrative changes, this school had a new principal; one who had previously worked at a COMPASS school and who asked for COMPASS to be implemented at the new school. No formal recruitment was conducted, but this Ontario school was added to the sample.

Withdrawals in Y3

Over the course of Y3, there were three schools that withdrew from the study. Among these schools, there were two that had participated twice before (Y1 schools, both from ON), and one that had participated once before (Y2 school, from AB). We provided the schools with an opportunity to share their reason(s) for declining to participate in Year 3 of COMPASS: One school indicated they had too many research obligations, one indicated they could not spare the class time, and one had a principal leave and the remaining administrators were feeling too overwhelmed with having to cover the vacancy themselves to be able to manage COMPASS that year. Similar reasons were expressed by schools who declined participation during baseline sampling as well [5], although the issues were much less widespread during Y3.

COMPASS Sample in Year 4 (2015-2016)

As shown in Figure 1, 81 schools participated during Y4 of COMPASS. Of the participating schools, 38 had participated in three previous data collections (Y1 schools), 41 had participated in two previous data collections (Y2 schools), one had participated in one previous data collection (Y3 school), and one participated for the first time (Y4 school). Of the sample, 72 schools were in Ontario and nine schools were in Alberta. The sample had slight changes in Year 4 due to additional recruitment and attrition in the sample, which are explained below.

Recruitment in Y4

All 87 Y3 schools were contacted via email and phone at the beginning of the school year in September 2015. In addition to the 87 returning schools, there was one additional Ontario school added to the COMPASS sample during Y4. Similar to the school added during Y3, administrative changes among schools resulted in a previous COMPASS principal switching schools and asking to have COMPASS implemented at the new school.

Withdrawals in Y4

During Y4, seven schools withdrew from COMPASS, all of which were located in Ontario. Among these schools, three of them had participated in COMPASS for three consecutive data collections (Y1 schools), and four had participated for two consecutive data collections (Y2 schools). As was done previously, we provided these schools with the opportunity to indicate why they were withdrawing from COMPASS. Among the reasons provided, all of the Y1 schools and the majority of the Y2 schools withdrew as a result of labour issues between their respective school boards and their teachers’ union locals. In addition, one of the schools refused to participate due to the principal falling ill. Again, we see here that Ontario labour issues was a major reason for schools declining participation in COMPASS.

Discussion

As described elsewhere [7,8], the COMPASS school sample increased substantially between Y1 to Y2, with the addition of 46 new schools. For the remainder of the COMPASS study (Y2 onwards) there was relatively little change, with only minor additions and losses over the 3-year period of data collections. In the original CIHR funding proposal, we had planned for a 10% attrition rate each year at the school level. Since formal recruitment for Phase 1 of COMPASS did not stop until Y3, we will look at Y1 and Y2 schools separately. By following this pattern, the Y1 schools were expected to have an overall attrition rate of 27.1% over the 4 years and the Y2 schools were expected to have an overall attrition rate of 19% over the 3 years. We can now see that we exceeded our expectations, and instead a much lower attrition rate persisted. Of the initial 43 Y1 schools, 38 were retained. This results in 11.6% overall attrition rate, compared to the expected 27.1%. From the 46 Y2 schools, 41 were retained. This results in 10.9% overall attrition rate, compared to the expected 19%. To demonstrate a year-to-year breakdown, Y2 had an attrition rate of 0%, all schools were retained from Y1 to Y2. During Y3 three schools were lost, resulting in a 3.37% attrition rate. During Y4 7 schools were lost, resulting in a 8.1% attrition rate. Each individual year demonstrates that the anticipated 10% attrition was not met and instead resulted in a lower than expected rate. This low rate of attrition illustrates the value placed on COMPASS—as a low-burden and high-value research and prevention tool—at the school-level by administrators. The most common factor for schools declining participation during Y3 and Y4 was labour issues at the board level. This factor has been a persistent issue with school recruitment in Ontario among the COMPASS sample which dates back to the baseline sample in 2012-2013 [5]. Without the ongoing cyclical spectre of labour unrest facing Ontario schools, it is safe to surmise that the COMPASS sample would have been larger at the outset and would have had an even lower attrition rate over time. Still, being able to build and maintain such a robust sample over 4 years is a testament to the quality and value of COMPASS to schools.

Although we have lost a small number of schools since the baseline sample, we have also been able to include additional schools without formal recruitment. There were two schools over the course of Year 3 and 4 added to the COMPASS convenience sample as a result of administrative changes. Principals from schools participating in COMPASS saw the value of the study and asked for COMPASS to be implemented in their new schools.

In conclusion, we feel that COMPASS has exceeded our school sample size expectations over the first four years of the study (Phase 1). We believe this was accomplished by adding substantial value to schools at a low burden, both in terms of their time and resources. As we move forward via the successful five-year renewal of COMPASS supported by CIHR (Phase 2, 2016-2017 to 2020-2021), we will continue to closely monitor our school sample and perform replenishment as required to maintain an appropriate school-level sample size required for our analytical modelling needs.

References

1. Leatherdale, S.T., Brown, K.S., Carson, V., Childs, R.A., Dubin, J.A., Elliott, S.J., Faulkner, G., Hammond, D., Manske, S., Sabiston, C.M., Laxer, R.E., Bredin, C. & Thompson-Haile, A. The COMPASS study: a longitudinal hierarchical research platform for evaluating natural experiments related to changes in school-level programs, policies and built environment resources. BMC Public Health 2014, 14:331.
2. Leatherdale ST, Burkhalter R: The substance use profile of Canadian youth: exploring the prevalence of alcohol, drug and tobacco use by gender and grade. Addict Behav 2012, 37:318-322.
3. Leatherdale ST, Manske S, Faulkner G, Arbour K, Bredin C: A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act 2010, 25;6. doi: 10.1186/1479-5868-7-6.
4. Leatherdale ST, McDonald PW, Cameron R, Brown KS: A multi-level analysis examining the relationship between social influences for smoking and smoking onset. Am J Health Behav 2005, 29:520-530.
5. Leatherdale ST, Papadakis S: A multi-level examination of the association between older social models in the school environment and overweight and obesity among younger students. J Youth Adolesc 2011, 40:361-372.
6. Thompson-Haile A, Leatherdale ST. Baseline Sampling and Recruitment Results. COMPASS Technical Report Series. 2013;1(4). Waterloo, Ontario: University of Waterloo. Available at: www.compass.uwaterloo.ca
7. Bredin, C, Thompson-Haile A, Leatherdale ST. Supplemental Sampling and Recruitment of Additional Schools in Ontario: Technical Report Series. 2015; 3(2). Waterloo, Ontario: University of Waterloo. Available at: www.compass.uwaterloo.ca
8. Wagner M, Bredin C, Thompson-Haile, A & Leatherdale ST. Alberta Baseline Sampling and Recruitment Results. COMPASS Technical Report Series. 2015; 3(1).Waterloo, Ontario: University of Waterloo. Available at: www.compass.uwaterloo.ca