Development of the COMPASS student questionnaire

Development of the COMPASS student questionnaire (PDF)

COMPASS technical report series, volume 2, issue 2, May 2014

Acknowledgements
Introduction
Methods
Identifying content measures for the questionnaire
Primary considerations for selecting measure for the questionnaire
Identifying core measures for the questionnaire
Identifying supplementary measure for the questionnaire
Identifying demographic measures for the questionnaire
Discussion
References

Acknowledgements

Authors

Chad Bredin, BA (Propel Centre for Population Health Impact, University of Waterloo, Waterloo, ON)
Scott T. Leatherdale, PhD (School of Public Health and Health Systems, University of Waterloo, Waterloo, ON)

Report funded by

The COMPASS study was supported by a bridge grant from the Canadian Institutes of Health Research
(CIHR) Institute of Nutrition, Metabolism and Diabetes (INMD) through the “Obesity–Interventions to
Prevent or Treat” priority funding awards (OOP-110788; grant awarded to S. Leatherdale) and an
operating grant from the Canadian Institutes of Health Research (CIHR) Institute of Population and Public
Health (IPPH) (MOP-114875; grant awarded to S. Leatherdale).

Suggested citation

Bredin C, Leatherdale ST. Development of the COMPASS Student Questionnaire.COMPASS Technical
Report Series. 2014;2(2). Waterloo, Ontario: University of Waterloo. Available at:
www.compass.uwaterloo.ca.

Contact

COMPASS research team, University of Waterloo, 200 University Ave West, BMH 1038 Waterloo, ON Canada N2L 3G1, www.compass@uwaterloo.ca.

Introduction

COMPASS is a longitudinal study (starting in 2012-13) designed to follow a cohort of grade 9 to 12
students attending a convenience sample of Ontario secondary schools for four years to understand how
changes in school environment characteristics (policies, programs, built environment) are associated
with changes in youth health behaviours [1]. COMPASS originated to provide school stakeholders with
the evidence to guide and evaluate school-based interventions related to obesity, healthy eating,
tobacco use, alcohol and marijuana use, physical activity, sedentary behaviour, school connectedness,
bullying, and academic achievement. COMPASS has been designed to facilitate multiple large-scale
school-based data collections and uses in-class whole-school sampling data collection methods
consistent with previous research [2-5]. COMPASS also facilitates knowledge transfer and exchange by
annually providing each participating school with a school-specific feedback report that highlights the
school-specific prevalence for each outcome, comparisons to provincial and national norms or
guidelines, and provides evidence-based suggestions for school-based interventions (programs and/or
policies) designed to address the outcomes covered in the feedback report (refer to:www.compass.uwaterloo.ca).

The student-level data in the COMPASS study are collected using the COMPASS questionnaire (Cq) (refer
to appendix A). The Cq is a 12-page machine-readable paper booklet that is completed by participating
students in the classroom setting. Survey items were specifically chosen to reflect both science-based
(e.g., obesity) and practice-based (e.g., bullying) concerns. Because the Cq collects data from large
whole-school samples during class time, it was purposefully made short (12-pages allows it to be completed in one 30-40 minute class period), and inexpensive (machine-readable forms). This technical report provides details on the development and selection of the measures used in the Cq.

Methods

Identifying content areas for the questionnaire

In order to address the primary research objectives for COMPASS (as outlined in the Canadian Institutes
of Health Research [CIHR] funding proposal), the Cq needed to include items related to the primary study outcomes. This included both core measures (obesity, marijuana-use, physical activity, alcohol use,
smoking (tobacco-use), and sedentary behaviour), and demographic measures (gender, age, ethnicity, and income). Based on the previous experience of research team members, we also wanted to ensure that the data collected in COMPASS were relevant to addressing some additional priority domains previously identified by school stakeholders that were not included as core measures in the CIHR funding proposal. The purpose of including some supplementary measures (eating behaviour, bullying, education outcomes, school connectedness, active commuting, and consumption of energy drinks), was to increase schools interest and engagement in the study, to add value to schools in terms of the breadth of data available, and to foster additional prevention actions within schools.

Primary considerations for selecting measure for the questionnaire

There were four primary considerations the COMPASS team used when determining which measures
would be used in the Cq:

Since the Cq could only collect self-report data given the active-information passive-consent permission protocols, the COMPASS team wanted to use self-report core outcome measures that were considered reliable and valid for use in youth populations of this age range;
To minimize the burden on schools and students and ensure survey completion in one class period (about 30-40 minutes - meaning no more than 12 pages in length), our team was challenged with having to select items that balanced both the depth of the data required for measuring each core outcome and the breadth of data that could be measured for each core outcome within this brief tool;
Given that COMPASS schools were purposefully sampled, the COMPASS team wanted the core measures in the Cq to be consistent with the measures used in other self-reported school-based surveys of youth in Ontario and Canada [e.g., the Ontario Student Drug Use and Health Survey (OSDUHS), the National Youth Smoking Survey (YSS), and the Health Behavior of School Children Survey (HBSC)] in order to make cross-study comparisons to existing population-based surveys; and,
As a means for helping school stakeholders and researchers to place COMPASS data and results into an appropriate context for action, the COMPASS team wanted the core outcome measures to provide data that would also be compatible with benchmarking against existing national (or international) public health guidelines or recommendations for youth populations (refer to appendix B for a sample of the benchmarking).

Identifying core measures for the questionnaire

Based on these four primary considerations, the COMPASS team identified the following core measures for obesity, marijuana-use, physical activity, alcohol-use, smoking (tobacco-use), and sedentary behaviour for use in the Cq.

Obesity (weight status)

The self-reported height and weight items in the Cq were designed to be consistent to those used in the Youth Risk Behavior Survey (YRBS) [6] and the YSS [7], although we adapted those measures to also provided respondents with both metric and imperial response options and a blank line prefaced by ‘‘My weight is ____ pounds/kilograms’’ and ‘‘My height is ____ inches/centimetres’’. The data derived from the Cq height and weight measures provide the data necessary to allow researchers to calculate the Body Mass Index (BMI) for each participant using any of the various BMI classification systems that are available (e.g., the International Obesity Task Force BMI classification [8] or the World Health Organization Child Growth Standards [9]). The Cq items for height and weight used to derive BMI scores have demonstrated reliability and validity [10]. Two additionally relevant weight status measures in the Cq include an item pertaining to weight status perceptions that is consistent with previous research [11] and a new measure asking students if they were actively trying to change their weight status (i.e., lose weight, gain weight, stay the same weight).

Marijuana use

The self-reported marijuana use items in the Cq (age of first use, frequency of use) are consistent to those used in the YSS [7]. The psychometrics properties of the marijuana use measures derived from the YSS are not available from Health Canada. The Cq also includes a new item asking students to report how difficult or easy they thought it would be to get marijuana if they wanted some. This question is derived from a similar tobacco-related question that has appeared in the YSS [7].

Physical activity

We used a modified version of the previously validated brief physical activity measures used in the School Health Action Planning and Evaluation System (SHAPES) [12-13]. Since the SHAPES physical activity measures did not measure vigorous physical activity (VPA) or moderate to vigorous physical activity (MVPA) accurately [12], the Cq measure was reworded to provide respondents with definitions and examples of ‘hard’ and ‘moderate’ PA (e.g., instructions not to include time in hard physical activity when calculating moderate physical activity (MPA)). Testing identified that the new Cq measures for physical activity produce reliable and valid VPA, MPA and MVPA estimates [14]. The Cq measures also provide the data required to determine if students are meeting the Canadian physical activity guidelines as defined by the Canadian Society for Exercise Physiology (CSEP) [15]. Additional measures in the Cq relevant to physical activity include physical activity of friends, participation in physical education classes, participation in varsity sports at schools, participation in intramurals or sports clubs at school, participation in competitive sports outside of school, active commuting, and strength training.

Alcohol use

The self-reported alcohol use items in the Cq (age of first use, frequency of use, binge drinking), are
consistent to those used in the YSS [7]. The psychometrics properties of the alcohol use measures derived from the YSS are not available from Health Canada. The Cq also includes a new item asking students to report consumption of alcohol mixed with an energy drinks.

Smoking (tobacco use)

The self-reported tobacco use items in the Cq (smoking susceptibility, age of first use, smoking status), are consistent to those used in the YSS [7]. Validity testing has identified that the self-reported measures used in the YSS produce accurate estimates of cigarette smoking among Canadian youth [16]. Additional measures in the Cq relevant to tobacco use include smoking susceptibility, peer smoking, age of onset,
cigarettes per day, quit attempts, and use of alternative tobacco products (hookah, e-cigarettes,
smokeless tobacco, roll-your-own cigarettes, cigars, cigarillos, blunt wraps, tobacco blended with marijuana, and pipe tobacco) and nicotine replacement therapy (NRT).

Sedentary behaviour

To assess sedentary behaviour, COMPASS made use of a modified version of the sedentary behaviour measures previously used in SHAPES [17-18]; the SHAPES sedentary behaviour measures have never been validated and the Cq measure adapted the wording to include examples of how to complete the question, new categories of sedentary behaviour (e.g., streaming TV shows or movies), and new response categories thatallow students to respond in 15 minute increments. Testing identified that the new Cq measures for sedentary behaviour produce reliable and valid average total sedentary behaviour (ATSB) estimates [14]. The Cq measures also provide the data required to determine if students are meeting the Canadian sedentary behaviour guidelines as defined by the Canadian Society for Exercise Physiology (CSEP) [19]. Additional measures in the Cq relevant to sedentary behaviour include time spent sleeping.

Identifying supplementary measures for the questionnaire

Using the same four primary considerations, and an additional consideration of making sure that
participating schools would be interested and engaged in the study by collecting some practice-based data, the COMPASS team identified the following supplementary measures for eating behaviour, bullying, academic achievement, school connectedness, active commuting, and consumption of energy drinks for use in the Cq.

Eating behaviour

While there are existing tools for measuring diet within youth population based on 7-day food recall or food diaries, these types of long, intensive and time consuming measures were not appropriate for use within the COMPASS protocol. Moreover, we wanted to be able to provide schools with meaningful and actionable data associated with the national benchmark for eating habits, namely the Canada’s Food Guide [20]. Considering no existing measures were available for youth populations pertaining to measuring eating behaviour specific to Canada’s Food Guide, we worked in partnership with Health Canada do develop a simple self-reported measure for use in COMPASS. Testing identified that the new Cq measures of eating behaviour specific to the four components of Canada’s Food Guide (vegetables and fruit, grain products, milk and alternatives, meat and alternatives) produce reliable and valid estimates [10]. Additional measures in the Cq relevant to eating behaviour include eating breakfast, eating in a school food program, buying lunch at restaurants and/or cafeteria, using vending machines at school, snacking at school, consumption of sugar sweetened beverages, consumption of energy drinks, and coffee consumption.

Bullying

A key supplementary measure of interest to schools was bullying. As such, to assess bullying, COMPASS
made use of a modified version of the bullying measures previously used in the Ontario Student Drug Use and Health Survey (OSDUHS) [21]; the OSDUHS bullying behaviour measures did not differentiate the types of bullying students experience or perpetrate so the Cq measures adapted the questions to include being a victim of or perpetrator of physical bullying, verbal bullying, and cyber bullying.

Education outcomes

Given the importance of academic performance for youth’s future opportunities, and the dearth of studies of examining these relationships among Canadian youth, it is deemed critical to include measures of academic performance. The Cq measures of educational performance include questions about academic performance (overall marks, most recent math and English courses taken, and marks in the math and English courses); academic engagement (tardiness or absence, attending class without completed homework), educational aspirations (the level of schooling they hope to complete), and educational expectations (the level of schooling they expect to complete).

School connectedness

School connectedness is the belief held by students that adults and peers in their school care about their learning as well as about them as individuals. The self-reported school connectedness items in the Cq, are consistent to those used in the YSS [7].

Identifying demographic measures for the questionnaire

In order to make cross-study comparisons to other relevant research in this domain, the COMPASS team wanted the demographic measures in the Cq (gender, age, grade, ethnicity, weekly spending money) to be consistent with the measures used in other self-reported school-based studies of secondary school aged youth in Ontario (and Canada).

Discussion

The goal for the COMPASS team developing the student questionnaire was to ensure that it included questions that a) were comparable to other youth health studies for consistency and reporting purposes; b) would produce results that could be directly compared to national benchmarks; c) could answer a wide variety of research questions; and d) would engage schools and other stakeholders to encourage action. We believe that the Cq provides robust and valid self-reported data designed to provide school stakeholders with local and timely knowledge and understanding of how to best intervene with their student population and create valuable linkages between schools and the local research and practice community, fostering future research and prevention collaborations.

References

Leatherdale ST, et al: The COMPASS study: a longitudinal hierarchical research platform fo evaluating natural experiments related to changes in school-level programs, policies and built environment resources. BMC Public Health. 2014, 14:331. doi:10.1186/1471-2458-14-331.
Leatherdale ST, Burkhalter R: The substance use profile of Canadian youth: exploring the prevalence of alcohol, drug and tobacco use by gender and grade. Addict Behav 2012, 37:318-322.
Leatherdale ST, Manske S, Faulkner G, Arbour K, Bredin C: A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act 2010, 25;6. doi: 10.1186/1479-5868-7-6.
Leatherdale ST, McDonald PW, Cameron R, Brown KS: A multi-level analysis examining therelationship between social influences for smoking and smoking onset. Am J Health Behav 2005,29:520-530.
Leatherdale ST, Papadakis S: A multi-level examination of the association between older social models in the school environment and overweight and obesity among younger students. J Youth Adolesc 2011, 40:361-372.
Brener N, Mcmanus T, Galuska D, Lowry R, Wechsler H: Reliability and validity of self-reported height and weight among high school students. J Adolescent Health 2003, 32:281-287.
Elton-Marshall T, Leatherdale ST, Manske SR, Wong K, Ahmed R, Burkhalter R: Research Methods of the Youth Smoking Survey (YSS). Chronic Dis Inj Can 2011, 32:47-54.
Cole, TJ, Bellizzi, MC, Flegal, KM, Dietz, WH: Establishing a standard definition for child overweight and obesity worldwide: International survey. BMJ 2000, 320(7244):1240-1243.
WHO Multicentre Growth Reference Study Group. WHO Child Growth Standards: Length/height for-age, weight-for-age, weight-for-length, weight-for-height and body mass index-for-age: Methods and development. Geneva: World Health Organization, 2006.
Leatherdale ST, Laxer RE: Reliability and validity of the weight status and dietary intake measuresin the COMPASS questionnaire: are the self-reported measures of body mass index (BMI) and Canada's Food Guide servings robust? Int J Behav Nutr Phys Act 2013, 10:42. doi: 10.1186/1479-5868-10-42.
Leatherdale ST, Wong S, Manske S, Colditz GA: Susceptibility to smoking and its association with physical activity, BMI and weight concerns among non-smoking youth. Nic Tob Res 2008, 10,499-505.
Wong S, Leatherdale ST, Manske S: Reliability and validity of a school-based physical activity questionnaire. Med Sci Sports Exer 2006, 38:1593-1600.
Leatherdale ST, Manske S, Wong S, Cameron R: Integrating research, policy and practice in school-based physical activity prevention programming: The School Health Action, Planning and Evaluation System (SHAPES) Physical Activity Module. Health Promotion Practice 2009, 10:254-261.
Leatherdale ST, Laxer RE, Faulkner G: Reliability and validity of the physical activity and sedentary behaviour measures in the COMPASS study. COMPASS Technical Report Series. 2014,2(1). Waterloo, Ontario: University of Waterloo. Available at: www.compass.uwaterloo.ca.
Canadian Society for Exercise Physiology: Canadian Physical Activity Guidelines for Youth – 12 to 17 years, 2011. http://www.csep.ca/CMFiles/Guidelines/CSEP-InfoSheets-youth-ENG.pdf [Accessed September 21, 2011].
Wong S, Shields M, Leatherdale ST, Malaison E, Hammond D: Assessment of the validity of selfreported smoking status among Canadians. Health Rep 2012, 23:47-53.7.
Leatherdale ST, Ahmed R: Screen-based sedentary behaviours among a nationally representative sample of youth: are Canadian kids couch potatoes? Chronic Dis Inj Can 2011, 31:141-146.
Wong SL, Leatherdale ST: Association between sedentary behavior, physical activity, and obesity: inactivity among active kids. Prev Chronic Dis 2009 6:1.
Canadian Society for Exercise Physiology: Canadian Sedentary Behaviour Guidelines for Youth –12 to 17 years, 2011. http://www.csep.ca/CMFiles/Guidelines/CSEP-InfoSheets-ENGTeen%20FINAL.pdf [Accessed September 21, 2011].
Health Canada: Eating Well with Canada’s Food Guide. Minister of Health; 2011. http://www.hcsc.gc.ca/fn-an/alt_formats/hpfb-dgpsa/pdf/food-guide-aliment/print_eatwell_bienmangeng.pdf [Accessed June 9, 2012].
Centre for Addiction and Mental Health (CAMH): Ontario Student Drug Use and Health Survey (OSDUHS). http://www.camh.ca/en/research/news_and_publications/ontario-student-druguse-and-health-survey/Documents/2013%20OSDUHS%20Docs/FormASS_2013OSDUHS_Final.pdf [accessed April 02, 2012].