Student-level data collection procedures (PDF)
COMPASS technical report series, volume 1, issue 5, July 2013
Table of contents
Acknowledgements
Introduction
Online survey implementation systems (OSIS)
Data collection start-up
Scheduling a data collection date
Class list enrolment information
School welcome folder
Welcome letter
School contact checklist
Teacher letters and classroom posters
Class information template
Permission materials
Prepare questionnaire bundles
Teacher instruction questionnaire letter
Classroom instructions
Classroom frequently asked questions (FAQ)
No permission list
Questionnaires and envelopes
Immediately before the school data collection
Confirmation call
Accessory materials for the data collection date
Procedures on the day of the data collection day at the school
Questionnaire administration
References
Acknowledgments
Authors
Audra Thompson-Haile, MA (Propel Centre for Population Health Impact, University of Waterloo, Waterloo, ON))
Scott T. Leatherdale, PhD (School of Public Health, University of Waterloo, Waterloo, ON)
Report funded by
The COMPASS study was supported by a bridge grant from the Canadian Institutes of Health Research (CIHR) Institute of Nutrition, Metabolism and Diabetes (INMD) through the “Obesity – Interventions to Prevent or Treat” priority funding awards (OOP-110788; grant awarded to ST. Leatherdale) and an operating grant from the Canadian Institutes of Health Research (CIHR) Institute of Population and Public Health (IPPH) (MOP-114875; grant awarded to ST. Leatherdale).
Suggested citation
Thompson-Haile A, Leatherdale ST. Student-level Data Collection Procedures. COMPASS Technical Report Series. 2013; 1(5). Waterloo, Ontario: University of Waterloo. Available at: www.compass.uwaterloo.ca.
Contact
COMPASS research team University of Waterloo 200 University Ave West, BMH 1038 Waterloo, ON Canada N2L 3G1 compass@uwaterloo.ca.
Introduction
COMPASS is a longitudinal study (starting in 2012-13) designed to follow a cohort of grade 9 to 12 students attending a convenience sample of Ontario secondary schools for four years to understand how changes in school environment characteristics (policies, programs, built environment) are associated with changes in youth health behaviours. COMPASS originated to provide school stakeholders with the evidence to guide and evaluate school-based interventions related to obesity, healthy eating, tobacco use, alcohol and marijuana use, physical activity, sedentary behaviour, school connectedness, bullying, and academic achievement. COMPASS has been designed to facilitate multiple large-scale school-based data collections and uses in-class whole-school sampling data collection methods consistent with previous research [1-4]. COMPASS also facilitates knowledge transfer and exchange by annually providing each participating school with a school-specific feedback report that highlights the school specific prevalence for each outcome, comparisons to provincial and national norms or guidelines, and provides evidence-based suggestions for school-based interventions (programs and/or policies) designed to address the outcomes covered in the feedback report (refer to: www.compass.uwaterloo.ca).
Once schools have been successfully recruited to participate in COMPASS [5], specific data collection protocols are followed to ensure that data collections are well-organized in order to minimize school burden, meet ethical requirements, and be consistent across all participating schools. This technical report explains the student-level data collection procedures from the moment the school has agreed to participate in COMPASS until the date the survey is completed.
Online survey implementation system (OSIS)
While the school board and school recruitment procedures for the COMPASS baseline sample have been described elsewhere [5], it is important to note that during the school recruitment process for COMPASS, the COMPASS recruitment coordinator uses the Online Survey Implementation System (OSIS) to help guide them through the recruitment process and to store all of the school-specific information associated with each school’s data collection procedures. The OSIS tool is hosted on a secure server housed at the University of Waterloo accessed only by COMPASS staff. OSIS helps facilitate consistent, well-document and well-coordinated school-based data collections, and provides the COMPASS research team with valuable qualitative data pertaining to the implementation of each school’s data collection protocol. As such, once a school has been recruited for COMPASS, the data collection checklist in OSIS provides step by step guidelines for implementing the data collection at the school. The different sections detailed in this technical report correspond with the specific chronological tasks in the COMPASS OSIS data collection checklist.
Data collection start-up
Scheduling a data collection date
Once a school has agreed to participate in COMPASS, the recruitment coordinator makes contact with the school administrator responsible for the study coordination (school contact) via phone or email to determine the date the school would like to conduct the study. Schools are permitted to select any data collection date and time that fits with their school schedule, assuming the COMPASS team has a minimum of four weeks’ notice. Based on our previous research experience with whole-school data collections [6-8], four weeks ensures sufficient time to prepare all of the necessary materials, to acquire parental consent, and coordinate the data collection procedures for that particular school.
Class list enrolment information
After booking a data collection date and time, the COMPASS recruitment coordinator asks the school contact for the total school enrolment number and class lists; specifically, teacher name, grade level, and class enrolment numbers are required (e.g., Mrs. Smith, grade nine, 25 students). Having total enrolment allows for the adequate number of permission materials and questionnaires to be shipped to the school. Knowing teacher names and the number of students in each class allows the materials to be personalized and pre-packaged with the correct number of questionnaires for each class. The school contact can send the information in pre-composed school lists, or can use the COMPASS class info template that is provided upon request.
School welcome folder
After booking a data collection date, the school contact is mailed a COMPASS welcome folder via courier. The folder is promoted as an organizational tool for the school contact in order to make their involvement in the project as easy as possible. The folder includes a:
- welcome letter
- school contact checklist
- teacher letter and classroom poster
- class information template (if needed)
Welcome letter
The welcome letter begins by thanking the school contact for agreeing to participate in COMPASS, explains the purpose of the different items included in the folder, and provides the name and contact information of the COMPASS recruitment coordinator.
School contact checklist
The school contact checklist provides the school contact with a clear list of tasks and timelines associated with participating in COMPASS. The goal is to clearly lay out all of the different processes and milestones on one clear timeline so that the data collection process is clearly outlined and described for the school contact, resulting in a smooth and consistent data collection experience. For instance, the checklist informs the school contact when they should expect shipments of permission materials, when the permission materials need to be mailed to parents and when questionnaire bundles will be arriving at their school. Each step is associated with a predetermined date that corresponds to their school’s data collection date. Dates on this checklist are automatically populated in OSIS as soon as the data collection date for the school is determined. This monitoring helps to ensure that there are no milestones or tasks that have not been completed according to the appropriate timeline in each participating school.
Teacher letters and classroom posters
Teacher letters and classroom posters are provided in the welcome folder to inform teachers and students that the COMPASS study is being conducted at their school. The school contact is asked to distribute these materials to all of the teachers in the school. The teacher letter provides background information on the purpose of the study, why the study is important, the benefits of their school participating, and what the teacher can expect to have happen in their class on the day of the data collection. The classroom poster provides information to students pertaining to the purpose of the study, the benefits of participating, what to expect on the day of the survey, and contact information for the COMPASS study in case they have additional questions, comments, or concerns.
Class information template
In rare cases where class enrolment information is not obtained when the data collection date is set, the welcome folder would also include a class information template. The class information template is a simple resource to be completed by the school contact that collects details on the school enrolment numbers, class lists, and room numbers required for facilitating the data collection. The COMPASS recruitment coordinator contacts the school contact via phone or email within a week of sending the welcome folder to ensure that the folder has arrived and that the class information template has been completed and returned.
Permission materials
Consistent with the study protocol approved for funding from the Canadian Institutes of Health Research (CIHR) and the University of Waterloo Office of Research Ethics, active information passive consent permission (referred to as ‘passive consent’) protocols are used for all schools participating in COMPASS. Passive consent is a procedure whereby the parent(s) or guardian(s) of an eligible student are mailed an information letter about the COMPASS study and are asked to either (a) call the COMPASS recruitment coordinator using the 1-800 phone number provided in the information letter, or (b) email the COMPASS recruitment coordinator using the COMPASS email address provided in the information letter should they not want their child to participate. All eligible students whose parent(s) or guardian(s) do not contact the COMPASS team to withdraw their child are deemed eligible to participate. At any time during the consent process or during the data collection, an eligible student is allowed to decline to participate or withdraw from the study. The appropriateness and ethical considerations of passive consent for longitudinal school-based research (consistent with the COMPASS study methodology) have been described previously [9-10].
Permission materials are prepared by the COMPASS team at least four weeks prior to a school’s data collection date and are based on the enrolment numbers provided by the school contact. Permission letters are photocopied, stuffed and sealed into pre-paid postage envelopes before being couriered to the school contact. The recruitment coordinator’s return mailing address is on the envelopes, should any need to be returned. The school contact is provided with instructions regarding the distribution of the letters, timelines for distribution, instructions to affix student/parent address labels to the envelopes, four and instructions on when to place them in the mail (parents are required to have the letters at least 10 days before the survey date to ensure that they have time to read the letter and respond should they not want their child to participate in COMPASS).
Voice-relay or syner-voice has become increasingly popular at the secondary school level. This method of communication calls all student households with an automated message from the school. As such, schools are offered this method of distributing permission information if it is already an established communication protocol at their school. In this case, the COMPASS recruitment coordinator provides the school contact with a permission script rather than the posted letters.
The COMPASS recruitment coordinator fields all phone calls and emails from parents. Upon receiving communication, the coordinator asks for the student’s name, grade and school they attend. This information is recorded in OSIS under “record refusal calls”. The form of refusal (phone call, email) and date are also recorded here. As a precaution, the recruitment coordinator’s voicemail is set with a specific recording to parents who wish to have their son or daughter placed on the no permission list, should a call be made before or after working hours. These messages are checked daily and recorded in OSIS immediately. Any letters that are returned to the recruitment coordinator (wrong address, no longer at this address) are also recorded in OSIS under refusal calls, as the parents did not have an opportunity to read the letter. They are marked as “letter returned” in the form of refusal section in OSIS. All student names added under the “record refusal calls” in OSIS are automatically uploaded onto the “no permission list” for the school.
Prepare questionnaire bundles
Two days prior to the data collection date, the school contact receives a shipment of questionnaire bundles from the COMPASS recruitment coordinator containing the appropriate number of copies of the COMPASS questionnaire (Cq) to be completed by eligible students. The shipment includes a “School contact questionnaire instructions” letter providing clear directions for effectively distributing the Cqs to the classroom teachers and information regarding data collection day.
The school contact is also provided with enough copies of a “height & weight teacher memo” for each teacher. This memo is printed on bright green paper to get teachers’ attention. The letter explains to teachers that students in this age group sometimes do not know their actual height and/or weight and that the COMPASS team would be very appreciative if they could read a brief paragraph asking students to weigh themselves and measure their height the day before the survey. The letter also serves as a reminder to teachers that the data collection is taking place.
Questionnaire bundles are provided for each participating class and are distributed to classroom teachers before the survey date. Each bundle is addressed to a specific teacher by a “classroom envelope form” affixed to the front of the classroom envelope. On data collection day, teachers will use this form to fill out how many students were present during class time, how many completed the Cq and record any comments relevant to data collection. Each questionnaire bundle includes the following:
- teacher instruction questionnaire letter
- classroom instructions
- classroom frequently asked questions (FAQ)
- no permission list
- questionnaires and envelopes
Teacher instruction questionnaire letter
The teacher instruction questionnaire letter is a guide to help teachers administer the Cq to eligible students in their class. It describes how long students have to complete the Cq, how to answer frequently asked questions, when to read out the classroom instructions for completing the Cq, how Cqs are distributed to, and collected from eligible students, and how to protect the confidentiality of students and their data (e.g., the teacher is not allowed to see any of the student responses on the Cq at any point in time or help students answer any questions).
Classroom instructions
Before eligible students are allowed to complete the Cq, the teacher is asked to read the aloud the classroom instructions that have been provided by COMPASS staff. These instructions describe to eligible students why the study is important, why their participation is important, how to appropriately complete answers on the Cq, what to do if they do not understand a question, how to withdraw from participating at any time, and how the data they provide remain confidential and secure. These instructions emphasize the anonymity of all Cq responses and the need for honest, accurate reporting.
Classroom frequently asked questions
The classroom frequently asked questions document provides teachers with a list of frequently asked questions and the appropriate answers. These questions were selected based on our previous research experience with whole-school data collections [6-8]. This document helps to ensure that teachers can answer some general methodological or process questions about COMPASS without having detailed knowledge of the study, research design, or data management. This document also helps to ensure that teachers do not provide inaccurate information/responses to students on questions pertaining confidentiality, permission to participate, and data security.
No permission list
Each teacher is provided with a “no permission list” for the COMPASS study. The list is provided to each classroom highlighting the names of all students in the school who are not allowed to participate based on parent(s) or guardian(s) withdrawing their child from the study (as described earlier in the section on permission materials). Instructions associated with this no permission list clearly describe that none of the students named on this list are allowed to participate in the study and that they should not receive a copy of the Cq.
Questionnaires and envelopes
The bundle for each classroom contains the exact number of Cqs and envelopes for each student with permission to participate according to that class’s enrolment list provided by the school. As per the COMPASS study protocols designed to protect student confidentiality, the purpose of the envelopes is to allow each eligible student to seal his/her completed Cq in a blank envelop so that the survey can be handed in anonymously.
Immediately before the school data collection
Confirmation call
The day before the scheduled collection date, the COMPASS recruitment coordinator calls or emails the school contact. This point of contact serves several purposes:
- to ensure questionnaire bundles have arrived
- to provide the school contact with the name(s) of the COMPASS data collector who will be coming to their school to conduct the data collection
- to inform the school contact that updated “no permission lists” will be faxed or emailed to the school should any parents call between the time the bundles were initially prepared and the day of the data collection
During this call, the COMPASS recruitment coordinator asks and answers any final questions about data collection day protocols that the school contact may have.
Accessory materials for the data collection date
A COMPASS data container (CDC) is prepared for all participating COMPASS schools. The CDC accompanies the COMPASS data collector to the school on the data collection day and contains school specific information and extra materials should the data collector need them on collection day. The CDC includes: the school Information form, the list of participating classes, the “no permission” list, the data collection feedback form, 50 extra Cqs, 50 extra Cq envelopes, five-10 extra frequently asked questions pages, and five-10 extra teacher instruction pages.
The school information form provides the data collector with the school contact’s name and school specific information. A list of participating classes is also provided to ensure that the data collector knows how many classroom questionnaire bundles should be returned to them before leaving the school. This list is automatically generated in OSIS from the original class list information the school contact provided. Although each teacher is already provided with a copy of the “no permission list” the data collector has a copy for their records.
Data collectors are asked to complete the data collection feedback form. This form is used to make notes relevant to the data collection; for example if there was high absenteeism on data collection date, if the school had any comments about COMPASS, and generally the data collector’s overall feeling about the collection. This information is collected to ensure that all protocols were followed as planned and so the recruitment coordinator can follow up with the school if needed. Usually no immediate follow-up is necessary, but if a data collector has noted, for example, that one class did not complete the survey at the proposed time but would complete it later in the week, the recruitment coordinator would know to follow up with the school contact to ensure the questionnaires were completed and the bundle was mailed back to them.
While questionnaire bundles are prepared to reflect the exact number of eligible participating students based on enrolment lists, additional questionnaires and student envelopes are made available in case extras are needed. This may occur due to a counting error, a teacher misplacing their bundle, or in times when eligible students on spare periods decide to complete the survey. Extra copies of all the printed materials found in the questionnaire bundle are provided for these reasons.
Procedures on the day of the data collection day at the school
Questionnaire administration
The COMPASS data collector arrives at the school approximately 30 minutes prior to the start of the data collection and meets with the school contact. He/she confirms that all teachers have received their questionnaire bundles and updated “no permission lists” (if applicable). The data collector asks the school contact to make an announcement before the data collection begins, if this has not already been done. Teachers administer the Cqs in their classrooms following all directions that were provided to them in the classroom instructions included in their classroom questionnaire bundle. Once students in a classroom have completed their Cqs and sealed them in their individual envelopes, the entire class’s individual envelopes are collected by one student (selected by the teacher prior to starting the survey) and sealed in a larger classroom envelope. That student then returns his/her classroom envelope to the school office where they are picked up by the data collector. This process helps to maintain student confidentiality while also allowing the COMPASS data collector to record which classes have completed their data collection. As classroom bundles are brought to the office, the data collector checks off the classroom teacher’s name to ensure that all bundles return. Should there be any missing class bundles after the allocated 40 minutes, the school contact is asked to contact that class, by PA, phone or in person to pick up the remaining bundle or have them brought to the data collector. All classroom bundles are put back into the original boxes used initially to ship them for return to the University of Waterloo for processing.
References
- Leatherdale ST, Burkhalter R: The substance use profile of Canadian youth: exploring the prevalence of alcohol, drug and tobacco use by gender and grade. Addict Behav 2012, 37:318- 322.
- Leatherdale ST, Manske S, Faulkner G, Arbour K, Bredin C: A multi-level examination of school programs, policies and resources associated with physical activity among elementary school youth in the PLAY-ON study. Int J Behav Nutr Phys Act 2010, 25;6. doi: 10.1186/1479-5868-7-6.
- Leatherdale ST, McDonald PW, Cameron R, Brown KS: A multi-level analysis examining the relationship between social influences for smoking and smoking onset. Am J Health Behav 2005, 29:520-530.
- Leatherdale ST, Papadakis S: A multi-level examination of the association between older social models in the school environment and overweight and obesity among younger students. J Youth Adolesc 2011, 40:361-372.
- Thompson-Haile A, Leatherdale, ST. Baseline Sampling and Recruitment Results. COMPASS Technical Report Series. 2013;1(4). Waterloo, Ontario: University of Waterloo. Available at: www.compass.uwaterloo.ca.
- Elton-Marshall T, Leatherdale ST, Manske SR, Wong K, Ahmed R, Burkhalter R. Research Methods of the Youth Smoking Survey (YSS). Chronic Dis Inj Can 2011, 32, 47-54.
- Ahmed R, Leatherdale ST, Manske SR, Reid J, Burkhalter R. Effectiveness of school-based recruitment procedures and modular data collections. J Youth Dev 2010, 5, 68-80.
- Leatherdale ST, Wong S. Modifiable characteristics associated with sedentary behaviours among youth. In J Pediatr Obes 2008, 3, 93-101.
- Hollmann CM, McNamara JR. Considerations in the use of active and passive parental consent procedures. J Psych Interdisc Appl 1999, 133, 141-156.
- White VM, Hill DJ, Effendi Y. How does active parental consent influence the findings of drug-use surveys in schools? Eval Rev 2004, 28, 246-260.