SCSRU Newsletter Vol. 19, Issue #1 – Winter 2018

Telephone data collection was once the go-to method for conducting survey research.  With the invention of the smart phone, and the popularity of text and email communication, web surveys are now in contention for the mode of choice for researchers.  Web surveys can be more cost effective, and data can be collected within a fraction of the time required for telephone surveys.  With the click of a button, web survey invitations are sent to thousands of people via email.   There are no long distance telephone charges, interviewer labour costs, and other lab fees associated with telephone surveys.  There are, however, many noteworthy benefits of conducting surveys by telephone; maintaining the position of telephone interviewing as a relevant, and at times, a necessary form of data collection.

Representativeness and validity

Telephone sampling frames provide a broader reach of participants, potentially resulting in a more representative sample of respondents compared to most web survey sampling frames.  Random digit dialing (RDD) is often used with telephone data collection, where landline and cell phone telephone numbers are generated randomly.  RDD provides the advantage of including unlisted numbers and cell phone only households that would otherwise be missed if the numbers were selected only from phone book listings.  Inclusion of cell phone numbers improves the likelihood of connecting with younger respondents and/or cell phone only households.  In contrast, the majority of the web panels from commercial survey firms are not probability based. This means that not everyone in the population has an equal opportunity to be selected for the survey; instead only those who have elected to be a member of the web panel can be contacted.  In addition, some web panel members are considered “professional” survey takers completing many surveys a month and therefore, may not be representative of a typical respondent. Not providing all subjects of the population an opportunity to voice their opinion can have serious consequences for the representativeness of the sample and the validity of the survey data.

Respondent participation

There is potential for better response rates when conducting surveys by telephone compared to other modes.  Telephone interviewers are trained in the use of dialogue for refusal conversion.  Respondents often disclose a number of reasons why they do not wish to participate in a survey.   Interviewers are able to tailor their response to these concerns, and counteract them with a situationally appropriate reply.  The use of tailored replies is often successful in converting refusals into completed surveys.  Interviewers also encourage participation by answering respondent questions regarding the research, assuring confidentiality, and by scheduling callbacks for a more convenient time.  It is far more difficult to refuse a person, than it is to ignore an email invitation.

Data quality

There are better assurances of data quality through telephone data collection.   Interviewers are skilled at conducting surveys, and coding responses.  In a web survey, respondents have the ability to speed through a survey with little thought given to the survey answers. However, in a telephone survey, live interviewers are able to provide data quality checks throughout the survey by probing when more information is needed and asking for clarification to ensure the answers are complete and accurate.  Interviewers are also able to document conditional responses, conflicting responses, and other notes that may be of benefit to the researcher.  Not only are interviewers highly trained, they are monitored to ensure the highest level of data quality, through both close supervision of their calls, and on-going data quality monitoring.  When it comes to data quality, data captured by telephone interviewers can be far superior to self-completed surveys by web respondents, particularly for long, complex and/or repetitive surveys.

Offering experiential learning opportunities for students

As an institution for higher learning, the University of Waterloo Survey Research Centre (SRC) maintains a staff of telephone interviewers who are highly engaged and intelligent. The SRC interviewing staff is largely composed of students from various faculties within University of Waterloo, Wilfrid Laurier University, and Conestoga College, who have an interest in gaining experience in primary research.  This level of education and skill set ensures a high standard of data quality, while providing students with valuable work experience. The opportunity to work as a telephone interviewer allows students to further their education in research methodologies and gain a foundation in professional development. 

There are many benefits to conducting surveys by telephone.  A sampling frame that is representative of the population helps to ensure the reliability of the data. RDD and cell phone samples further avoid demographic bias and reach a wider range of respondents.  A strong emphasis is made on quality control for surveys conducted over the telephone with interviewers, while information collected through self-administered web surveys must be evaluated through other means. Telephone interviewers have a strong impact on improved response rates and data quality. The use of probing techniques and refusal conversion training helps to increase participation and capture accurate responses.  Although data collected by telephone may not be the most cost effective approach, what is lost in terms of costs is more than made up for with the reliability and quality of the data captured.

The experts at the Survey Research Centre provide questionnaire design, data collection and top-line data analysis for telephone surveys as well as for web, mail and mixed-mode surveys. For more information on how we can help you choose the research methodology to best meet your research objectives and produce valid, reliable and useful data, please contact us.

Back to the top

In January 2015, the Province of Ontario passed legislation legalizing online gambling.  A longitudinal study to examine the impact of this legislation on gambling behaviour was conducted by the Survey Research Centre, University of Waterloo, on behalf of Dr. Robert Williams, University of Lethbridge, and the Alberta Gambling Research Institute. The purpose of the study was to evaluate the impact of legalization of online gambling on gambling behaviour, particularly online gambling behaviour.

Population prevalence studies of gambling serve several important purposes. They establish the current overall prevalence of gambling, the prevalence of each form of gambling, personal expenditures on each form of gambling, and the prevalence of problem gambling¹. This information is very useful in understanding the overall recreational value of gambling to society, the negative social impacts of providing legalized gambling, the actual number of problem gamblers in need of treatment, the proportion of gambling revenue derived from problem gamblers, and the types of gambling most strongly associated with problem gambling. Changes in the prevalence of problem gambling from one time period to the next, and/or differences between the prevalence in one jurisdiction relative to another, provide important information about the incidence of problem gambling and the potential effectiveness of policies implemented to mitigate gambling’s harm (Volberg, 2007; Williams & Volberg, 2012).

Three waves of survey data collection were conducted: fall 2012, fall 2013, and winter 2016. In order to maintain consistency, the questionnaire, perfected in the Best Practices in Problem Gambling research, was used in order to compare the prevalence rates to other types of gambling that were researched in previous iterations of that study. The questionnaire was presented as a “health and recreational activities” survey to potential participants in order to more accurately measure gambling activity, as opposed to presenting the survey as a ‘gambling study’ which typically results in over-representation of problem gamblers.

For the 2012 and 2013 waves of the Internet Gambling Prevalence in Ontario study, data were collected from members of an online panel group. Across the two waves n=3,622 respondents were recruited. These 3,622 respondents were re-invited to complete the survey in 2016. A total of 1,589 surveys were completed in 2016; 73.4% of the surveys completed in 2016 were returning respondents from the previous 2013 wave and 26.6% were from the 2012 wave. The sample represented Ontario residents age 18 years or older.

In 2016 telephone recruitment also took place, contacting new respondents to ensure better representativeness, compared to focusing solely on online panel participants. A total of 44,236 telephone records, including both cell phone and landline numbers, were attempted, with 1,501 telephone surveys completed.

Composition and Quality of Online Sample

Duplicate respondents are a possibility any time panel sample is used. To minimize duplicate respondents, the SRC created and assigned unique access identifiers to each respondent on a one-to-one basis with the list of unique panel member IDs. This means that each SRC ID corresponded to one specific panel member prior to the launch of the study. Because each SRC ID was matched in this way, the only possible source for duplicates was duplicate recruitment by the panel firm, which was assured to be of low risk.

Quotas

Web and telephone data collection used quotas to ensure that age and gender distributions would fall within an acceptable range to represent census proportions.

The Survey Research Centre, University of Waterloo is proud to assist Dr. Williams and the Alberta Gambling Research Institute with this important research. For more information on how the Survey Research Centre can help you to better understand populations of interest, or help you to conduct complex longitudinal studies, please contact us.

¹ Problem gambling is defined as having difficulties limiting money and/or time spent on gambling which leads to adverse consequences for the gambler, others, or for the community (Neal, Delfabbro, & O’Neil, 2005). It includes ‘pathological gambling’ (equivalent to severe problem gambling) that is characterized by severe difficulties in controlling gambling behaviour leading to serious adverse consequences.

Back to the top

Governing and Accelerating Transformative Entrepreneurship Survey (GATE):Dr. Sarah Burch of the Department of Geography and Environmental Management at the University of Waterloo is studying small and medium sized businesses with regard to their role in fostering the wellbeing of communities and their potential to create significant technical and social innovation.  The objectives of the GATE study are to better understand the barriers and opportunities that these businesses face in adopting and implementing more sustainable business practices.  A total of 1,735 online surveys were completed among small and medium sized businesses in Toronto, Vancouver, and London (UK). The sampling frame of small and medium sized businesses was compiled through publicly available records and though purchased lists. The sampling frame for Vancouver businesses also included members from a panel firm.  The survey was conducted from May 2017 to August 2017.

Pacific Northwest Social Survey: Dr. Sarah Wilkins-Laflamme from the University of Waterloo, as well as principal investigator Dr. Paul Bramadat and his research team at the University of Victoria, are studying the new realities and trends of the Cascadian religious and spiritual landscape. Cascadia is a bioregion located mainly in British Columbia and the U.S. states of Washington and Oregon. The research group’s specific focus is the differences between the Canadian and U.S. Cascadian regions regarding religious/spiritual beliefs and practices; inclusivity of religious minorities; and the impact in the public sphere and on social policy. Quotas were put in place for region (Canada and U.S.), gender and age group. A total of 1,510 surveys were completed online by members of an online panel group. The study was conducted in October 2017.

Survey on Ethical Behaviour (Policy 33): The University of Waterloo is undertaking a revision of Policy 33, Ethical Behaviour. The SRC is collecting survey data among the university community in an effort to provide a better understanding of the university work environment with respect to human rights and ethical behaviour. The research will also inform the university about the allocation of support resources for individuals experiencing difficulty in the workplace. The survey launched in October and is being administered both online and on paper to undergraduate students, graduate students, faculty and staff. The online survey will remain open until March 2018.

Back to the top

• In Fall 2017, the University of Waterloo announced a campus-wide launch of Qualtrics, a powerful online survey system, to support the needs of the campus community to host secure, confidential online surveys. This software is available for use by all University of Waterloo staff, faculty and graduate students.  In response to this announcement and to continue its support of survey research within the UW community, the Survey Research Centre is now offering Qualtrics programming services to University of Waterloo faculty, staff and graduate students. Visit our Survey Research Services page for more details about this new service.
• Are you a University of Waterloo graduate student with questions about survey research? The SRC is forming a graduate student network to connect and support graduate students interested in discussing or learning about best practices in survey design and methodology. [updated 2023: not currently active]
• We are happy to welcome Janice Lam to the SRC in the role of Project Support and Administration. Janice is responsible for day-to-day administrative tasks and is the contact person for consultation and proposal requests.

• Dr. Geoffrey Fong, founder and Principal Investigator for the International Tobacco Control Policy Evaluation (ITC) Project, has been named a fellow of The Royal Society of Canada (RSC) and winner of the 2017 CIHR_IPPH Trailblazer Award. The ITC Project evaluates the impact of tobacco control policies in over 25 countries and according to the RSC citation, has made “groundbreaking contributions to advancing science and policies to accelerate and strengthen governmental and advocacy efforts to combat the global tobacco epidemic”.

Back to the top