Notice: The Statistical Consulting and Survey Research Unit was previously called the Survey Research Centre. Our unit is referred to by this name in old newsletters.
The Pros and Cons of Conducting Your Own Online Survey Research
Requests to complete online surveys are ubiquitous, whether it is a retail experience survey after a Walmart purchase, an employer survey about how you travel to work or a survey on your experience navigating a website. Free or inexpensive do-it-yourself online survey software has provided open access to a powerful research method. However, conducting research on your own using free or inexpensive software may not always result in the collection of reliable or useful data. While by no means an exhaustive list, below are some of the key pros for, and cons against, conducting online survey research with free or low cost survey software rather than working with a professional research centre.
Pros:
Access - Anyone with internet access can use free survey software to create a survey.
Ease - If a member of the research team has expertise in survey research, and the survey questions are simple i.e., there is no complex question routing required, do-it-yourself survey software may be fairly straight forward to use. Programming experience is not necessary for some free software. However, researchers must be prepared to spend the time to program and test the survey properly.
Low Cost - When researchers want quick and easily obtained survey results that are not being used to influence key decisions or meet rigorous academic publishing standards, free or inexpensive do-it-yourself software may be an alternative to not conducting the research at all, especially when there is no budget available to outsource the data collection services. Using this software can also provide a cost-effective way to collect pilot data or test survey questions. It may still be wise to seek expert advice on the sampling approach and questionnaire development.
Cons:
Data quality - Determining the most effective sample and survey method, and constructing a well- designed survey that addresses the key research objectives, are vital to ensure that valid, reliable and useful data are collected. Researchers who are inexperienced with survey research have a higher risk of erroneously introducing bias through their sampling approach or developing survey questions that are improperly structured or worded. Either of these may result in invalid research results.
Confidentiality - Survey respondents might be more likely to trust survey invitations from a professional research centre. A research centre can provide assurance that respondents will not be personally identified either through the data provided to the researcher or through processes such as machine identifiers that are used by some free survey software. This assurance of confidentiality can lead to higher respondent participation and more candid survey responses.
Data security – Data security is an important and growing consideration. The data that are collected using some free survey software are often stored outside of Canada, beyond the protection afforded by Canadian privacy laws. In some cases the data may not be adequately secured and thus at risk of theft. A reputable research centre stores the data securely on private, protected servers within strong firewalls and have strict protocols for data access and retention.
Weighing the costs
The old adage “You get what you pay for” is true for survey research. A poorly selected sample or a badly designed survey can impact the accuracy and meaningfulness of the data collected, potentially producing misleading research conclusions. This can have a wide-ranging effect if the data are published and/or are used to influence policy decisions. Seeking professional help to ensure surveys are conducted using best practice approaches will not only maximize survey response rates, but also prevent costly mistakes.
Help is available
Anyone can ask survey questions, but it takes expertise to collect meaningful data. If you require valid, reliable and useful results for key decision making purposes and/or publishing, it is best not to monkey around; contact the experts. The University of Waterloo offers many avenues for researchers seeking professional survey research services:
Conducting survey research? The Survey Research Centre (SRC), University of Waterloo, has expertise in survey design, methodology and data collection, conducting hundreds of surveys since its inception in 1999. The SRC offers services from simple consultation to a complete package of design, data collection and top-line analysis – using a cost-recovery pricing model. Data collection services include online survey hosting, telephone call centre services, mail survey and face-to-face interviewing services. uwaterloo.ca/scsru/survey-research
Crunching data? The Statistical Consulting and Collaborative Research Unit (SCCRU) offers services in statistical consultation including, but not limited to study design, data checking and data analysis. The SCCR also promotes interdisciplinary collaborations with experts within the Department of Statistics and Actuarial Science to solve complex academic research problems. uwaterloo.ca/scsru/statistical-consulting
Looking for available quantitative social science research data? The Research Data Centre (SWO-RDC) offers secure access to confidential micro-data from Statistics Canada censuses and surveys. Staffed by a Statistics Canada analyst, SWO-RDC provides statistical software and technical support to researchers from the University of Waterloo or Wilfrid Laurier University. https://crdcn.org/south-western-ontario-rdc-swo-rdc
The ITC Project: Survey Research to Accelerate the Fight Against the Global Tobacco Epidemic
Tobacco use remains the number one preventable cause of disease and death worldwide. A dramatic shift is occurring in tobacco use from high-income countries to low- and middle-income countries, and simultaneously the global demand for e-cigarettes is growing rapidly.
The International Tobacco Control Policy Evaluation (ITC) Project (www.itcproject.org) is a prospective cohort study founded by University of Waterloo Professor Geoffrey T. Fong to evaluate the impact of tobacco control policies of the World Health Organization (WHO) Framework Convention on Tobacco Control (FCTC) – the first-ever public health treaty, adopted in May 2003.
In 2002, the ITC Project began with the first wave of the ITC Four Country (4C) Survey in Canada, US, Australia, and UK. To date, the ITC Project has conducted over 100 survey waves in 28 countries, inhabited by 50% of the world’s population and 70% of the world’s tobacco users over 6 continents. The project includes high-income countries such as the 4C, France, and South Korea, and low/middle-income countries such as China, India, Bangladesh, Brazil, Mexico, Romania, Kenya, and Zambia.
The survey methods used across the 28 countries are diverse, but despite the diversity, the ITC surveys and survey protocols are linked and coordinated, allowing for strong potential for cross-country comparisons in evaluating policy impact.
Since 2010, the Survey Research Centre (SRC) at the University of Waterloo has been a key contributor to the ITC Project during the ITC 4C Survey’s transition from telephone surveys to interactive web surveys, a transition necessitated by the increasing challenges in conducting telephone surveys for this study.
Measuring the impact of e-cigarettes and vapourized nicotine products
The productive collaboration between ITC and the SRC continues with ITC’s new $20-million research project from the US National Cancer Institute. This project measures the impact of policies/regulations on e-cigarettes and other vapourized nicotine products (VNPs) to address key unanswered questions about the impact of such policies on VNP use and the effectiveness of VNPs as a cessation aid. Findings from the new 4C Project on e-cigarettes are highly anticipated by policymakers to inform effective regulation of VNPs.
The ITC Project has helped many countries in advancing tobacco control policies. In 2010, ITC findings presented at the Health Standing Committee of the Canada House of Commons showed that impact of the health warnings on cigarette packs introduced in 2001 had declined significantly. This contributed to the revision of those warnings, which were introduced in June 2012.
Advancing plain packaging
In recent years, plain/standardised packaging—which replaces attractive graphic pack design with dull-coloured packages and restricts pack shape, size, and font, while retaining the mandated health warnings—has emerged as a significant demand reduction policy measure, and the ITC Project has contributed to its advance.
In December 2015, the ITC Project worked with the British Heart Foundation (BHF) to produce a report reviewing the research supporting plain packaging. The BHF/ITC Report was presented in the British House of Lords. In May 2016, plain packaging was introduced in the UK. More recently, the ITC Project submitted a review of the evidence supporting plain packaging to the Canadian Government, which will soon be announcing its own plain packaging regulations.
ITC evidence has also been used to defend plain packaging in Australia, the first country to introduce plain packaging, in legal challenges.
One ITC study demonstrated that plain packaging significantly increased the effectiveness of the Australian health warnings; another ITC study found that Australian smokers’ support for plain packaging increased significantly after plain packaging was introduced. Finally, Professor Fong and two other ITC researchers have served as experts for the Australian government in defending plain packaging from the challenge brought at the World Trade Organization by four countries (with legal assistance from the tobacco industry).
The ITC Project’s successful collaboration with the SRC has led to major contributions to promote strong evidence-based approaches to combatting the world’s number one preventable cause of death and disease. The Survey Research Centre, University of Waterloo is proud to assist the ITC team with this important research. For more information on how the Survey Research Centre can help you to better understand populations of interest, please contact us.
Featured Projects
Canadian Paramedic Health and Wellness Survey: The Canadian Safety and Security Program (CSSP) and researchers at the University of Waterloo and Wilfrid Laurier University were interested in identifying and documenting the workforce profile, demographics and health and wellness of Canadian paramedics and other emergency medical responders. Mixed-mode methodology was employed. An online survey link was emailed, and paper surveys with a pre-paid return envelope were mailed, to numerous paramedic work locations across Canada. A snowballing approach was implemented by the research team for the online survey recruitment. A link to the online survey was provided in the mailed paper survey package and in targeted communication vehicles such as the Paramedic Association of Canada e-newsletter and through email campaigns to the Paramedic Chiefs. The Survey Research Centre (SRC) provided consulting services on the survey methodology used, and was responsible for hosting and data collection for the online survey and for data entry of the returned mailed surveys. The online survey was launched in May 2016 and remained open for approximately 9 weeks, closing in August 2016. Data collection for this first ever, comprehensive research of Canadian paramedics and emergency medical responders resulted in over 2,500 completed surveys.
Grand River Hospital Quality of Life Study: Making efforts towards continually improving the patient experience is a priority at Grand River Hospital (GRH). Consequently, GRH, a multi-site hospital serving Waterloo Region and Guelph-Wellington, has commissioned the Survey Research Centre (SRC) to conduct research to better understand patients’ experiences for two of its clinical programs: 1) Complex Continuing Care, 2) Mental Health program in both the Specialized Mental Health and Adult Inpatient Mental Health units. This study follows the same ‘best practice’ methodological approach developed previously by the SRC and GRH. On-site, face-to-face interviews take place with patients in each of the three program areas, in two hospital campuses. Patients are interviewed one to two days prior to their discharge date. Only patients who are age 18 or older are contacted to complete the on-site Quality of Life survey. The first wave of data collection occurred in November and December 2016 and a second wave is slated for February and March 2017. It is estimated that N=208 patients will be contacted during the first two waves of this research. More information about GRH is available at http://www.grhosp.on.ca/about/quality-and-patient-safety
Ontario Retirement Survey: Researchers from the Statistics and Actuarial Science department at the University of Waterloo were interested in studying financial retirement expectations and experiences of Canadians. The objective of the study was to develop a better understanding of the concerns and risk preferences of individuals who are either close to retirement, or who are already retired. Data collected will allow the researchers to assess whether the retirement income plans offered by insurance companies, or provided through occupational pension plans, are adequately meeting the needs of retirees. The research was partially funded by the Canadian Institute of Actuaries. The Survey Research Centre provided survey design and sampling consultation, as well as data collection for this project. An online survey was conducted among Ontario respondents aged 50 to 80 who self-identified as either pre-retired or retired. An online panel firm was used for participant recruitment. The online survey was launched in September 2016 and remained open for 3 weeks, closing in October 2016. The goal of 1,000 completed surveys was successfully reached.
Announcements
- We are pleased to announce that Beth McLay has joined the SRC on a permanent basis as Project Manager. We also welcome Mariam Mobasher as Project Support and Administration.