SCSRU Newsletter Vol. 15, Issue #1 – Summer 2013

By: Sascha Lecours

After the SRC’s recent move from Sawtooth’s WinCati to its new Voxco telephone software, the University of Waterloo needed to replace Sawtooth’s Sensus web software with new software for creating and managing web surveys. The software selected for this purpose was MMIC (Multimode Interviewing Capability). MMIC is a data collection system developed by the RAND Corporation, a nonprofit organization that conducts research in areas such as economics, science and technology, and sociology.

MMIC has been used to conduct research by survey institutes around the world including: the National School of Development at Peking University, Trinity College Dublin, the International Institute for Population Sciences - Mumbai, the Harvard School of Public Health, the RAND corporation American Life Panel, and many others.

MMIC has a number of useful new features compared to the previous software:

• Survey design and management is done entirely in-browser, meaning that it is not necessary to download or install anything in order to program or maintain a web survey - any machine with a web browser and internet access can be used, allowing for a great deal of flexibility for the survey designer or administrator.
• MMIC’s programming interface is very straightforward and user friendly. To program a simple survey (i.e. one without complex routing requirements), all that is required is to set the question and answer text for each question simply by typing in the text as you want it to appear in the survey and then specifying the order that the questions should appear. No special programming code or jargon is needed. As a result, little to no web survey programming knowledge is necessary to program a simple web survey in MMIC. In terms of designing a straightforward web survey, MMIC is extremely easy to use in comparison to most other web survey software.
• MMIC’s survey programming tools are also very powerful; as easy as it is to design a simple web survey, MMIC offers a great deal of possibilities for more advanced survey design, such as the use of loops, arrays, easy integration of HTML or PHP code, very strong tools for dynamic text, and good support for non-English languages and characters.
• MMIC offers a number of useful features to facilitate survey design and review, such as the ability to automatically generate the survey in flow-chart form or show the survey in image form as a series of screenshots.
• MMIC exports data to STATA, SPSS and .CSV formats directly.

The MMIC development team has created an extensive user-manual which provides in-depth documentation on the features of MMIC. Additionally, the SRC has developed a quick-start user manual for use by the University of Waterloo community. This quick-start guide emphasizes setting up low-complexity surveys and includes step-by-step instructions with screenshots. It provides all of the information needed to program, launch, and maintain a MMIC web survey.

Overall, MMIC is a considerable improvement in terms of ease-of-use and flexibility when creating web surveys.

Back to the top

By: Lindsey Webster

The 2012 AAPOR conference was held in sunny Orlando Florida, known for its world class theme parks and golf courses. I had previously been given the opportunity to attend the 2009 AAPOR Conference which was held in Hollywood, Florida and was very excited to return to the beautiful state. My husband and I flew down together instead of driving (as we did previously) in order to have time to experience all that Orlando had to offer.

The conference was held at the JW Marriott Orlando Grande Lakes hotel. The hotel was like a resort in paradise, surrounded by tropical gardens, waterfalls and a lazy river complete with inner tube rental. The grand lobby was a jaw dropping spectacle filled with flowers and an enormous fountain. Our room had a wonderful view of the beautifully landscaped golf course which my husband relished every morning. Every day that I was attending sessions, you could be sure that he was golfing!

The conference theme was Evaluating New Frontiers in Public Opinion and Social Research. It focused on evaluating the strengths and weaknesses of new methodologies and modes of data collection and explored the implications on the profession of survey research. I enjoyed numerous fascinating sessions coinciding with this theme including Interactive and Gaming Techniques to Improve Surveys, New Technologies and Web Surveys, and Comparing Data Collected Using Mobile Devices with Other Survey Modes. There were also some more traditional sessions on topics such as issues in non-response, cell phone sampling, and the role of the interviewer which I also attended. I saw my old friends and colleagues from ASDE Survey Sampler. I also had the chance to meet the representative from Voxco, our current CATI/CAWI software provider. I visited their booth in the exhibit hall as well as others from representatives of key service and product providers. The AAPOR conference is never about all work and no play. A new social event this year was a post-banquet party, held in the exotic citrus garden. It was a great way to mingle with other attendees while enjoying specialty cocktails and unplugged acoustic entertainment.

I jumped at the opportunity to take a highly-recommended short course called Web Survey Design, presented by Mick Couper. SRC clientele have expressed increasing interest in web surveys which is a trend within our profession. The goals of the course were to help understand how web survey design may differ from that of other survey modes. It covered a number of evidence-based design guidelines for designing web survey instruments to minimize measurement error, missing data and break offs. Designing questions appropriately can have a big impact on the quality of the data that is captured. For example, it is important to know when and how to use drop boxes, check boxes, radio buttons, and text fields to our advantage. The goal is to match the choice of tool and the design of the tool to the type of response desired.

The layout and format of the screen can significantly impact the usability of the web survey thus affecting response rates as well as the data captured. The use of images, choice of font, colour and text size can all affect the user’s experience. A web survey should be designed to focus the respondent on their task of reading, comprehending, and responding to the questions. Applying principles of visual perception and respecting visual flow can aid respondents in this task. It is important to achieve a balance between focus on task and aesthetically pleasing design. The course was very informative and Mick was a very captivating presenter. I feel that the SRC can apply many of the techniques that I learned in this short course. It was certainly a highlight of the conference for me.

On my days off my husband and I visited some of Florida’s best attractions. As a rollercoaster enthusiast and animal lover, Busch Gardens was at the top of my list. It was the perfect combination of zoo and amusement park. Our top two favorite rides were Sheikra and Cheetah hunt. The latter being the parks newest and longest thrill ride at a length of 4,400 feet. I also loved Walkabout Way, where I could get up close and personal to kangaroos and other animals from Australia. A trip to Florida wouldn’t be complete unless you get a chance to see the states notorious predator, the American Alligator! The best place to do this is at the Lone Cabbage Fish Camp where you can take a guided airboat ride through the everglades. It was wonderful to be able to experience the Alligator in its natural habitat. We also saw a number of birds and other animals while flying over the water on this exhilarating boat ride. When you return to the dock you can feast on Alligator, frog’s legs and other swamp delicacies at the Lone Cabbage restaurant. The Alligator is a favorite of mine... mmmm tastes like chicken!

The 2012 conference certainly did not disappoint. It was an invaluable experience once again and a great opportunity to be part of this yearly gathering of professionals in public opinion research. I look forward to the next conference and welcome more opportunities to learn about the current trends in survey research.

Back to the top

By: Andrew Kraan

In February 2012, SRC launched a new data collection tool for telephone, web, and mixed mode surveys called Voxco. Previously, the Centre was using Sawtooth’s Mixed-Mode software. Switching to Voxco has brought many improvements over Sawtooth’s software, along with a few issues.

Several features of Voxco have enhanced study management capabilities. The sample management tools make it easy to organize records based on variables such as phone number, time zone, and language group. The callback rules allow for creation of sophisticated settings for determining when records enter the calling queue. Callbacks can also be rescheduled for large quantities of records. The data extraction process permits the selection or omission of specific fields, and delivers datasets in multiple file formats (including several Excel formats, SPSS, CSV, etc.). It is quite simple to view and edit data using the sample browser. Voxco contains a “print questionnaire” function which outlines the text and values of each variable in a questionnaire and serves as an indispensable aid when conducting data quality monitoring. The email manager makes it possible to manage large groups of outgoing messages, while still addressing them to individual respondents. One drawback is that it is not possible to have multiple e-mail inboxes in Voxco, which makes sorting emails for multiple projects somewhat tedious.

In terms of programming, Voxco enables a higher degree of customization. Previously, with Sawtooth’s software, programmers were forced to work within rigid ‘templates’ which did not permit for code modifications. Our programmers can now write the HTML code themselves, and control exactly how a survey appears and functions. Furthermore, it is now possible to have a truly “mixed mode” survey which is accessible both via phone and web. Running the survey through a web browser makes it easy for respondents to complete a survey on their own on the web, over the phone with an interviewer, or by using a combination of the two modes.

From the interviewers’ perspective, Voxco’s clean, neutral aesthetic is much more visually appealing than the cluttered, oversaturated appearance of WinCATI. While our interviewers were relieved to switch to a system that was easier on their eyes, it took some time to adjust to using a browser based program which experiences errors when someone inadvertently clicks ‘back’ or ‘refresh’. One advantage of WinCATI was the ability to read previous notes on a record before accessing it. In Voxco, interviewers cannot read existing notes without entering the record first, which forces them to code a disposition even if a call attempt is not made. Fortunately, there is a hierarchy of dispositions which ensures that important/final dispositions (e.g. complete, refusal) are never overwritten. However, completion and productivity statistics can be distorted in situations where a call attempt is recorded even though the record is accessed but not called.

The reporting tools in Voxco have caused some frustration, as productivity and completion reports often don’t calculate correctly. This usually occurs in cases where studies have multiple and/or complex, ‘completion’ dispositions. The monitoring tool used to evaluate interviewer performance has also been unreliable at times, forcing the SRC to employ band-aid solutions which fail to address the root of the problem.

The most significant issue with Voxco is that it tends to be very sensitive to network connection issues between different servers, as the Voxco system requires a constant and synchronized connection between servers. Issues can arise when data are being backed up or if an interruption to the Internet connection occurs. These situations are often difficult to predict. The only solution that seems to fix these problems is to reset the servers and manually re-start each survey individually within Voxco. This is not ideal, as it requires kicking out any respondent who might be in the middle of a web survey, as well as possibly interrupting telephone dialing.

Voxco support is an improvement over the support that was provided by Sawtooth, especially in terms of service provider accessibility. It is usually easy to reach someone and small requests (e.g. increasing the number of possible connections) have been dealt with promptly and efficiently. However, problem resolution for larger issues (e.g. server connections, monitoring) sometimes require one of our managers acting as an intermediary between a Voxco representative and a University of Waterloo server administrator in order to determine the true source of the problem.

Having used Voxco for more than a year now, the SRC can certainly say that it is an improvement over the WinCATI software. There is increased efficiency in managing studies, more freedom for programmers to customize their code, and more reliable support.

Back to the top

CAMH Harm from Alcohol Pilot

The purpose of this research is to collect estimates of the extent, costs and type of harm to others from alcohol use. This pilot marks the first time in Canada that researchers have explored harm to others from alcohol use, as the focus has previously been on harm to self. Participants 18 years and older were recruited through random digit dialing of both landline and cell phone records in British Columbia, Saskatchewan, Ontario, Quebec and Nova Scotia.

CAMH – Smoking Research Study Survey

The purpose of this research is to determine the efficacy of smoking cessation programs that involve mass distribution of Nicotine Replacement Therapy. The study is designed as a single-blinded panel survey with random assignment to an experimental and control condition and involves two stages of recruitment and three telephone interviews. Households across Canada are contacted using a Random Digit Dialing approach. Eligible respondents over the age of 18 are screened and recruited to either an experimental or control group, with those in the experimental group receiving a free 5 week supply of nicotine patches. Eligible respondents in both groups are also asked to provide a saliva sample. Follow-up interviews are then scheduled to take place 8 weeks and 6 months after completing the initial survey.

CEMC TAM and CSG Workshop Evaluation

The purpose of the study is to evaluate and monitor the Centre for Education in Math and Computers (CEMC) Think About Math (TAM) and Computer Science For Young Women (CSG) outreach programs offered each May and June to high school aged girls. The surveys obtain a snapshot of current school courses taken and involvement in enrichment programs, as well as measure current attitudes about math and computers, and assess the individual elements of the outreach programs themselves.

International Tobacco Control 4C9 (UK, Australia, Canada, USA)

The ITC Project is a multi-disciplinary collaboration of more than 80 researchers in 23 countries who evaluate the effects of national tobacco control policies on smoking behavior. Its principal funding comes from the NIH and CIHR. Geoff Fong, Mary Thompson and Dave Hammond are investigators on the project. The re-contact and replenishment web surveys for the 4C9 countries are conducted by the SRC. The UK and Australia Re-contact data collection launched in February, 2013, with replenishment data collection for these two countries starting in April. The Canada and US components of the study will take place throughout fiscal 2013-14.

Fall Prevention Study, University of Guelph

An eight month campaign running from October 2013 through May 2014 will focus on fall prevention among young children aged 1-4. The eight month campaign consists of a social marketing campaign as well as two additional interventions. Effectiveness of the campaign and the interventions will be assessed by conducting telephone interviews across six centres. Two centres will be exposed to the social marketing campaign only. Two of the centres will be exposed to the social marketing campaign as well as the two additional interventions. The last two centres will be the control group and will not be exposed to the social marketing campaign or the additional interventions. Two waves of data collection are proposed; a pre-wave prior to the launch of the social marketing campaign and interventions, and a post-wave conducted one year later.

McGill Child Health Study

The purpose of this research is twofold; first, the researchers are trying to understand the ways in which societal/environmental conditions in the home, school and community impact children’s body weight, BMI and healthy living-related behavior. The second purpose is to explore how biological (i.e. genetic and neurocognitive) differences impact the way children respond to these societal/environmental conditions. The focus is specifically on children between the ages of 6-12 living in the greater Montreal area, with the primary data being collected from the parent/guardian who is most familiar with the eating and physical activity habits of the child. Respondents are recruited into the study by completing a phone survey. The study also involves two separate follow-up components that telephone respondents can elect to complete; the provision of a DNA saliva sample from the child and completion of 20 psychological rating statements, and an in-depth complementary follow-up survey which respondents can choose to complete online or on paper.

Menu Labeling Study

The purpose of this research is to examine food choices in restaurants and the impact, if any, of visible nutrition information for menu items. A random digit dialing approach that included both cell phone and landline records was used to achieve a representative sample of the adult (i.e. 18 years or older) population in Canada and the U.S. 1000 surveys were conducted in each separate jurisdiction, with B.C. considered a separate jurisdiction from the rest of Canada. The study consists of administering two identical surveys one year apart. The first wave of interviews took place from October to December, 2012, with the wave 1 study being piloted from late September to mid-October, 2012.

Problem Gambling DVD Intervention

The Interprovincial Lottery Corporation (ILC) and the Ontario Problem Gambling Research Centre (OPGRC) are interested in conducting research to examine the link between self-exclusion and treatment. Dr. Scott Maitland is conducting a 30-month intervention project to develop and test an intervention video that encourages self-excluders to seek treatment. The SRC is involved in two phases of data collection; a Time 1 on-line survey to collect baseline measures of gambling behaviour, demographic questions, and assess motivation to seek treatment and perceptions of various treatments and resources, and a Time 2 on-line survey administered 6 months after the Time 1 survey to measure compliance with the self-exclusion agreement and to measure gambling behaviour and severity, as well as perception of the self-exclusion program and motivation and behaviour for seeking treatment.

Propel Break-It-Off Study

Break It Off (BIO) is an on-line social media smoking cessation campaign that launched in mid-January, 2012 across 5 Provinces; Saskatchewan, Manitoba, Ontario, Nova Scotia and PEI. The objective of the research was to collect feedback from BIO participants and to assess the impact of the on-line cessation program by comparing smoking behavior among BIO users versus non-users with the same age and smoking profiles. BIO Users and non-users were invited to complete on-line surveys. Data collection occurred from May, 2012 to January, 2013.

Back to the top

We are pleased to announce the appointment of Dr. Martin Cooke as a new Co-Director of the SRC. Dr. Cooke is currently jointly appointed to the Department of Sociology and Legal Studies and the School of Public Health and Health Systems at the University of Waterloo, and is an Affiliated Scientist with the Propel Centre for Population Health Impact. Along with teaching in the Masters of Public Health program, Dr. Cooke is also involved in several ongoing research projects in collaboration with colleagues across Canada. His research interests include: the social demography and health of Aboriginal peoples; social inequality, the welfare state, and the life course; and population ageing and retirement. Dr. Cooke graduated with an Honours BA from the University of Winnipeg in 1997, and received his MA and PhD in Sociology from the University of Western Ontario in 1999 and 2005, respectively. Dr. Cooke’s appointment at the SRC started on July 1, 2013. He joins fellow Co-Director Dr. Matthias Schonlau of the Department of Statistics and Actuarial Science.

Congratulations to Co-director Matthias Schonlau for being awarded a 2013 Insight Grant from the Social Sciences and Humanities Research Council. Matthias's five-year research project, entitled “Semi-automatic analysis of open-ended questions in surveys” was awarded $191,135 in funding. Traditionally, text answers have been hand-coded which makes their use expensive. Machine learning techniques can automate categorization but accuracy suffers relative to human accuracy. The goal is to develop a methodology that achieves accuracy comparable to humans using machine learning techniques only for text answers that are easy to categorize.

Call for research partners: The SRC will be conducting another wave of the Waterloo Region Area Survey in Fall 2013. If you are interested in participating, please contact [contact updated 2023: Tony Ly or call 519 888-4567 ext. 35071].

The SRC is saddened to announce that Kathleen McSpurren, Senior Manager is currently on medical leave. Kathleen is an integral member of the SRC team and has been greatly missed during her absence. Our thoughts and best wishes are with her.

Back to the top