SCSRU Newsletter Vol. 17, Issue #1 – Winter 2016

A common theme at this year’s American Association for Public Opinion Research (AAPOR) conference - a meeting place for survey researchers - was the still-increasing popularity of mobile devices used by respondents completing web surveys. Smartphones and tablets continue to be used in ever greater numbers, even when respondents are explicitly instructed not to use them. This is especially true for web surveys whose invitations are distributed by email, as emails are increasingly likely to initially be opened on a mobile device. A respondent who is unable to immediately complete the survey upon reading the email on their smartphone or tablet is unlikely to return to the survey and complete it on a laptop/desktop after having already read the invitation email. Therefore, it is now a virtual certainty that web survey data will contain completed surveys obtained from users of mobile devices, if it is at all possible for users to complete them on mobile devices. The possibility of mobile devices being used for web surveys affects both the question types that should be used and the overall design of the survey. What should researchers do to make surveys more mobile compatible?

Survey Length: Less is More

The total length of the survey should be reduced as much as possible. There are several factors why mobile users are more likely than computer users to abandon a long survey or speed through the survey at the expense of providing accurate answers Mobile users are conscious of data usage and battery life, they have to focus on a smaller screen and are often using their mobile device within an environment that contains more distractions.

Question Types: Choose Wisely

The limited screen size of smartphones means that measures must be taken to prevent respondents from needing to scroll in order to view an entire question and associate answer options. Lengthy lists (that would require vertical scrolling in order to have a legible text size) should be avoided, while horizontally-oriented answer scales need to be designed with limited screen width in mind. Grid-based questions, with a list running vertically down the left side with horizontally-arranged rows of radio buttons on the right, are likely to require both vertical and horizontal scrolling – the worst of both worlds. Even if only a slight amount of scrolling is needed, the possibility of obscuring answer options (for example the labels at the top of a grid question) means respondents may select incorrect choices.

Layouts: Make the Most of Screen Space

Reduce the size of logos, headers, and other branding and images to allow for more room for the survey content itself. Whenever possible, shorten question text to just the essentials. Respondent instructions should also be kept as concise as possible, lest the space occupied by detailed instructions cause more data issues than the total absence of instructions as the respondent is forced to scroll or resize in order to provide a response. Where grids are impractical for mobile users, consider converting them to a list of drop-downs. Break long lists into several pages rather than require respondents to scroll, and display only one question per page whenever possible to maximize screen space per question. To measure the impact of mobile-compatible design on your surveys closely monitor respondent device type and drop-out points where respondents are abandoning the survey to identify potential problem areas in the survey for mobile users. The experts at the Survey Research Centre provide questionnaire design, programming, hosting and top-line data analysis for web-based surveys. For more information on how we can help you with mobile-friendly web surveys, please contact us.

Back to the top

What did you think about your last hospital visit? How would you rate your experience overall? Each hospital in Ontario is legislated by the Ontario government to measure patient experience at least once a year. Grand River Hospital, a 571-bed hospital serving 700,000 people in the Waterloo Wellington Region of Southwestern Ontario, has gone one step further. They have partnered with the Survey Research Centre, University of Waterloo to develop a ‘best practice’ methodological approach to collecting patient experience data. The result? Collection of patient experience data that provides actionable measures for improvement of patient care programs.

Grand River Hospital identified a need to more fully understand their patients’ experience with hospital care in a more timely manner and to identify opportunities for improvement to support patient-centred care. Their previous approach to obtain patient experience feedback resulted in small sample sizes of a limited number of patient populations and the lag time between patient discharge and receipt of feedback resulted in significant time delays to action improvement. To better understand how different methods affect survey response rates and response ratings to questions, Grand River Hospital launched a pilot study over a three-month period with the Survey Research Centre, University of Waterloo to better measure the patient experience. The key focus of the pilot was to capture patient experience data in a more timely fashion to allow better patient experience recall, as well as a broader sampling of patients to ensure sufficient representation from 3 clinical programs and services.

Patient experience survey data was collected via two different methodological approaches:

1. On-site face- to- face surveys administered at Grand River Hospital for two inpatient clinical programs and;
2. A telephone survey administered by the Survey Research Centre for patients who experienced emergency department services

For both approaches, eligible patients being discharged were identified on a daily basis over a two-week period. All patients provided consent to be contacted at the point of admission to the hospital. Surveys were conducted at several points throughout a month period.

For the on-site phase, patients were approached at the hospital just prior to discharge and given the opportunity to complete a short survey with an interviewer from the Survey Research Centre. Patients who preferred not to complete the survey at the Hospital were given a copy of the survey that could be completed by the patient and returned to the Survey Research Centre.

For the telephone phase, Emergency patients were contacted by telephone shortly after discharge and administered the survey at home. The result? Through the pilot study, Grand River Hospital realized an increase in response rates for the non-emergency clinical departments from 32% for the traditional mail-out method to 91% for the on-site surveys conducted by the Survey Research Centre. Response rates also increased for the emergency department from 21% for the traditional mail-out method to 84% for the telephone surveys conducted by the Survey Research Centre.

Further to the pilot study, Grand River Hospital has maintained the post discharge telephone approach in the emergency department and expanded the face-to-face approach to six additional clinical programs. Grand River Hospital has been able to maintain a 70% response rate for the emergency department and a 95% survey response from on-site patients.

The ability to provide timely information from patients is resulting in Grand River Hospital’s ability to identify and put improvements in place across clinical programs, if necessary. Based on this work, staff has made improvements to provide better care. While in hospital, patients and families are more actively engaged in being involved in their care on a day to day basis. This has not only improved communication between patients/families and direct care providers, it has also improved safety. Staff has also worked to improve the discharge process. This has been achieved through the development of discharge teaching tools in collaboration with a patient’s family members. Use of this tool helps patients and family members who support them to better manage care after the patient leaves the hospital.

The Survey Research Centre, University of Waterloo is proud to assist the Grand River Hospital with this important research program. For more information on how the Survey Research Centre can help you to better understand your customers, please contact us.

Back to the top

Co-director Dr. Schonlau recently published two survey methodology papers. One paper investigated whether the quality of answers of respondents enrolled in web survey panel over time decreases. Specifically, the study investigated straightlining which refers to giving the same answer in a series of questions arranged on a grid. Straightlining is an indicator of satisficing, in which respondents give an answer that is “good enough” rather than giving the best possible answer. The paper, co-authored with Dr. Toepoel in the Netherlands, found that straightlining substantially increases for several years after respondents join the panel. This contributes to the growing body of work about survey quality of “professional” respondents.

In the second paper, Dr. Schonlau investigated a type of open-ended question that often appears near the end of surveys: “Do you have any other comment?” These so-called final comments are sometimes used for anecdotes but have never been systematically analysed. For the Dutch LISS and Immigrant panels, the study finds about half of the comments are neutral, about half of the comments are negative and only about 2% of the comments are positive. Neutral comments include respondents’ additional reflections about the topic of the survey and personal comments (including travel plans that might interfere with responding). Negative comments are of special interest because they might reveal problems with the questionnaire and we don’t want respondents to be unhappy. Many negative comments were about question difficulty or questions being unclear with fewer respondents complaining about the survey not being applicable to respondents’ situation, the survey being too long, and technical problems in answering.

Cynthia McLauchlan, a master’s student supervised by Dr. Schonlau, won an honourable mention for her paper “Are final comments in web panels predictive of attrition?” at the biannual meeting of the European Survey Research Association. Cynthia McLauchlan now works for Statistics Canada.

Schonlau, M., Toepoel V. Straightlining in Web survey panels over time. Survey Research Methods, August 2015, 9(2), 125-137. https://ojs.ub.uni-konstanz.de/srm/article/view/6128.

Schonlau, M. What do web survey panel respondents answer when asked “Do you have any other comment?” Survey Methods: Insights from the Field, August 2015. Retrieved from http://surveyinsights.org/?p=6899.

Back to the top

• Cancer Care Ontario: Cancer Care Ontario has developed a workplace health promotion program to improve the health of its employees. An on-line baseline survey was administered to employees to learn more about their health and their interest in workplace health programs. A follow-up online survey was carried out after one year of piloting a physical activity project to assess knowledge and behaviour change.
• SPAACE 2 SPAACE: SCAALAR and SPAACE to SPAACE were the first national population-based surveys on the prevalence, perceptions, and experiences of food allergies in Canada. Study findings informed the development of food allergy labeling legislation in Canada. There is general concern that food allergies are becoming an epidemic.  A follow-up telephone survey is being conducted to determine if any changes in prevalence, perceptions, and experiences have occurred over the past five years. Contact: Dr. Susan Elliott (GEM): env-allergy@uwaterloo.ca.
• UWaterloo Alumni Entrepreneur: The University of Waterloo is internationally recognized for producing successful entrepreneurs. To capture key information surrounding this phenomenon, approximately 100,000 UWaterloo alumni were recently invited to complete an online survey. The study is expected to provide important insights into the entrepreneurship and innovation characteristics of UWaterloo graduates, including prevalence rates of entrepreneurship, rates of new venture creation and the role UWaterloo plays in encouraging entrepreneurship and innovation.

Back to the top

• SRC co-director Matthias Schonlau has been elected President of the Survey Methods section of the Statistical Society of Canada (SSC). He will serve a year as president elect (2015-2016) followed by a year as president (2016-2017).
• Cynthia McLauchlan, a master’s student supervised by Dr. Schonlau, won an honourable mention for her paper “Are final comments in web panels predictive of attrition?” at the biannual meeting of the European Survey Research Association. Cynthia McLauchlan now works for Statistics Canada.
• Dr. Schonlau recently published two survey methodology papers. One paper investigated whether the quality of answers of respondents enrolled in web survey panel over time decreases. In the second paper Dr. Schonlau investigated a type of open-ended question that often appears near the end of surveys: “Do you have any other comment?”.
• SRC Project Manager Lindsey Webster went on maternity leave last May and will be returning to the SRC this July. In the meantime, the SRC would like to welcome Beth McLay, interim Project Manager and Arianne Manary, interim Project Support and Administration.

Back to the top