SCSRU Newsletter Vol. 16, Issue #1 – Fall 2014

By: Lindsey Webster

The 2014 AAPOR conference was held in Anaheim, California, the home of Disneyland and its sister theme parks. The conference theme was “Measurement and the Role of Public Opinion in a Democracy”. Many of the sessions touched on some aspect of this year’s theme, for example: whether public opinion has a negative or a positive effect on policy makers, or if legislators and other leaders should take public opinion into account in decision making. The conference also featured presentations discussing the power of social media, including how they have increased the opportunity for the public to express their opinion and in turn, how this has affected the world. There were also sessions on technology, new ways of capturing data, and measuring survey usability. I enjoyed an interesting session about the use of eye tracking devices to examine the visual design of web surveys. The sessions about data collection through mobile devices were particularly relevant as this is something that the Survey Research Centre will be exploring in the near future.

The short course that I attended this year was titled “Going Mobile with Survey Research”. With 91% of all adults in the U.S. owning a cell phone, 52% owning a smart phone, and 42% owning a tablet, the need to explore data collection on mobile devices is paramount. The course began by distinguishing the differences between cell phones and smart phones, and between tablets and PC’s. Screen size, internet access, and the functionality of these devices were presented. It is also important to understand the owners of the devices and the session compared the profiles of average users by age, race, income and urbanicity. Best practices were discussed including how respondent burden should play a part in the design of surveys. Typically, respondent burden is measured in terms of the length or administration time of the survey. Respondent burden in a mobile world is somewhat different. Respondents aren’t just responding at home anymore, and could be called while doing a variety of important activities. Data consumption for both receiving and sending information can be costly depending on the respondent’s data plan. Slower bandwidth increases time required to send and receive data. Battery life is also a factor where persistent GPS capture or extra http requests can drain mobile device power. Practices for reducing respondent burden have changed compared to the past when respondents only used home phones and PC’s to complete surveys. The ease of use, speed of the survey tool, respondent privacy, and visual appeal are factors that now need to be more closely considered. Lastly, the session explored how to design surveys for mobile devices.

This was the first year that I attended the Association of Academic Survey Research Organizations (AASRO) luncheon. There was a fantastic presentation done by one of the members on the challenges of surveying in a multi-device world. It was also a great opportunity to meet professionals from other academic survey data collection centres.

The 2014 conference was definitely memorable. It was a fantastic learning experience once again and a great opportunity to be part of the yearly gathering of professionals in public opinion research. I look forward to the next conference and am eager to learn what the theme will be!

Back to the top

By: Sascha Lecours

Navigating the internet with a serious visual impairment can be a tremendous challenge. Using only screen-reading software to access the content of a webpage can be difficult even with a website designed to be compatible with accessibility software – and it is nearly impossible if the website was not designed with this compatibility in mind.

To that end, the University of Waterloo has committed to reaching level “A” compatibility with the World Wide Web Consortium Web Content Accessibility Guidelines (WCAG) 2.0 as of January, 2014. Level “A” compatibility means implementing web design practices that may have little effect for a visitor not using accessibility software, but will have a great benefit for those who do. This change applies to nearly all web content put forward by the University of Waterloo, including the SRC’s web surveys. Programming accessible web surveys entails its own set of unique challenges compared to programming accessible websites in general.

Choosing the Right Tool for the Job

Web survey software generates much of the “under-the-hood” web content automatically, including the HTML/CSS code which controls the general layout of a web page. When the software structures the page in a way that conflicts with web accessibility guidelines, this can require additional steps on the part of the survey programmer.

Unfortunately, the built-in code used by survey software often fails to meet accessibility guidelines. Common offenses are the use of invisible “table” cells to control the overall layout of a website (rather than present tabular data), or the failure to properly label “header” cells in legitimate tables, which means that screen readers are not provided with the information needed to give context to table cells by listing the contents of the header cell for a given row or column. These issues can be tackled by using custom HTML files to lay out survey pages, rather than program defaults. This means that programmers need to write the “under-the-hood” elements of the survey pages themselves using HTML and CSS, and implement the proper measures to make sure the survey is compatible with accessibility programs such as screen readers. These measures include:

• ensuring that “tables” are used only to present tabular data (or tabular questions/answer options) with properly labelled column and row headers (used by screen readers to give context to table cells which might contain only a single radio button otherwise),
• ensuring that radio and checkbox buttons are accompanied by hidden “label” information that allows screen reader software to identify the answer corresponding to the button,
• making sure that the survey can be navigated using a keyboard alone (i.e. no mouse required),
• ensuring that repetitive “header” text or images can be “skipped over” when a screen reader presents the page by including an invisible “skip link” that can be used to jump the user to the question text,
• ensuring that all images present are accompanied by descriptive “alt text” that provides the information or context conveyed by the original image, and
• including full transcripts for videos and audio elements.

Another important practice is the use of semantic tags to identify text. Rather than using HTML to tag text as “italicized”, it should be tagged as “emphasized” to convey the intended semantic meaning of the formatting choice, not merely the visual effect on the text. This also includes proper HTML-labelling of headers and titles, and other text formatting. Screen reading software can use properly labelled headers, tables, and lists to orient the user quickly with the content of the page.

Unfortunately, many of the practices above are not present by default with survey design software that was originally developed when accessibility was not a priority for those hosting web surveys. This seems to be changing as software providers are developing programs geared towards the creation of more accessible surveys; an example is the upcoming “Acuity4” survey design tool from Voxco. Until these tools become more widely available, programmers will need to use knowledge of HTML and CSS to ensure that their survey pages conform to accessibility guidelines.

Beyond Programming - Survey Design Choices

Some challenges in accessible survey design stem from issues other than software or programming - they are embedded in the fundamental design of the survey itself. Some examples include:

• Unnecessary use of images to convey information. Images must be summarized using explanatory “alt text” for users unable to view the images themselves, which can lead to differences of content or interpretation between users viewing the images and those using the text only.
• Use of colour alone to express emphasis or stress differences between words, rather than semantic markup, which can be understood more clearly with a screen reader.
• Use of colors with insufficient visual contrast. For example, using light-coloured text on a white background may not meet accessibility criteria for minimum contrast between text and background.

More fundamental issues include the use of questions laid out in a tabular format. This type of question format can be particularly difficult and tedious to navigate using web accessibility software compared to more conventional question layouts such as open-ended questions or a single enumerated scale with radio buttons. This is true to some degree even if the tabular questions do technically adhere to accessibility guidelines by including proper labels and headers. When designing a survey to be accessible, avoiding unnecessary use of the “table” format for questions will make web content easier to navigate for those using screen readers.

The Future is Accessible

The future for accessible web survey design is bright. Software providers know that accessibility is a necessary feature for their clients, and soon it will be the norm that web software packages allow for automatic generation of surveys that follow web accessibility guidelines. Until then, web survey programmers must expand their skill sets and knowledge to ensure that their web survey content is accessible to all, as it should be.

Back to the top

The Waterloo Region Area Survey is a community survey conducted among Waterloo Region residents, aged 18 or older. This survey is a unique opportunity for local organizations and government agencies to collect high quality survey data for a fraction of the cost of a full-fledged telephone survey. Each year, interested research partners buy survey space according to their needs and objectives. The Survey Research Centre asks a standard set of demographic questions every year and all research partners receive the data for the demographic questions in addition to the data for their survey questions.

From August to early September, 2014, the Survey Research Centre the 2014 Waterloo Region Area Survey. The Survey Research Centre has conducted this study since 1998. The study originally started as a paper and pencil survey that was mailed to respondents’ homes and was administered every two years. Since 2011 the study has been conducted as a telephone survey, and has been administered every year. In 2014 the Survey Research Centre added a web component to the survey. In 2012 and 2013, telephone respondents were asked if they would be interested in participating in the study again, if the survey were offered on-line. Respondents who indicated interest in participating in the web survey were contacted via e-mail for the 2014 wave and asked to complete the survey. Administration of the 2014 web survey ran concurrently with the telephone data collection. For the 2014 wave, 400 telephone surveys and 131 web surveys were completed. The response rate for the web survey exceeded our expectations, with 33.7% of those invited to participate, actually completing the on-line survey.

The Survey Research Centre plans to continue running the Waterloo Region Area Survey on an annual basis, as a mixed-mode, telephone and web survey. The SRC plans to continue to supplement the number of possible web participants with every subsequent iteration of the survey. The next wave of data collection is planned for summer 2015. The SRC will circulate an initial call for research partners via email in January 2015. If you would like to receive this notification, or if you have any questions about the Waterloo Region Area Survey, please do not hesitate to contact: [contact updated 2023: Tony Ly or call 519 888-4567 ext. 35071].

Back to the top

Centre for Addiction and Mental Health - Smoking Research Study Survey

The purpose of this research is to determine the efficacy of smoking cessation programs that involve mass distribution of Nicotine Replacement Therapy. The study is designed as a single-blinded panel survey with random assignment to an experimental and control condition and involves two stages of recruitment and three telephone interviews. Households across Canada are contacted using a Random Digit Dialing approach. Eligible respondents over the age of 18 are screened and recruited to either an experimental or control group, with those in the experimental group receiving a free 5 week supply of nicotine patches. Eligible respondents in both groups are also asked to provide a saliva sample. Follow-up interviews are then scheduled to take place 8 weeks and 6 months after completing the initial survey.

Cancer Care Ontario (CCO) Workplace Health Employee Survey

CCO is developing a Comprehensive Workplace Health Promotion (CWHP) program to improve the health of its employees. As part of a situational assessment, an on-line baseline survey was administered to employees to learn more about their health and their interest in various workplace health programs. To assess changes in knowledge and behaviour, a follow-up online survey is planned for one year after the implementation of the first CWHP project - a physical activity pilot project.

CEMC TAM and CSG Workshop Evaluation

The purpose of the study is to evaluate and monitor the Centre for Education in Math and Computers (CEMC) Think About Math (TAM) and Computer Science For Young Women (CSG) outreach programs offered each May and June to high school aged girls. The surveys obtain a snapshot of current school courses taken and involvement in enrichment programs, as well as measure current attitudes about math and computers, and assess the individual elements of the outreach programs themselves.

Fall Prevention Study, University of Guelph

An eight month campaign running from October 2013 through May 2014 focused on fall prevention among children aged 1-4. The eight month campaign consisted of a social marketing campaign as well as two additional interventions. Effectiveness of the campaign and the interventions is being assessed by conducting telephone interviews across six centres. Three centres were exposed to the social marketing campaign only. One of the centres was exposed to the social marketing campaign and the two additional interventions. The last two centres are the control group and was not exposed to either the social marketing campaign or the additional interventions. Two waves of data collection are planned; a pre-wave prior to the launch of the social marketing campaign and interventions. The prewave was conducted from June to September, 2013. And a post-wave conducted one year later, from June to September, 2014.

Grand River Hospital Patient Experience Pilot

Grand River Hospital (GRH) has identified a need to more fully understand their patients’ experience with hospital care. They are interested in developing a ‘best practice’ methodological approach to collecting patient experience data that produces actionable results. GRH currently participates in a province-wide patient experience survey; however there is a lack of full coverage to all clinical programs and services. A separate pilot study with data collection conducted by the SRC was conducted to assess whether a different methodological approach would produce better outcomes. Three patient groups were considered for the pilot study: emergency, surgery inpatient, and medicine. Surveys were conducted by telephone with emergency patients shortly after discharge from the hospital, and on-site surveys were administered with surgery inpatient and medicine patients prior to their discharge from the hospital.

Healthy Weights Organization Survey

The Healthy Weights Connection is an intervention aimed at promoting healthy weights and preventing obesity among Aboriginal children in Canada. The theory of the intervention is that by promoting better cooperation and resource sharing among the various organizations whose activities may affect the risk of obesity among Aboriginal children, local public health systems will be better able to provide culturally-appropriate programming. The intervention consists of providing a community health worker (“site coordinator”), to work at building opportunities for collaboration among local Aboriginal-specific and “mainstream” organizations, as well as a website, newsletters, and other knowledge-exchange activities. The organization survey is being administered as part of the project evaluation plan and will attempt to measure changes in the degree to which institutions in the local public health system serve Aboriginal children and youth in the community, and the amount of collaboration and coordination among them.

Higher Education Quality Council of Ontario (HEQCO) Web Survey

The HEQCO has implemented an internal process for projects that consists of providing project teams with update meetings, professional development for graduate students, and support with report writing. The HEQCO was interested in creating an on-line survey to collect feedback from project teams in order to evaluate the effectiveness of this internal process.

International Tobacco Control 4C9 (UK, Australia, Canada and US) and 4C10 (UK and Australia only)

The ITC Project is a multi-disciplinary collaboration of more than 80 researchers in 22 countries who evaluate the effects of national tobacco control policies on smoking behavior. Its principal funding comes from the NIH and CIHR. Dr. Geoffrey Fong of UW Psychology is the Principal Investigator. The recontact and replenishment web surveys for the 4C9 countries are conducted by the SRC, with the US Replenishment conducted in late 2014/early 2015. The 4C10 UK and Australia Re-contact data collection launched in August, 2014, with replenishment data collection for these two countries starting in October, 2014.

Problem Gambling DVD Intervention

The Interprovincial Lottery Corporation (ILC) and the Ontario Problem Gambling Research Centre (OPGRC) are interested in conducting research to examine the link between self-exclusion and treatment. Dr. Scott Maitland is conducting a 30-month intervention project to develop and test an intervention video that encourages self-excluders to seek treatment. The SRC is involved in two phases of data collection; a Time 1 on-line survey to collect baseline measures of gambling behaviour, demographic questions, and assess motivation to seek treatment and perceptions of various treatments and resources, and a Time 2 on-line survey administered 6 months after the Time 1 survey to measure compliance with the self-exclusion agreement and to measure gambling behaviour and severity, as well as perception of the self-exclusion program and motivation and behaviour for seeking treatment.

Retirement Planning of Sexual Minority Adults Study

Understanding the nature of retirement planning is increasingly important due to an aging Canadian population. As families become increasingly diverse to include families of lesbian, gay, bisexual, transgender and two-spirit (LGBT) adults, researchers have begun to examine the retirement planning experiences of LGBT adults. The SRC assisted with administering an on-line survey to collect data to examine the nature of retirement planning among LGBT adults and determine potential challenges faced.

Back to the top

The SRC is pleased to announce that Dr. Martin Cooke was the recipient of the Angus Reid Practitioners/Applied Sociology Award issued by the Canadian Sociology Association. The award has been created to recognize sociologists who undertake community action projects that bring social science knowledge, methods, and expertise to bear in addressing community-identified issues and concerns and celebrates a Sociologists’ contribution to sociological practice that has served as a model for working with a community, organization or public service. In addition to being Co-Director at the SRC, Dr. Cooke works with both the Department of Sociology and Legal Studies and the School of Public Health and Health Systems at the University of Waterloo.

Congratulations to Co-director Matthias Schonlau for being awarded an Insight Development Grant from the Social Sciences and Humanities Research Council. Matthias's two-year research project is entitled “Final respondent comments as a diagnostic tool in web surveys”. Respondents’ final comments at the end of a survey (e.g., "Do you have any other comments on the interview?") are routinely collected but typically ignored. Such final comments are a potentially rich source of information about respondents’ experience and interaction with the survey process. The aim of the project is to learn what respondents are trying to communicate and to find out whether this information is predictive of attrition and survey quality metrics.

The SRC staff was deeply saddened by Kathleen McSpurren’s passing. Kathleen passed away on July 29, 2013 after a hard fought battle with cancer. She is greatly missed at the Centre.

Back to the top