Notice: The Statistical Consulting and Survey Research Unit was previously called the Survey Research Centre. Our unit is referred to by this name in old newsletters.
Survey Rating Scales – Do’s and Don’ts
One of the more common question formats used when developing a survey is the rating scale question. A rating scale uses a set of categories designed to elicit information about a quantitative or qualitative attribute and is often used to measure attitudes, opinions and behaviours.
Examples of rating scales are:
|
Unipolar Rating Scales |
|
“How satisfied were you with services you received today?”
|
|
A unipolar scale prompts a respondent to think of the presence or absence of a quality or attribute. |
|
Bipolar Rating Scales |
|
“Do you exercise more or less compared to 5 years ago?”
|
|
A bipolar scale prompts a respondent to balance two opposite qualities or attributes. |
Good structure of rating scale questions is important as it can improve the quality of the data collected. According to research presented at the 2016 American Association for Public Opinion Research (AAPOR) conference, there are several dos and don’ts to keep in mind when designing rating scale questions:
Do cover the entire measurement continuum
“How often do you eat out?”
- A lot of the time
- Some of the time
- Only a few times
In this example, not all the possible answer options are covered. “Always” and “Never” answer options are missing.
Don’t use overlapping points
“What is your age?”
- 0-20 years
- 20-40 years
- 40- 60 years
- 60 years or over
These answer options are not mutually exclusive. If your age is 20, 40, or 60, two answer options could be selected. A better way to present the options is 0-19, 20-39, 40-59, 60 years and older.
Don’t use unbalanced scales
“How would you rate your flight experience today?”
- Poor
- Fair
- Good
- Very Good
- Excellent
This is an unbalanced scale as there are more positive points than negative points. A better way to present the options is Very poor, Poor, Fair, Good, Excellent.
Do be specific.
“How would you rate your overall stress level?”
- Very high
- High
- Medium
- Low
- Very low
“Stress” to one person may not be the same to another depending on what the respondent is using as a comparison (e.g.,physical stress, mental stress, emotional stress?) Over what period of time is the stress level being measured (e.g., today, over the past year?). Be more specific to ensure all respondents and the researcher share the same meaning of the question. “How would you rate your mental stress level over the past month?"
Do use the optimal length of scale.
The scale should be long enough that it covers the entire measurement continuum but short enough that the meanings of the scale points are unique. Research indicates that for data validity, the optimal length for unipolar scales is 5 points and bipolar scales is 5-7 points. The survey mode is also a factor. For example, surveys that will be conducted mainly on smart phones should use shorter scales to minimize screen scrolling.
Do label scale points.
“Rate the performance of your child’s teacher”
Very poor 2 3 4 Excellent
It is not evident what is meant by the numbers. Does “3” mean “Fair” or does it mean “Unsure”? Could it mean “Good”? Fully labelled scale points (without numbers) with clear, unambiguous words provide a consistent understanding of the meaning of the scale point for both the respondents and the researchers. A better way to present the options is:
Very poor Poor Fair Good Excellent
These are just a few of the insights to consider when designing survey rating scale questions. The experts at the Survey Research Centre at the University of Waterloo provide survey design, programming, hosting and top-line data analysis for web, telephone, mail and face-to-face surveys. For more information on how we can help you with survey design questions, please contact us.
LGBT Retirement Study
“Not only is the population changing, but what retirement looks like is not going to be the same as a one-time transition out of paid work. It’s going to be more complex.” says Dr. Steven Mock, assistant professor in the Department of Recreation and Leisure Studies at the University of Waterloo.
People are working longer or transitioning slowly from full-time work to part-time work prior to retirement. Others are returning to the work force after being retired. The reasons why may vary, but one thing is sure, factors such as marital status and sexual orientation can greatly influence the retirement planning process.
Dr. Steven Mock’s research included administration of an online survey hosted by the Survey Research Centre, University of Waterloo, to collect information about retirement planning among adults age 30 or older. To be eligible to complete the survey, it was not necessary for participants to identify as a sexual minority, but participants were asked this question as part of the survey. The online survey was hosted by the Survey Research Centre for nearly 6 months. Participants were recruited from a number of different sampling sources including a referral process that involved e-mail communication being sent by participating agencies to their respective employees, additional referral sample through employee resource groups and community groups, as well as recruitment through two online panel firms.
A total of 550 surveys were completed across the various sample sources. Some of the key findings included that the more lesbian, gay and bisexual (LBG) individuals felt supported and the more “out” they were, the less uncertain they were about timing of retirement and the more secure they felt about retirement finances. Similarly, the more supported LGB adults felt and the more open they were about their sexual orientation, the more they sought retirement advice from friends and family.
In sum, although some similarities between sexual minorities and the general population were found in terms of perceptions of retirement planning a few unique issues were found. Namely, for LGB adults, feeling supported appears to have a particularly strong impact on the retirement planning and being “out” seems to facilitate planning and conversations about planning.
The Survey Research Centre, University of Waterloo is proud to assist Dr. Mock with this important research. For more information on how the Survey Research Centre can help you to better understand populations of interest, please contact us.
Featured Projects
- Healthy Weights: The Healthy Weights Connection is an intervention aimed at promoting healthy weights and preventing obesity among Aboriginal children in Canada. The theory of the intervention is that by promoting better cooperation and resource sharing among the various organizations whose activities may affect the risk of obesity among Aboriginal children, local public health systems will be better able to provide culturally-appropriate programming. The intervention consists of providing a community health worker to work at building opportunities for collaboration among local Aboriginal-specific and “mainstream” organizations, as well as a website, newsletters, and other knowledge-exchange activities. An online survey of these organizations was administered as part of the project evaluation plan to measure changes in the degree to which institutions in the local public health system serve Aboriginal children and youth in the community, and the amount of collaboration and coordination among them. Data collection for the online organization survey occurred in 2014. An online follow-up survey is slated for late 2016.
- The Region of Peel Tobacco Study: The Region of Peel was interested in exploring key themes related to tobacco use and to address identified gaps in adult males’ knowledge related to tobacco use and cessation. This two-phase project included the administration of a quantitative online survey, followed by an in-depth examination of the survey results through the administration of focus groups and one-on-one telephone interviews with purposeful samples of adult male smokers. All focus group participants and those doing the one-on-one telephone interviews completed the quantitative survey prior to participating in the follow-up discussion. The SRC partnered with Strategic Science Inc. on this project. The SRC was responsible for administration of and data analysis for the online survey and scheduling focus group participants. Strategic Science Inc. was responsible for conduct and analysis for the follow-up discussions. The web survey was launched in 2014 and remained open for nearly 5 months, closing in March 2015.
- UWaterloo Graduate Studies Admission Survey: To assist with recruitment and marketing of graduate studies programs, the University of Waterloo Graduate Studies Office has developed an on-line survey for students who accept or decline offers of admission to the UWaterloo On-site or Online graduate programs. First conducted in 2014, this survey is being conducted annually over a five-year time period.
Waterloo Region Area Survey
Announcements
- We are very pleased to announce the appointment of SRC Interim Co-Director, Christian Boudreau, PhD. Dr. Boudreau has been a faculty member in the Department of Statistics & Actuarial Science at the University of Waterloo and the Associate Director of the Data Management Core of the International Tobacco Control Policy Evaluation (ITC) Project since 2005. The ITC Project conducts longitudinal surveys amongst smokers and non-smokers in more than 25 countries. Dr. Boudreau thus brings extensive knowledge and expertise on applied survey sampling and survey methodology. In addition to surveys, his research interests include longitudinal studies, competing risks and survival analysis.
- Dr. Mary Thompson, Associate Director of the Survey Research Centre and Distinguished Professor Emerita at the University of Waterloo, has received two honorary doctorates in 2016. The first doctorate is from Vancouver Island University where Dr. Thompson was honoured for her work with social and health scientists to evaluate government policies and measures that help prevent illnesses and early deaths. The second doctorate is from Western University. Dr. Thompson was honoured for her pioneering and foundation work on the theory of estimating functions, survey methodology, biostatistics and important interdisciplinary collaborative research.
- We are very happy to welcome back SRC Project Manager Lindsey Webster, who has returned from maternity leave. She has resumed her duties as project manager and overseeing the telephone call centre.