The purpose of this guide is to help academic administrators at Waterloo interpret scores collected through the Student Course Perceptions (SCP) survey, specifically with respect to understanding the contextual factors that may impact those scores. It will be updated and informed by ongoing testing and monitoring of the SCP process, discussions with advocacy groups at Waterloo, input from department Chairs and others acting in an administrative role, experiences and reports shared by other Canadian institutions, and continued review and analysis of the literature. Read about what to expect from the SCP data report.
Reviewing the SCP data report
The results of the SCP survey revolve around numbers: a five-point Likert scale rating system, means and standard deviations, charts, and tables. However, effective use of this tool requires a less tangible sense of how to interpret those numbers—requiring attention to context, an understanding of instructors’ concerns, and the awareness to recognize when bias may be at play.
Chairs and other academic administrators need to be mindful when reviewing the results of student course perception surveys, to ensure that these biases do not unfairly impact instructors undergoing review. It is imperative that academic administrators take potential bias into account when interpreting the results of SCP surveys. The following steps are critical to reviewing SCP results in a way that optimizes efficacy and equity. In practical terms, these steps will intersect and overlap as academic administrators work through SCP results.
Contextualize
- Empirical research conducted at Waterloo and beyond indicates that SCP scores can be affected by variables that do not reflect teaching effectiveness.
Critique
Review SCP scores with a critical eye.
- Don't over-rely on the mean scores without incorporating other evidence
- Consider the standard deviation
- Pay attention to the distribution of ratings
- Think about the response rate
Compare
- Take a closer look at any scores that stand out as unusual
- Look for patterns
- Pay attention to scores that are higher or lower than expect, or that reflect unusual patterns over time or between courses
- Refer to several SCP scores over time, when using them to inform decisions
- Avoid focusing on small differences that are not meaningful
Communicate
The Teaching Assessment Processes team offers support for interpreting scores, ongoing monitoring of survey results, data analysis to support further investigation.