Critique: Review scores with a critical eye

Review SCP scores with a critical eye. Ensure that you understand what the mean score does and does not represent and how the score relates to the typical distribution of ratings for this tool.

Don’t over-rely on the mean score without incorporating other evidence and instruments.

The mean score provides information about the “typical score” for students’ perceptions of the quality of instruction for a specific course but does not provide a complete picture. More about how to interpret the mean score.

Consider the standard deviation.

Standard deviation indicates the variability of data—the degree to which SCP scores vary around the mean. A higher standard deviation means that there is high variability in the data. A lower standard deviation means that there is less variability in the data. A low standard deviation inspires more confidence that the mean represents the ‘typical case.’ Note: the standard deviation can also be affected by extreme outliers. More information about the standard deviation.

Pay attention to the distribution of ratings.

An unusually high or low score may require investigation but should not be considered in isolation. The mean is not the best measure to focus on because it is extremely sensitive to outliers. That is, a few low ratings can pull the mean downwards, which is particularly problematic for courses with a small number of students.

Pay attention to the response rate for a particular course.

Don’t assume that average score represents the collective experience of the entire class. Exercise caution, especially when there is a low response rate. Scores associated with higher response rates are a better reflection of the collective experience. The size of a course determines what should be considered to be an adequate response rate. See 'response rate necessary to have a ‘very precise,’ ‘precise,’ ‘somewhat precise,’ or ‘general estimate’ of the collective experience of students based on class size.'