Grant recipients:
Christopher Fletcher, Geography and Environmental Management and Scott Anderson, Centre for Teaching Excellence
(Project timeline: May 1, 2019 - April 30, 2020)
Project team:
Christopher Fletcher, Geography and Environmental Management; Scott Anderson, Centre for Teaching Excellence; Jessica Turecek, Norman Kearney and Maria Battaglia, School of Environment, Resources and Sustainability
Description
Data now pervades every facet of our education, work and social lives. The University of Waterloo is a strong proponent of “data literacy”, defined here as the ability to distill meaning from data, and to critically evaluate quantitative information from academic literature, research reports and other media. Data literacy is increasingly recognised as integral to applied research in the social sciences, as well as the natural sciences and engineering.
This project will help to promote statistical and data literacy among students in a second-year statistics and research methods course in the Faculty of Environment, many of whom do not possess a strong foundation in mathematics. The project will assess whether a series of innovative approaches to teaching and learning, including deployment of an online learning platform, a transition to project-based learning, and the ability to work with real-world data, can help to improve data literacy, student confidence, intellectual independence and interest in undertaking further research. The impact of these changes will be assessed using perception surveys of students and faculty, and quantitative data on students’ grades before and after implementation. This project also constitutes a first step toward the longer-term objective to fully transition the course to project-based learning, and to increase integration and alignment with the first-year prerequisite course.
Project Summary
An understanding of how to interpret data is critical to any modern post-secondary degree, and the University of Waterloo is a strong proponent of “data literacy”. However, a major pedagogical challenge for students in the Faculty of Environment is a lack of the mathematical foundations required for data science. This project asks whether implementing a new online learning platform in a second-year course on applied statistics for research helps Environment students to improve their data literacy, and whether it increases their interest in conducting research. A student survey found overwhelmingly that the learning platform did help them to learn the course content, and the more experience that the instructor had using the platform the more the students reported benefitting from it. The survey results also revealed strong relationships between final grades, gender, interest in research, and trust in the scientific method, which will help to inform future course development.
Research Questions
- How did students’ perceived data literacy change after taking the course?
- How much did the Pearson MyLab online learning platform help students to learn the course material?
- What is the relationship between students’ final grade, attitudes toward scientific method, and their interest in conducting research?
- What role do gender, final grade and confidence in statistics play in data literacy?
- To what extent are the instructor and Teaching Assistants important in helping students improve their data literacy and interest in research?
Findings/Insights
- Assessed over four terms (F19 to W21), mean perceived data literacy increased significantly after taking the course. However, the increase was not the same in every term: in F19 it actually decreased by about 5%, while in all three other terms it increased by 5-15%.
- On average, 83% of students agreed that “MyLab helped me to learn the course content more than I would have without it”, but again this result varied by term. In F19, only 65% agreed, while in W21 agreement was 97%, with monotonically increasing agreement over time. The students who perform better in the class (final grade 80+%) tend to agree much more strongly with this statement than those with final grades MyLab contributed moderately or more to their satisfaction in the class” than males (62%).
- Students who perform better in the class (final grade 90%+) are significantly more likely to disagree with the statement “The traditional scientific method is inherently bias-free" and report significantly lower trust in science and academia than those whose final grades are lower
- We find no significant difference between male- and female-identifying students in reported final level of data literacy, or in the change in data literacy after taking the course. However, males are much more likely to report greater confidence in their ability to complete a range of tasks related to statistics and data science, despite their final grades being lower on average (29% of females score 90+, compared to 20% of males).
- We found a highly significant difference in the reported final level of data literacy, and the change in data literacy, for female-identifying students in terms involving full or partial in-person instruction (F19 and W20), compared to the terms when instruction was fully online (F20 and W21). 73% of female students reported an increase in data literacy in-person, compared to 94% online. Interestingly, there was no comparable difference for male-identifying students, which appears to suggest the effect is not explained by overall lower satisfaction in F19.
- Female students taking the course in-person also reported consistently lower levels of confidence in numerous components of the course (understanding natural variability, interpreting and reporting results, cleaning data etc.) than those taking the course online. Female students who took the course online were also significantly less likely to fall in the lower tercile of final grades
- These interesting findings suggest female-identifying students may be benefitting substantially from learning statistics and data science in an online setting. While this observation is confounded by several important factors such as the changing familiarity of the instructor with MyLab, the general disruption to learning associated with the pandemic, and the varying quality of TAs in different terms, none of these factors would appear to offer a complete explanation for the findings, which leads us to conclude that we may have identified a real phenomenon that warrants further study.
- There is clear evidence that reported satisfaction with the TAs is associated with differences in numerous other variables: (i) perceived data literacy, (ii) reported change in data literacy, (iii) overall satisfaction with the course, (iv) perceived understanding of course concepts, and (v) reported ability to interpret and report the results of their own statistical analyses, which are all lower in F19 vs the three other terms. While causality cannot be established from this data, these numerical results combined with comments in student course evaluations leads us to conclude that TAs leading in-person tutorial sessions play a very important role in contributing to student learning and satisfaction in the course.