Image used under the Pexels license.
Grant recipients:
Michael Wallace, Statistics and Actuarial Science
Henry So, Statistics and Actuarial Science
(Project timeline: September 2018-August 2019)
Description
We will study flexible assessment and metacognition in an entry-level statistics course. Flexible assessment will involve a grading scheme where students may choose whether to complete some assessments. Though all are incentivized, some examinations and assignments need not be completed to attain full marks in the course. Through follow-up questionnaires we will learn how students decide which assessments to complete, whether they perceive having that choice as beneficial to their learning, and encourage them to engage directly in metacognition by considering how they approach assessment-related decisions. This will foster deeper learning both within this class and across the broader Waterloo curriculum, with students benefiting from enhanced metacognitive skills while other instructors are aided by the study results. Finally, the study itself will motivate some aspects of the course material by exposing students to experimental and survey design in a more tangible manner, further encouraging engagement and understanding.
Research Questions
By implementing flexible assessment into STAT 231, we intended to give students greater autonomy in how they approach their learning, encourage students to think about their assessment decisions and how this affects their approach to course material, and alleviate some of the stresses (and concomitant pressures on mental wellness) that can result from overburdened assessment structures. With these goals in mind, we investigated the following questions:
- What factors lead students to succeed in this course?
- Do students think that flexible assessment helps them?
- What factors impact student decisions about assessments?
Do students engage differently with the course because of flexible assessment, and if so, how?
Findings/Insights
We found that the amount of time students spent preparing for tests and completing homework had no relation to their test scores and homework grades respectively. On the other hand, the number of assessments students elected to complete throughout the course was associated with their final grade.
Do students think that flexible assessment helps them?
Yes: Students reported that both weight transfer of assessment and a dropped-homework marking scheme were beneficial to their academic performance in STAT 231, beneficial to their academic performance in other courses, and beneficial to their general well-being, including mental and physical health:
|
Survey 1 Mean (1-5 scale) |
Survey 2 Mean (1-5 scale) |
|
|
Weight Transfer: STAT 231 |
4.21 |
4.38 |
|
Weight Transfer: Other courses |
4.22 |
4.42 |
|
Weight Transfer: Well-being |
4.37 |
4.49 |
|
Dropped-HW: STAT 231 |
4.37 |
4.45 |
|
Dropped-HW: Other courses |
4.29 |
4.43 |
|
Dropped-HW: Well-being |
4.34 |
4.44 |
Notably, there is an increase in the perceived benefit of weight transfer as students progress throughout the course, especially with regards to students’ performance in STAT 231 and performance in other courses. These increases are unlikely to be attributable to random variation in responses, instead providing evidence in favour of a genuine change in perceptions as the course progressed.
What factors impact student decisions about assessments?
By qualitatively coding student responses to why they took each written test, we found that most students’ decisions about whether to write assessments came down to one of a small number of factors, which we summarise below, including some ‘example responses’ (which may not reflect the full range of responses within a given factor, and are presented for illustration). We also provide the number of responses coded this way; note that some responses had more than one code.
|
Factor |
Why did students write tests? |
# (%) |
Why did students not write tests? |
# (%) |
|
Well-being |
To avoid stress/pressure later |
33 (3.2%) |
To avoid stress/pressure now, or health issues |
28 (18.7%) |
|
Difficulty |
Thinks a test will be easy |
34 (3.3%) |
Thinks a test will be hard |
1 (0.7%) |
|
Force |
To avoid receiving a zero mark for a test |
48 (4.7%) |
N/A |
0 (0%) |
|
Habit |
“I don’t ever skip exams”, “It’s my responsibility to” |
79 (7.7%) |
N/A |
0 (0%) |
|
Indifference |
“It can’t hurt my grade” |
104 (10.1%) |
“I don’t really need to…” |
4 (2.7%) |
|
Knowledge |
To test knowledge or review the material |
253 (24.7%) |
“I haven’t caught up” |
10 (6.7%) |
|
Marks |
To get the best mark possible |
328 (32.0%) |
“I did good enough on a previous test” |
11 (7.3%) |
|
Study |
“I felt prepared”, “I studied well”, to practice for final |
195 (19.0%) |
“I wasn’t prepared enough” |
52 (34.7%) |
|
Time |
Has more time to study now/less time to study later |
30 (2.9%) |
Has less time to study now, has a time conflict |
90 (60.0%) |
|
Weight transfer |
Doesn’t want to depend on weight transfer |
111 (10.8%) |
Was willing to transfer weight |
18 (12.0%) |
|
Other |
None of the above |
40 (3.9%) |
None of the above |
6 (4.0%) |
We found no evidence that the reason a student takes a test has an impact on their test grade.
Do students engage differently with the course because of flexible assessment, and if so, how?
By qualitatively coding student responses to whether they engaged with the material in the course as a whole, we found that by the end of the course, an increasingly large number of students perceived that they engaged differently with the course material because of flexible assessment (in particular, weight transfer):
|
Survey 1 |
Survey 2 |
|
|
Engaged differently |
106 (36.2%) |
168 (54.7%) |
|
Did not engage differently |
139 (47.4%) |
78 (25.4%) |
|
Unclear |
48 (16.4%) |
61 (19.9%) |
For students who engaged differently because of flexible assessment, their reasons as to why were generally one of several things, described below:
|
Reason |
How did student engage differently? |
# (%) |
|
Well-being |
Student felt less stress, pressure, or maintained better health |
157 (26.2%) |
|
Content |
Student focused on the content (over assessment marks) |
17 (2.8%) |
|
Distribution |
Student better managed their overall course load |
57 (9.5%) |
|
Effort |
Student “slacked off”, procrastinated, studied less, or studied less effectively |
72 (12.0%) |
|
Plan |
Student felt they could plan better with a “second chance” |
52 (8.7%) |
|
Study |
Student studied more, or more effectively |
31 (5.2%) |
|
Time |
Student more effectively managed their time in STAT 231 |
37 6.2%) |
(Note that some responses had more than one code, some did not receive a code, and some students who reported they did not engage differently received a code due to the content of their response)