Flexible Assessment and Metacognition in an Entry-Level Statistics Course

Grant recipients:

Michael Wallace, Statistics and Actuarial Science

Henry So, Statistics and Actuarial Science

(Project timeline: September 2018-August 2019)

Description

We will study flexible assessment and metacognition in an entry-level statistics course. Flexible assessment will involve a grading scheme where students may choose whether to complete some assessments. Though all are incentivized, some examinations and assignments need not be completed to attain full marks in the course. Through follow-up questionnaires we will learn how students decide which assessments to complete, whether they perceive having that choice as beneficial to their learning, and encourage them to engage directly in metacognition by considering how they approach assessment-related decisions. This will foster deeper learning both within this class and across the broader Waterloo curriculum, with students benefiting from enhanced metacognitive skills while other instructors are aided by the study results. Finally, the study itself will motivate some aspects of the course material by exposing students to experimental and survey design in a more tangible manner, further encouraging engagement and understanding.

Research Questions

By implementing flexible assessment into STAT 231, we intended to give students greater autonomy in how they approach their learning, encourage students to think about their assessment decisions and how this affects their approach to course material, and alleviate some of the stresses (and concomitant pressures on mental wellness) that can result from overburdened assessment structures. With these goals in mind, we investigated the following questions:

  • What factors lead students to succeed in this course?
  • Do students think that flexible assessment helps them?
  • What factors impact student decisions about assessments?

Do students engage differently with the course because of flexible assessment, and if so, how?

Findings/Insights

We found that the amount of time students spent preparing for tests and completing homework had no relation to their test scores and homework grades respectively. On the other hand, the number of assessments students elected to complete throughout the course was associated with their final grade.

Graphical representation of results

Do students think that flexible assessment helps them?

Yes: Students reported that both weight transfer of assessment and a dropped-homework marking scheme were beneficial to their academic performance in STAT 231, beneficial to their academic performance in other courses, and beneficial to their general well-being, including mental and physical health:

Survey 1 Mean (1-5 scale)

Survey 2 Mean (1-5 scale)

Weight Transfer: STAT 231

4.21

4.38

Weight Transfer: Other courses

4.22

4.42

Weight Transfer: Well-being

4.37

4.49

Dropped-HW: STAT 231

4.37

4.45

Dropped-HW: Other courses

4.29

4.43

Dropped-HW: Well-being

4.34

4.44

Notably, there is an increase in the perceived benefit of weight transfer as students progress throughout the course, especially with regards to students’ performance in STAT 231 and performance in other courses. These increases are unlikely to be attributable to random variation in responses, instead providing evidence in favour of a genuine change in perceptions as the course progressed.

What factors impact student decisions about assessments?

By qualitatively coding student responses to why they took each written test, we found that most students’ decisions about whether to write assessments came down to one of a small number of factors, which we summarise below, including some ‘example responses’ (which may not reflect the full range of responses within a given factor, and are presented for illustration). We also provide the number of responses coded this way; note that some responses had more than one code.

Factor

Why did students

write tests?

# (%)

Why did students

not write tests?

# (%)

Well-being

To avoid stress/pressure later

33 (3.2%)

To avoid stress/pressure now, or health issues

28 (18.7%)

Difficulty

Thinks a test will be easy

34 (3.3%)

Thinks a test will be hard

1 (0.7%)

Force

To avoid receiving a zero mark for a test

48 (4.7%)

N/A

0

(0%)

Habit

“I don’t ever skip exams”, “It’s my responsibility to”

79 (7.7%)

N/A

0

(0%)

Indifference

“It can’t hurt my grade”

104 (10.1%)

“I don’t really need to…”

4 (2.7%)

Knowledge

To test knowledge or review the material

253 (24.7%)

“I haven’t caught up”

10 (6.7%)

Marks

To get the best mark possible

328 (32.0%)

“I did good enough on a previous test”

11 (7.3%)

Study

“I felt prepared”, “I studied well”, to practice for final

195 (19.0%)

“I wasn’t prepared enough”

52 (34.7%)

Time

Has more time to study now/less time to study later

30 (2.9%)

Has less time to study now, has a time conflict

90 (60.0%)

Weight transfer

Doesn’t want to depend on weight transfer

111 (10.8%)

Was willing to transfer weight

18 (12.0%)

Other

None of the above

40 (3.9%)

None of the above

6

(4.0%)

We found no evidence that the reason a student takes a test has an impact on their test grade.

Do students engage differently with the course because of flexible assessment, and if so, how?

By qualitatively coding student responses to whether they engaged with the material in the course as a whole, we found that by the end of the course, an increasingly large number of students perceived that they engaged differently with the course material because of flexible assessment (in particular, weight transfer):

Survey 1

Survey 2

Engaged differently

106 (36.2%)

168 (54.7%)

Did not engage differently

139 (47.4%)

78 (25.4%)

Unclear

48 (16.4%)

61 (19.9%)

For students who engaged differently because of flexible assessment, their reasons as to why were generally one of several things, described below:

Reason

How did student engage differently?

# (%)

Well-being

Student felt less stress, pressure, or maintained better health

157 (26.2%)

Content

Student focused on the content (over assessment marks)

17 (2.8%)

Distribution

Student better managed their overall course load

57 (9.5%)

Effort

Student “slacked off”, procrastinated, studied less, or studied less effectively

72 (12.0%)

Plan

Student felt they could plan better with a “second chance”

52 (8.7%)

Study

Student studied more, or more effectively

31 (5.2%)

Time

Student more effectively managed their time in STAT 231

37 6.2%)

(Note that some responses had more than one code, some did not receive a code, and some students who reported they did not engage differently received a code due to the content of their response)