"As a reminder, best practice guidelines encourage instructors to provide in-class time for students to complete their course evaluations; when possible, instructors should devote the initial or middle minutes of class for this activity (without taking these steps, student response rates are expected to be low using online rating tools). Instructors may want to advertise when in-class time will be devoted to course evaluations to ensure that students bring an electronic device with them ([our online evaluation platform] can be accessed by computer, tablet or smartphone).” - email to Faculty of Science instructors, 2019
In light of my previous post related to regaining classroom time by eliminating in-class advertising, I though it might be helpful to share my experiences with in-class course evaluations that are common at the University of Waterloo and at most academic institutions. Although class evaluations during my first few years of teaching involved a planned end-of-term class interruption by administrators who circulated evaluation forms to all students while I left the room, our institution adopted an online platform several years ago. In doing so, and in order to address projected lower participation rates for an online process, best-practice recommendations to instructors indicate that we should set aside class time for students to fill out their online course evaluations.
THE PROBLEM
From my perspective, there are several problems with shifting responsibility for administering online evaluations to instructors.
1: Quality. To improve the quality of student feedback, especially the open ended questions, providing 10-15 minutes in a busy classroom setting is insufficient for adequate reflection and expression. With my own online evaluation implementations since Fall 2014, conducted entirely outside of my classrooms (because I declined to offer class time), answers to questions were much longer than I had seen previously with paper-based evaluations. Given this potential for students to provide more extensive feedback, hosting brief online evaluations in class can send the wrong message to students that a “quick job” is all we need for our course evaluations. Worse, students using smartphones to complete the evaluation will be providing superficial responses at best.
2: Logistics. In large classes (e.g., 400+ students), like mine, requiring that all students bring laptops and smartphones to class on a given day is an enormous waste of time and effort for everyone. Most students forget such a request and laptops/smartphones are largely seen as an unwelcome distraction by many instructors.
3: Frustration. An instructor’s repeated emails, online reminders, and in-class announcements are annoying for professors and students alike, and detract from the quality of student-professor interactions and a carefully crafted learning experience. Worse, students who do not attend the in-class evaluation will still require reminders to complete their evaluation, such that the entire class continues to receive reminders even after the task is completed. Noise.
4: Bias. Some professors might bring treats to class on evaluation days, play music while the evaluation is ongoing, stand at the front and watch students complete their evaluations, or disappear and allow the class to become distracted. Each class will be different and the varying moods and conversations in a classroom will bias the outcomes of a course evaluation.
5: Privacy. Laptop screens are not private and, as a result, student evaluation data may be observed by others. In addition to compromising privacy, this can also bias student opinions.
6: Instructor workload. With an online evaluation system administered in class, an instructor must now advertise the course evaluation system, explain the process, provide a link, add extra material to the course syllabus, send email reminders to students, request laptops and smartphones be brought to class, show an information slide during the evaluation, then issue several more reminders before the end of the evaluation period. In other words, all administration tasks for the evaluation are shouldered by instructors who are already stretched thin delivering the course itself.
7: Timing. The last two weeks of term are particularly hectic for students and are not conducive to optimal class attendance, nor to self reflection. In addition, when would you like to complete a course evaluation? Before the final exam or after the course is complete? Some would choose after to reflect on the course in its entirety but this is not possible with our system.
8: Data. Although current best practices are that classroom time should be set aside for the process, some instructors will comply while others will not. Policing the various process variations is impossible, students may complain about inconsistencies, and apples-to-oranges datasets will then prevent course-to-course or year-to-year comparisons of evaluation results.
Although an online evaluation process that requires class time lacks elements of innovation and convenience that would ideally characterize adoption of a new system, “fixing” a new online system process will likely remain low priority unless instructors speak out and decline to participate in a broken best practice process. Unfortunately, many instructors are still willing to give up class time, as they've done for many years before. Because of this requested class time sacrifice, as per tradition, migration to an online system is likely not accompanied by a noticeable decrease in response rates. However, shouldn’t an online system do better than simply hold modest response rates steady? Couldn’t there be a way to leverage a new system to increase response rates, reduce administrative burden, and improve data quality?
THE SOLUTION
Might the University of Waterloo consider adopting a process similar to that used by Dartmouth College (http://www.dartmouth.edu/~reg/guides/ceval/)? Dartmouth College uses an online course assessment method that opens in the middle-to-late term and closes a week into the following term. Students can fill out the course evaluation anytime in that window, even after the final exam. Once they have completed the course evaluations for all of their courses, they are then able to see their course grades. If they choose not to fill out their evaluations, they will still be able to see their grades, but must wait a few days into the following term. In other words, there is a grade release delay if they choose to opt out.
An incentivized system like that offered by Dartmouth College is simple yet innovative. This process can improve student feedback quality by allowing students time to complete their evaluations well and avoids requests by profs for students to bring laptops and smartphones to class. Student/instructor frustration is decreased because emails and reminders are unnecessary. Bias is not an issue because the classroom is reserved for learning rather than engineering an evaluation environment. Privacy concerns are addressed, instructor administrative burdens are reduced, and class time is regained. Timing is flexible and policing of classroom evaluation activities is no longer required. Data will be apples-to-apples across courses and faculties.
In conversation with one Dartmouth College professor, the opinions shared were entirely positive. This professor did not think that students rushed through the surveys just to access marks. Like any evaluation process, some students rush and others take more time and provide more thoughtful responses: "Data we get from these are pretty robust”. In addition, this professor teaches multiple courses and sees response rates >90% ("even 100%") consistently and effortlessly. Do the students like the Dartmouth College system? Yes. This professor suggested that students enjoy the evaluation process - they like having a voice.
CONCLUSION
I would very much like to see the University of Waterloo adopt a simple, innovative, and incentivized course evaluation solution like that offered at Dartmouth College. Beyond the University of Waterloo, my hope is that the ideas shared here might benefit others who work to identify an innovative and incentivized online evaluation system that reduces administrative burden for everyone while increasing response rates and data quality.