All instructors want to assess fairly, but doing so can sometimes require extra effort, especially when a large number of students are being graded by multiple assessors. Consider following these guidelines the next time you find yourself working with a team of assessors.
Prior to the assessment
- Communicate expectations, requirements, and guidance to students:
- Read the cover page of the assessment;
- Read the whole question before starting on an answer;
- Complete all requirements;
- Where appropriate, follow approaches modeled earlier in the course for similar questions;
- Aim for clear, complete, and well organized responses.
Before the assessment session
- Personally hire knowledgeable, trustworthy assessors (as few as needed to complete the assessment expeditiously).
- Construct a rubric or grading key as soon as the assessment is designed.
- Determine and arrange logistics for assessment day, including the location; expected duration of the session; snacks and coffee, tea, or juice for especially long sessions.
- Allocate assessment components or questions to assessors such that they will complete their allotted assessments roughly at the same time.
- After the test has been completed by students, but before the assessment session, perform a sample assessment of up to 10 tests (either randomly or choosing particular students’ assessments) and adjust the rubric or assessment key (to possibly include alternative solutions).
- After testing is completed by students, but before the assessment session, provide the test to the markers for them to review prior to the marking time.
At the assessment session
- Use control and tracking sheets to track which assessor marked which test (using a rubric in LEARN will track assessors automatically) and for statistics on mark breakdown within an assessment.
- If possible have only one assessor for each part of every assessment.
- Seat assessors together in one location to facilitate communication between groups of assessors.
- Have each group of assessors mark a sample of up to 5 exams for their assigned part, then discuss results of their sample, and potentially adjust the assessment key (the lead assessor or instructor should be part of this conversation among the groups or assessors).
- Have assessors forward problematic responses (such as illegible handwriting or ambiguity due to English as a second language issues) to the instructor to assess.
- During the assessment session, have the instructor review samples of marked exams for consistency/issues between assessors.
- After all of the assessments have been completed, have each assessor or group of assessors identify and record common gaps. (This information can help the instructor refine the learning activities the next time the course is offered.)
After the marking session
- Use three recorders to enter the grades: one to read, one to enter the grade, and one to watch.
- Review assessments with low marks, especially for final exams where the overall course mark is near pass/fail or minimum program requirements.
- Look for and investigate surprises such as a wide range of assessment marks, bearing in mind the overall academic performance of the cohort of students taking your course.
- Confirm and communicate to students the basis for an appeal, gather assessments deemed to qualify for review, and review them all at once, if possible.
- Determine what will be communicated to various groups of students based on their performance on the assessment.
Support
If you would like support applying these tips to your own teaching, CTE staff members are here to help. View the CTE Support page to find the most relevant staff member to contact.
This Creative Commons license lets others remix, tweak, and build upon our work non-commercially, as long as they credit us and indicate if changes were made. Use this citation format: Tips to Address Assessment Consistency with Many Assessors. Centre for Teaching Excellence, University of Waterloo.