Welcome to the project webpage for the GenAI Shadow Grading for Large Survey Courses, a University of Waterloo Teaching and Learning AI Fellowship initiative exploring how GenAI-supported assessment tools can improve grading consistency, feedback quality, and scalability in large undergraduate courses while maintaining strict human oversight and ethical safeguards.
Project overview
Providing consistent, timely, and high-quality assessment feedback in large classes (> 150 students) remains a premier logistical bottleneck in higher education. When grading is distributed across massive teams of teaching assistants, courses frequently suffer from grading inconsistencies up to 15–20% alongside delayed feedback turnarounds that stall student learning. This project will develop and evaluate an ethics-forward, scalable model for GenAI-supported assessment in a massive first-year survey course (ENVS 195, ~650 students, 12 TAs). Microsoft 365 Copilot will be rigorously instructor-trained with course rubrics, annotated exemplars, and calibration guidance to generate a parallel "shadow" grade and feedback dataset. While TAs continue to assign official grades, this project will scientifically quantify TA-AI alignment and assess how automated tools can optimize feedback clarity and consistency.
Key pedagogical innovation & AI components
- The Problem Addressed: Traditional calibration sessions are resource-heavy and involve tough tradeoffs between grading speed and accuracy, while exploratory AI grading remains rare due to ethical and reliability anxieties.
- The GenAI Approach: Using Copilot Studio credit packs, a custom Copilot agent is fed strict instructor-provided rubrics, lecture materials, and past assignment samples spanning multiple performance tiers to accurately mirror human TA preparation.
- Strict Human Oversight: The AI functions exclusively as a research shadow; scores are completely de-identified, voluntary, and have zero impact on official student grades.
Project fellows
Dr. Christine Barbeau, Associate Professor, Teaching Stream, Faculty of Environment Associate Dean, Teaching & Environment Teaching Fellow
Dr. Erin O’Connell, Associate Professor, Teaching Stream & Associate Chair, Undergraduate, Faculty of Environment
Transferability and broader relevance
Adaptability for Instructors
The project delivers a plug-and-play assessment framework engineered specifically for cross-faculty adoption. Regardless of an instructor's prior comfort with artificial intelligence, the completed workflows provide an intuitive blueprint for training custom agents using standard course rubrics, easily mapping onto any discipline reliant on distributed marking teams.
Anticipated outputs and project deliverables
As the project advances through its developmental phases, the team will publish and distribute a suite of practical tools for the broader UWaterloo community. Deliverables will be shared on this project website and via broader University-wide communication channels. Stay tuned!