CTAPT Phase 3 consultations (July 2021)

CTAPT Overview

In May 2020, Senate endorsed CTAPT’s recommendation that the University adopt multi-faceted teaching assessment, and that CTAPT’s mandate be renewed. Phase 2 started March 2021.
Goals for Phase 2
- Develop and recommend feasible and meaningful processes for implementation of teaching dossiers (TD) and peer review of teaching (PRT) as routine parts of the assessment of teaching for regular faculty members at Waterloo.
- Consult and communicate with the campus community to prepare the ground for broad acceptance of these processes when they are launched.
- Coordinate the creation of toolkits (guidelines, templates, rubrics, samples, etc.) to support the processes for TD and PRT.
- Provide recommendations about how these processes might be extended to instructors who are not regular faculty.
- Suggest which University processes and Policies might require revision to be better aligned with new practices of teaching assessment.
CTAPT is consulting the campus community about its initial impressions about what to recommend.
CTAPT anticipates presenting recommendations to Senate in late 2021.
This document provides details about CTAPT’s current thinking.

Teaching assessment update

Separate groups on campus are working on:
- student course perception surveys (SCP)
- assessment of graduate student supervision
University is working towards adopting a comprehensive definition of teaching effectiveness against which teaching of all kinds can be measured.
A draft definition was presented at Senate in June 2021:

	Design	Implementation	Learning experience	Professional development
Overview	Planning Creation	Design to action	Student perceptions of their learning experience	Activities supporting growth as instructor/ supervisor
Areas of focus	Framework Alignment Assessments	Communication Promotion of student engagement Variety of teaching strategies/practices Assessments/feedback	Rapport Responsiveness Engagement with learning Diversity/inclusivity	Reflection Continuous improvement Collaboration, mentorship, and leadership
Examples of possible indicators	Learning objectives Learning materials Learning activities Teaching philosophy	One-on-one/small group/large group interactions Synchronous/ asynchronous instructional activities	Classroom environment Outside-of-class (i.e., interaction with instructor/ supervisor)	Refinement of instructional materials/approaches Workshops and conferences (participation/leadership) Scholarship of Teaching and Learning Mentorship (colleagues/students)

When CTAPT and other groups are finished their work (target: end of 2021), conversations will begin about how to “stitch” together the instruments and their results in a holistic way that is fair and equitable in order to recommend how to produce an annual/bi-annual teaching “score.”
Triangulation of several sources of assessment should lead to an overall mechanism that is more reliable and should reduce the impact of bias contributed by any one instrument/method.

Update on teaching dossiers

A Teaching Dossier (TD) is to teaching what a CV is to research. Through narrative, it documents and supports claims about teaching based on multiple forms of empirical evidence. It describes documents and materials, which collectively suggest the scope, quality, and impact of teaching.

Teaching dossier for annual/bi-annual performance review (APR/BPR)

Normally between two and three pages. Not a complete teaching dossier, but really a “description of teaching for ABR/BPR.”
Includes:
- List of details about teaching and supervision, teaching scholarship, curriculum work, course renewal and development, professional development.
- Narrative/reflection about Design, Implementation, Student Experience, Professional Development, connected to definition of teaching effectiveness via examples. Possible prompting question: “Consider what you have done with course design and implementation, comment on student experiences, how has PD affected your practice, what are next steps for you. Where possible, provide evidence from you and your students, and peers if available.”
- Space to comment on SCP resubts.

Teaching dossier for tenure and promotion (T&P)

“Focussed” (roughly eight pages) at both points for T&P.
Aligned with APR/BPR forms; should be able to feed over time into larger TD for T&P.
Includes Teaching Philosophy with evidence to support.

Support materials to be developed

Guidebooks, templates, samples, rubrics.
Examples pertaining to each bullet in longer teaching effectiveness definition.
Elaboration on prompts provided for documentation.

Update on peer review of teaching

Peer review of teaching (PRT) = A reflective process through which colleagues offer feedback about teaching for either formative or summative purposes. It is an intentional and structured process in which a peer reviews course material and observes teaching with the aim of providing feedback, based on predetermined goals and criteria.

Peer review for summative assessment

Structured process to review course.
Consists of review of course materials, pre-observation meeting, observation, post-observation meeting.
Observation uses pre-defined narrative form/report structure.
Reports made available to unit heads / performance committees.
Frequency: twice pre-tenure/continuing; every five to ten years post-tenure/continuing; frequency for other instructors TBD.
Ideal: each cycle includes reviews from two colleagues.

Peer review for formative assessment

Structured process, but no report to unit head.
Can use traditional model; otherwise, could be tied to teaching squares, reciprocal pairs, etc.
Prompted by instructor or by unit.
Can occur at all points in career.
Participation should count as professional development for APR/BPR.

Reviewers

Summative peer reviewers should receive service credit.
Train reviewers for summative peer review of teaching using centrally-developed modules.
Lighter training for formative peer reviewers.
Ideal: one reviewer from subject area, one reviewer outside of subject area.
Instructor has ability to suggest and/or veto reviewers and to appeal review.
Administrative support necessary to manage reviewer lists, who has been reviewed when and by whom, etc.

Content to review

Course materials: course outline, assigned readings, assessments, rubrics.
Synchronous components: in-person or online.
Asynchronous components.
Communication: possibly including discussion boards, scheduled availability, etc.

Support materials to be developed

Specific criteria based on definition of TE
Forms for reviewer
Training modules and materials

Questions for discussion

What questions do you have about these models and suggestions?
What obstacles do you envision to adopting models like these?
If TDs and/or PRTs are being used in your unit, what works well that we should consider?
How can we best ensure motivation and buy-in from faculty members, chairs/directors, and senior administration?
Regarding peer review:
1. How should peer reviewers be chosen?
2. Would colleagues in your unit be open to having reviewers from outside of your unit/faculty?
3. How often should peer review be done post-tenure/post-continuing?
What else would you like to tell CTAPT?