Using complementary methods for the assessment of teaching
- No one source of evidence is the best source of information for ALL teaching activities.
- Utilizing multiple methods and sources of evidence increases the reliability, validity and fairness of the assessment of teaching; when possible, triangulate multiple sources or forms of evidence.
- Using Peer Review of Teaching (PRT) and Teaching Dossiers (TD) along with data from student course perception surveys (SCPS) enables instructors to document evidence of teaching effectiveness more comprehensively and provide context for results (see Table 1) .
Table 1: Overview of a complementary approach to assessment
| Dimension of teaching effectiveness | Sources | Methods for providing evidence/information |
|---|---|---|
| Design |
Instructor Peer |
TD: Instructor provided descriptive data explaining beliefs and rationale for teaching approach (teaching philosophy) and methods used with evidence, such as syllabi, assignments, or learning activity examples. PRT: Peer review of course materials, such as syllabi and learning materials or activities |
| Execution |
Peer Instructor Student |
PRT: Peer observation of the implementation of teaching methods and approaches, review of materials, pre-and post-meeting with instructor TD: Instructor descriptions on using methods, strategies and approaches, and reflection on implementation. SCPs: Student’s experience with specific aspects using appropriate student survey questions |
| Student experience |
Student Peer Instructor |
SCPs: Appropriate student survey questions about student’s perceptions of their own experience PRT: Peer observation based on set goals for observation and using appropriate tools, review of course materials TD: Instructor narrative on strategies and approaches used and reflection on implementation; contextualization or responses to survey results (and PRT if completed) |
| Professional development |
Instructor Chair Peer |
TD: Description of scholarly approach - reflection and application of practices; evidence of professional development (e.g. teaching workshops, conferences, SoTL publications, teaching awards, mentorship, etc.) Chair: Letters of support, knowledge of instructor’s profession development, involvement in committees, curriculum work, or mentorship Peers: Letters of support in relation to mentorship, contributions, etc. |
Table 2: Mapping methods to dimensions of teaching effectiveness
- According to Arreola’s “rule of thumb” for selecting sources of evidence, “...the important principle to follow in identifying sources is always to select the source which has the best opportunity to observe first-hand the performance to be evaluated” (Arreola 2007: 47 in Murphy et al. 2009: 228; see also Chism 1999; Mills in Seldin 1999; and Seldin 2006).
- Keeping in mind that more than one method may be a source of evidence, the Table below outlines which method provides the most direct source of evidence for each descriptor of teaching effectiveness and is thus a “primary source” of evidence. Primary sources are not more important, rather, they provide the most direct source of evidence; secondary sources are those that provide additional observable evidence.
- Primary Sources are indicated with two checkmarks
- Secondary Sources are indicated with one check mark
- Not a source is indicated with an “X”
| Dimension of teaching effectiveness | Peer review | Teaching dossiers | Student Course Perceptions survey |
|---|---|---|---|
| DESIGN | |||
| Planning | |||
| Builds course around a design framework from the literature, from the field of study, or created by the course author | ✔ | ✔✔ | X |
| Clearly defines course-level and activity-level learning objectives | ✔ | ✔✔ | ✔1 |
| Includes learning material that reflects current scholarship from the field or that is clearly relevant | ✔ | ✔✔ | X |
| Structures material in a logical and coherent order | ✔ | ✔✔ | ✔* |
| Sets pacing, workload and performance standards appropriate for the course level and topic | ✔ | ✔✔ | ✔✔2 |
| Includes experiential components, professional connections, or practical applications, when possible. | ✔ | ✔✔ | X |
| Plans a variety of teaching/learning strategies to promote student engagement and deep approaches to learning | ✔ | ✔✔ | X |
| Incorporates a diversity of experiences, viewpoints, and backgrounds in course materials | ✔ | ✔✔ | X |
| Adheres learning materials, activities, and assessments to University accessibility policies | ✔ | ✔✔ | X |
| Alignment | |||
| Aligns course design with program expectations | ✔ | ✔✔ | X |
| Aligns course objectives and learning outcomes with course content and delivery | ✔ | ✔✔ | X |
| Develops fair and equitable assessment methods that align with course objectives and outcomes | ✔ | ✔✔ | ✔3 |
NB: Piloted SCPS questions that correlate with descriptors are indicated with a footnote (see also Appendix). An asterisk (*) indicates that there is no corresponding pilot questions at this time, but where surveys or other student evidence may be a sources of evidence.
1 “The instructor identified the intended learning outcomes for this course” (Based on Student Course Perception Survey questions piloted by CEPT2, accessed September 19, 2019).
2 “The course workload demands were…” [Select a choice from a scale of very low to very high] (ibid).
3 “The intended learning outcomes were assessed through the graded work” (ibid).
| Dimension of teaching effectiveness | Peer review | Teaching dossiers | Student Course Perceptions survey |
|---|---|---|---|
| EXECUTION | |||
| Communication | |||
| Communicates course-level and activity-level objectives as well as teaching/learning approach and rationale to students | ✔ | ✔✔ | ✔✔1 |
| Describes and explains material clearly using a pace appropriate to the context | ✔✔ | ✔ | ✔✔4 |
| Demonstrates enthusiasm for the subject | ✔✔ | ✔ | ✔✔* |
| Uses technology, media or other teaching tools effectively | ✔✔ | ✔ | ✔* |
| Student Engagement | |||
| Promotes student participation, peer interactions, or other active engagement with course content | ✔✔ | ✔✔ | ✔* |
| Uses teaching/learning strategies that encourage student engagement and deep approaches to learning | ✔✔ | ✔✔ | ✔* |
| Variety of Elements | |||
| Adapts to evolving classroom contexts | ✔ | ✔✔ | X |
| Adopts a variety of instructional practices, content types, and assessments that recognize diversity of learners | ✔ | ✔✔ | X |
| Assessments and Feedback | |||
| Enables students to prepare for assessments through instructional practices | ✔ | ✔✔ | ✔✔5 |
| Communicates clear expectations and instructions for assessments | ✔✔ | ✔ | ✔✔* |
| Provides performance feedback in a timely manner | X | ✔ | ✔✔6 |
| Provides directions for student improvement individually or collectivel | X | ✔✔ | ✔✔* |
4 “The instructor helped me to understand the course concepts” (ibid).
5 “The course activities prepared me for the graded work” (ibid).
6 “Graded work was returned in a reasonable amount of time” (ibid)
| Dimension of teaching effectiveness | Peer review | Teaching dossiers | Student Course Perceptions survey |
|---|---|---|---|
| STUDENT EXPERIENCE | |||
| Rapport | |||
| Fosters a supportive learning environment | ✔✔ | ✔ | ✔✔7 |
| Establishes a climate of intellectual openness | ✔✔ | ✔ | ✔✔7 |
| Shows concern for students' success and wellbeing | ✔✔ | ✔ | ✔✔ |
| Interacts professionally and respectfully with students | ✔✔ | ✔ | ✔✔ |
| Responsiveness | |||
| Provides sufficient opportunities for student contact inside and outside of class | ✔ | ✔✔ | ✔✔ |
| Responds to student inquiries and questions in an appropriate timeframe | X | ✔✔ | ✔✔ |
| Diversity | |||
| Promotes the success of all students by addressing the variety of their experiences, viewpoints, and backgrounds | ✔✔ | ✔✔ | ✔ |
| Engagement and Learning | |||
| Generates and maintains student interest | ✔✔ | ✔ | ✔✔8 |
| Fosters students' intrinsic motivation and responsibility for their own learning | ✔ | ✔✔ | ✔ |
| Seeks student input on course learning experience | X | ✔✔ | ✔ |
| Provides evidence of student learning | ✔ | ✔✔ | X |
7 “The instructor created a supportive environment that helped me learn” (ibid).
8 “The instructor stimulated my interest in the course” (ibid).
| Dimension of teaching effectiveness | Peer review | Teaching dossiers | Student Course Perceptions survey |
|---|---|---|---|
| DEVELOPMENT | |||
| Reflection | |||
| Reflects on and assesses teaching and learning practices | X | ✔✔ | X |
| Engages in a scholarly approach to teaching through determining and implementing best practices | X | ✔✔ | X |
| Growth and Continuous Improvement | |||
| Participates in professional development activities | X | ✔✔ | X |
| Makes thoughtful and deliberate changes to practices or develops innovations in response to new information about best practices or to other opportunities as they arise | X | ✔✔ | X |
| Regularly revises or updates course content, assignments, format, or teaching strategies in response to feedback and reflection | X | ✔✔ | X |
| Collaboration, Mentorship and Leadership | |||
| Demonstrates leadership related to teaching and learning within the University and in the broader educational community | X | ✔✔ | X |
| Interacts and works with colleagues around teaching and learning | X | ✔✔ | X |
| Provides and receives mentorship related to teaching, including with teaching assistants | X | ✔✔ | X |
| Contributes to the Scholarship of Teaching and Learning (SoTL) by disseminating findings from research on instructional practices | X | ✔✔ | X |
