Introduction
The Complementary Teaching Assessment Project Team (CTAPT) was formed in Winter 2018 to research and develop methods of assessing teaching and learning and to provide recommendations that are useful for both formative and summative assessment, based on empirical evidence and feedback consultations with the University of Waterloo community. The focus of CTAPT is to recommend additional methods other than student course perception surveys that should be used campus-wide. This backgrounder summarizes key findings on one of CTAPT’s research questions – “What is effective teaching?” While defining and evaluating teaching effectiveness (TE) are interconnected, this backgrounder focuses on defining TE. In the coming months, CTAPT will produce more information about tools.
Grounded in an extensive literature review and environmental scan of current U15 teaching evaluation practices, the project team developed a broad definition of teaching effectiveness framed by four dimensions – Design, Execution, Experience, Development – each consisting of subdimensions describing evidence-based principles of effective teaching. These dimensions and principles have wide application across faculties and disciplines and build on existing University of Waterloo conceptions of effective teaching outlined in Policy 77 and proposed in the Undergraduate Learning: Issue Paper (May 2018). (For more information on how these dimensions connect to the literature, please see the Dimensions of Teaching Effectiveness: Links to Literature document).
The consultation survey
CTAPT is consulting with the University of Waterloo community through the CTAPT Teaching Effectiveness Survey to gather feedback on this definition. This is the first phase of consultations. The next phase will focus on key findings on current and best practices of complementary methods of teaching assessment. Phase two of the consultation process will also illustrate how the proposed definition of TE may be incorporated into evaluation tools and instruments.
Key findings
Shared understanding of teaching effectiveness
Recent initiatives aimed at fostering and promoting teaching quality in Higher Education have stimulated a renewed interest among scholars in reexamining how to define and measure teaching effectiveness for a variety of purposes such as annual reviews, promotion and tenure, and the improvement of teaching and learning.1 One of the essential components of a complementary or multi-faceted approach to the assessment of teaching is a clear and shared understanding of teaching effectiveness.2 One goal of CTAPT, therefore, is to recommend a definition of teaching effectiveness (TE) that would be useful for complementary methods of assessing teaching and learning. As a starting point, Policy 77 (University of Waterloo) includes a statement of effective teaching that forms the basis for a shared understanding: “The purpose of teaching is to facilitate learning. Thus, effective teaching draws the strands of a field together in a way that provides coherence and meaning, places what is known in context, lays the groundwork for future learning, and opens the way for connections between the known and the unknown.” It further states that faculty members are expected to evaluate student work fairly, provide constructive feedback, be reasonably available for consultations, and respect the integrity, diversity, and confidentiality of students.
Broad definition of teaching effectiveness and central features
The task of determining how to define TE has been tackled by identifying broad dimensions and sub-dimensions using a variety of theoretical perspectives and quantitative and qualitative approaches from disciplinary, teacher, and student perspectives.3 While no standardized definition of TE exists, our review of the literature confirms that there is consensus on essential characteristics. Decades of research has resulted in the identification of several central features that cut across disciplines and modes of instruction.4 Scholars also agree effective teaching is focused on student learning, which requires “alignment” facilitated by teaching skills and practices that promote “higher order learning processes” or “deep learning.”5 In a similar vein, the Undergraduate Learning: Issue Paper proposes a “broad and conceptually simple” definition from the literature of effective teaching as being “oriented to and focused on students and learning,” where learning is “a process that leads to change, which occurs as a result of experience and increases the potential for improved performance and future learning” (p2). It identifies five evidence-based principles of effective teaching relevant to the Waterloo context: “use of alignment in course and curriculum design, fosters motivation, embodies inclusivity, encourages deep learning, and enables lifelong learning” (ibid).
Informed by this research and existing University of Waterloo policies, CTAPT developed a broad definition of teaching effectiveness framed by four central dimensions: Design, Execution, Experience, Development. Each dimension consists of sub-dimensions describing evidence-based principles of effective teaching adapted from Undergraduate Learning Issue Paper (2018), Allen et al. (2009), Bain (2004), Chickering and Gamson (1987), Hativa et al. 2001, and Ramsden (2000, 2003) that are relevant and applicable to the University of Waterloo context (see Table below):
- DESIGN: Planning and Alignment
- EXECUTION: Communication, Student Engagement, Variety of Elements, Assessments and Feedback
- EXPERIENCE: Environment, Responsiveness, Diversity, Engagement and Learning
- DEVELOPMENT: Reflection, Growth, Collaboration and Mentorship
Central features are consistently identified by different stakeholders
Studies investigating teacher and student perspectives on TE provide insight on where subdimensions are consistent or divergent; variances tend to be a matter of emphasis or a result of context. For example, studies examining teachers’ perspectives show that descriptors of “good teaching” are consistent across disciplines.6 Findings on student perspective are similar, although students place greater emphasis on the importance of the “Experience” dimension, particularly student-teacher interrelationships such as approachability, rapport, caring about students, respect, patience, trust, and enthusiasm.7 Another study reveals students’ perception of good teaching varied based on their expectation of approaches normally used in their discipline, demonstrating the importance of “context.”8
Rationale for a broad definition of TE - consideration of context
Conceptions of teaching effectiveness are multi-faceted and “context-bound.”9 Following Allen et al. (2009) and Devlin and Samaracwickrema (2010: 118), factors such as course parameters (size, level, etc.), mode of teaching, student ability, and disciplinary pedagogical approaches to teaching and learning are viewed as “contexts” that influence teaching effectiveness rather than a determining factor of TE in itself.10 Defining TE in a way that accounts for all these factors would be a difficult task. There are also university and department contexts, as well as wider social, political, economic, technological and demographic change forces that influence conceptions of TE (ibid). The University of Waterloo already acknowledges this rationale; Policy 77 outlines a broad conception of TE that recognizes the complexity and diversity of teaching at the University of Waterloo by stating: “because these standards are intended to apply university-wide to faculty members engaged in complex intellectual endeavors, they cannot be expressed in absolute quantitative terms.” A broad conceptualization of TE, therefore, acknowledges the need for a university-wide shared understanding of TE as one aspect of an effective and transparent evaluation framework while recognizing the complexity and diversity of teaching and learning contexts. Because some sub-dimensions would be more or less relevant depending on context, additional or more detailed sub-dimensions descriptors and weights should be refined at the faculty and/or department level by considering context, faculty values, and discipline specific practices.11
| Design | Execution | Experience | Development |
|---|---|---|---|
|
Planning - Builds course around evident design framework - Clearly defines course level and activity-level learning objectives - Includes learning material that is relevant and/or current scholarship from the field - Structures material in a logical and coherent order - Includes experiential and/or professional connections, when possible - Sets workload and performance standards appropriate for the course level and topic - Plans a variety of teaching/learning strategies to promote student engagement and deep learning Alignment - Aligns program expectations, course objectives and learning outcomes with course content, delivery of course and learning activities, and assessments - Develops appropriate and sufficient assessment methods that align with course objectives and outcomes |
Communication - Describes and explains material clearly using an appropriate pace - Demonstrates enthusiasm for the subject - Uses technology and/or media effectively Student Engagement - Promotes student participation, peer interactions, and engagement with course content - Uses teaching/learning strategies that encourage student engagement and deep learning Variety of Elements - Adapts to evolving classroom contexts - Adopts a range of teaching strategies, methods, and/or modalities that recognize diverse talents and ways of learning Assessments and feedback - Prepares students for assessments by delivering content - Communicates clear expectations and instructions for assessments - Provides performance feedback in a timely manner - Provides directions for student improvement individually or collectively |
Environment - Is approachable - Builds appropriate rapport with students Responsiveness - Provides opportunities for student contact inside and outside of class - Responds to student inquiries and questions in an appropriate timeframe Diversity - Promotes inclusivity and diversity by acknowledging variety of experiences, viewpoints, and backgrounds Engagement and Learning - Generates and maintains student interest - Fosters intrinsic motivation - Seeks student input on course learning experience - Provides evidence of student learning |
Reflection - Reflects on and assesses teaching and learning practices - Engages in a scholarly approach to teaching and learning through determining and implementing best practices - Takes an iterative approach to continuous improvement and growth. Growth - Participates in professional development activities - Makes changes to practices and/or develops innovations - Regularly revises or updates course content, assessments, format, or teaching strategies Collaboration and Mentorship - Leads activities to promote teaching and learning - Interacts and works with colleagues around teaching - Provides and receives mentorship related to teaching - Contributes to the Scholarship of Teaching and Learning (SoLT) |
Bibliography
Chism, Nancy. 1999. Peer Review of Teaching: a Sourcebook. Anker Publishing Co.
Ramsden, Paul. 2003. Learning to Teach in Higher Education. RoutledgeFalmer.
Endnotes
1 See Donna Ellis https://cte-blog.uwaterloo.ca/?p=4166 and University of Waterloo Strategic Plan in Action 2013-2018, and Bridge to 2020 Issue Papers. Internationally, the United Kingdom (UK) and Australia have led the way in terms of conducting studies to gather evidence (e.g. Bahia 2017; Crookes et al. 2018; Dias-Mendez et al. 2012; Gourlay and Stevenson 2017; Wood and Su 2017). Debates about standardizing definitions of TE is felt strongly in UK because of 2015 Green Paper and 2016 white paper, which in introduced assessment of teaching at the HEI level through the Teaching Excellence Framework that would allow institutions to increase fees depending on results (Gourlay and Stevenson 2017). Critiques of UK Model include criticism of the notion of ‘excellence’ itself and neoliberal approach, which views students as customers and uses a marketing model for HEI (see Bahia 2017; Bartram et al. 2018; Saunders et al. 2017). On the other hand, a climate of accountability and focus on quality enhancement in the UK also led to the establishment of 74 Centres for Teaching and Learning, with the purpose of recognizing and rewarding excellence in HE and promote development (Allan et al. 2009: 362).
2 Arreola 1995; Chism 1999; Smith 2014; Wright et al. 2014.
3 Devlin and Samarawickrema 2010: 112.
4 See for example Allan et al. 2009: 364; Bain 2004; Bartram et al. 2018: 2, 5; Hill 2014; see also Harris 1998.
5 For example Biggs 2003; Bain 2004; Devlin and Samarawickrema 2010: 112; Parpala et al .2011; see also Ramsden 2003; Trigwell, Prosser, and Waterhouse 1999.
6 See studies by Samples and Copeland 2013, and Bartram et al. 2018, for example. Samples and Copeland’s large mixed method study comparing perspectives of Engineering academics with a wider university sample across faculties showed that essential descriptors defining good teaching where consistent across both samples (2013: 181), echoing findings from other studies (e.g. Bain 2004 and Lowman 1995). Bartram et al.’s (2018) mixed methods study compared English and Australian academic views of teaching excellence and found views to be consistent with both samples.
7 Allan et al. 2009: 366, 368 and Hill 2014: 58.
8 Parpala et al. 2011: 560-1.
9 Bartram et al. 2018; Biggs 2003; Parpala et al. 2011: 3.
10 For example, characteristics such as “includes interactions” or “includes group work” would depend on course context, and thus it would be problematic to standardize such a characteristic across disciplines (Parpala et al. 2011: 557-8).
11 Arreola 1995: 1; Berk 2005; Wright et al. 2014: 14.