Grant recipients:
Ada Hurst, Department of Management Sciences
Oscar Nespoli, Department of Mechanical and Mechatronics Engineering
(Project timeline: May 2015 to July 2016)
Description
The goal of this project was to assess the quality and impact of a face-to-face peer review activity in engineering capstone design courses as it is currently implemented in progress update meetings. It was hoped that a better understanding of how peer review is delivered and utilized by students in their projects, instructors could more strategically develop review meetings and related assessment formats that support the peer review and draw on its maximum benefits.
Questions Investigated
Three research questions were investigated:
- How can the formative assessment and feedback provided by students (as novices) during progress update meetings be characterized?
- How can the formative assessment and feedback provided by instructors (as experts) during progress update meetings be characterized?
- What do we find when we contrast and compare traditional progress update meeting format feedback with the proposed peer-to-peer/instructor format?
Findings/Insights
We found that in order to characterize feedback, we needed to develop a typology to identify and describe its key elements. We characterize feedback in design review meetings as having a feedback type dimension, and a design process content dimension. The characterization of type is summarized in Figure 1, with a more detailed characterization found in our working paper [1] (PDF). The characterization of design process content can be found in [1].
Some of the data has been coded using grounded theory, and our preliminary findings
reveal (as aggregated data):
- Peer feedback is characterized as having a high percentage of Interpretation (I) relative to Evaluation (E) and Recommendation (R).
- Instructor feedback is characterized as having a greater percentage of Evaluation (E) comments over Peer feedback, in both instructor-only and mixed format (instructor and peer) progress update meetings.
- Instructor feedback is characterized as having a greater percentage of Recommendation (R) comments over peer feedback, in both instructor-only and mixed format (instructor and peer) progress update meetings.
- In comparing and contrasting the feedback provided by the Instructor in both the Instructor only and Peer and Instructor meetings, we find that the instructor feedback type percentages are very similar regardless of the meeting type
Dissemination and Impact
A working paper ([1]) was presented in a working session at the American Society for Engineering Education (New Orleans), in June 2016:
Hurst A. and Nespoli O.G., A Two-Dimensional Typology for Characterizing Student Peer and Instructor Feedback in Capstone Design Project Courses, American Society for Engineering Education (ASEE) Annual Conference and Exposition, New Orleans, Louisiana, June 2016.
Impact of the Project
- Teaching: Peer review has been incorporated into GENE403/GENE404 Interdisciplinary Capstone Design course as a key learning and assessment feature of the course.
- Connections with people from different departments, faculties, and/or disciplines about teaching and learning: Connections have been made at the ASEE conference regarding our work. Additionally, the Chair in Design Engineering at University of British Columbia, has expressed an interest in our findings in order that he might evolve the mechanical engineering capstone course to use this teaching and assessment methodology.
References
Project reference list (PDF)
Working paper (PDF)