Grant recipients:
Kenneth Manson, School of Pharmacy
Andrew Tolmie, School of Pharmacy
Henry Halapy, University of Toronto
Annie Lee, University of Toronto
Diana Spizzirri, Ontario College of Pharmacists
(Project Timeline: May 1, 2017 - April 30, 2018)
Description
The School of Pharmacy has collaborated with the University of Toronto (UofT), the Ontario College of Pharmacists (OCP), and the Hospital Pharmacy Residency Forum of Ontario (HPRFO), in developing the Ontario Pharmacy Patient Care Assessment Tool (OPPCAT). All four organizations are committed to utilizing OPPCAT for the assessment of pharmacy learner performance in direct patient care settings.
In investigating the shared employment of this innovative assessment tool, the collaborators wish to ensure OPPCAT accurately measures learner performance by collecting information on its content, practicality, usability, and user satisfaction from learners and preceptors. Recognizing that OPPCAT is used to assess learners during late-curricular and post-graduate experiential learning, it is important to understand how learners experience the assessment feedback in preparation for independent practice as pharmacy professionals. The gathering and analysis of these data will inform future iterations of OPPCAT.
Questions investigated
Pharmacy learners and preceptors/assessors were surveyed to gather feedback regarding the recently adopted OPPCAT. Specific research goals were:
- To assess the accuracy and content validity of OPPCAT’s competence domains.
- To assess the practicality and usability of OPPCAT.
- To advance learner awareness of their knowledge and practice strengths and areas for improvement at entry-to-practice.
- To assess the overall satisfaction of learners and preceptors/assessors regarding the use of OPPCAT in capturing pharmacy learner competence.
Findings/Insights
Preliminary review of the survey findings affirms overall satisfaction, practicality and usability of the assessment tool. Survey results indicated the layout was easy to follow, the glossary was helpful, learner’s performance was accurately captured and important content was not missing. Aspects of OPPCAT to be considered are refining descriptors in the patient care domain, adding more glossary terms, exploring a common tool platform and expanding the professionalism domain. Assessment of professionalism of learners in practice-based settings is challenging. The OPPCAT group is re-examining/examining external resources such as the Ontario College of Pharmacists - Code of Ethics, the Association of Faculties of Pharmacy of Canada - Professionalism Guide and the Association of Faculties of Medicine of Canada Entrustable Professional Activities to help inform how this metric is assessed.
In addition, there were opportunities identified for ongoing preceptor/assessor training. Specific details surrounding ongoing training will be shared with the Preceptor Development sub-committee for review and implementation. The OPPCAT group continues to meet regularly for detailed reviews of results and where indicated, establish appropriate changes to evolve the tool accordingly.
Dissemination and impact
- At the individual level: As the results are being incorporated into an official version of OPPCAT, it will be implemented as an assessment for learners immediately (when applicable for each respective program).
- At the Department/School and/or Faculty/Unit levels: Within each organization, the OPPCAT is unique to the experiential component of the respective programs. As such the survey results are a shared aspect of general reporting on learner’s experiential processes. In addition, where there are areas of the curriculum or training prior to experiential rotations, whereby improvements can be made, the findings will be shared as a guide for ongoing quality improvement.
- At the institutional (uWaterloo) level: Within the institution, OPPCAT is unique to the experiential component of the Pharmacy program. The OPPCAT Group is in the process of seeking a Creative Commons license OR a form of intellectual property protection through a publication. After that time, the group anticipates widespread sharing at all levels (including other experiential programs outside of pharmacy and healthcare).
- At the provincial, national and/or international levels: We presented at the Canadian Society of Hospital Pharmacist’s Professional Practice Conference (February 2018), Canadian Pharmacy Education and Research Conference (June 2018), and the Council on Licensure, Enforcement and Regulation Conference (September 2018).
Impact of the project
- Teaching: We have already adjusted the following aspects of the assessment tool based on the survey results: rating scale definition, some domain element definitions, glossary document definitions - with more planned over the coming months.
- Involvement in other activities or projects: Aspects of the satisfaction survey that have a direct connection to the curriculum and student preparedness will be reviewed by the respective Experiential team as well as being reflected in future curriculum modification discussions. A subgroup was formed to focus on preceptor/assessor training for developing collaborative training materials.
- Connections with people from different departments, faculties, and/or disciplines about teaching and learning: The survey results and the OPPCAT assessment tool have generated interest in the tool on a national level from pharmacy regulators, residency programs, and schools of pharmacy asking to utilize the tool. Interprofessional education programs are interested in the collaborative nature of the tool.
References
Project Reference List (PDF)