General Assessment Design Strategies

Overview

As with any assessment guide, we recommend keeping a focus on the basics.

• Align assessment with intended learning outcomes and course activities appropriate to your disciplinary and institutional context (class size, year level, prerequisites, etc.)
• Design for accessibility and inclusion
• Clarify and amplify course policies, explaining “why” and defining terms
• Use guides like this for ideas and options to adapt, rather than seeking strict rules or one-size-fits-all solutions
• Consult with support staff as needed, and watch for upcoming CTE workshops

General Assessment Design Strategies for Avoiding Inappropriate Use of GenAI

Many faculty have good and principled reasons why they want students to avoid the use of generative AI tools altogether. We know that GenAI can be undetectable, and that a game of “cat and mouse” or an “arms race” around detection tools is doomed to fail. Our Guide includes this section about avoiding inappropriate use generally, and then under each assessment type you’ll find pointers about specific strategies that may help with this goal.

• Have a clear generative AI use policy for your course, for individual assignments, and/or for individual assignment components. Be very explicit about the kinds of uses that are and aren’t permitted. For example, “editing” is not interpreted in the same way by everyone. A clear policy might state: “You may use GenAI to correct spelling, punctuation, and basic grammar, but not to generate re-worded sentences for conciseness, clarity, etc.” Keep in mind that GenAI use policies will vary from course to course and students will benefit from as much clarity as you can provide.
• Consider co-creating a GenAI use policy with your students. Engaging students in the course policy creation process supports openness, honesty, trust, human agency, and connection which are important components of integrity. When students and the instructor share perspectives on productive/unproductive and learning-supporting/learning-inhibiting uses of GenAI all gain a broader understanding of potential benefits and pitfalls of GenAI use. When students and the instructor agree together on a GenAI use policy, this policy is more likely to be understood, respected, and remembered.
• Amplify your generative AI use policy. In addition to including your GenAI policy in the course outline, draw students’ attention to it at multiple points during the term (not just at the start of term). Engage students in a discussion about GenAI’s merits and drawbacks, productive and unproductive uses, learning-supporting and learning-inhibiting potential, and its relationship with academic integrity.
• Institute complementary assessment policies such as students must earn an overall passing grade on the in-person course assessments to pass the course, students may be asked to answer questions about their submitted work during an oral debrief, and/or students are expected to participate in an oral debrief following the submission of their assignment, project, exam, etc.
• Engage students in active learning such as discussions, debates, simulations, games, role playing, problem solving, one-minute papers, etc. (see the CTE Teaching Tip: Active Learning) in the classroom, online, and outside of the classroom. When active learning takes place in synchronous in-person or online settings, the opportunities for generative AI to provide meaningful and timely assistance is diminished. When active learning takes place outside the classroom or in asynchronous online environments you can consider requiring work associated with learning activities (e.g. discussion notes, debate outline, problem proofs/solutions, personal reflection on the activity, photographs, etc.) to be included as part of an assignment or in a graded portfolio. 
• Scaffold assessments throughout the course so that students iterate and respond to feedback. Have students “show their work” at each stage of an assignment/project (e.g., submit drafts, revisions, etc.), respond to peer or instructor feedback as they go, explain how they used feedback, why feedback was actioned or ignored, and the impact of feedback (or course-based learning) on the next stage or iteration of their work. Give students credit for this work and signal its value by building it into the assessment scheme.
• Integrate reflection into assessment. Reflection can help students become more aware of the learning process. Have students answer questions such as “What challenges did you face?”, “What insights did you gain?”, “What skills did you develop?” as part of an assessment. Reflection can also help students connect their learning to various aspects of their lives.  Consider building such questions such as “How is this material meaningful to you?” and “How would you use [Concept/Skill X] in your professional or personal life?” into assessments. Keep in mind that reflections can be completed in writing, in a video, in a voice memo, in a class or small group discussion, in an online discussion forum, or in a one-on-one or small group meeting with you or a TA/Lab Assistant.
• Include metacognitive components in assessments. Questions like the following can help students deepen their learning and develop an awareness of important learning process skills: “Describe how you learned this [concept/skill],” “Summarize how you will integrate this [concept/skill] into your current disciplinary knowledge or practice,” and “Explain where you encountered decision points and why you made the choices you did.” You might also consider having students map connections between their learning in your course and other courses, or between your course and extracurricular interests and activities.
• Encourage deep connections between information and ideas. Create questions and assignments that require students to engage in deep thinking to make connections between information sources (e.g., between course materials) and with student-generated content (e.g., class discussions, discussion board posts, student presentations, etc.). Consider assessments that build in ways for students to discover the genesis of ideas, information, and authority in the discipline and how these have changed or been challenged over time.
• Leverage course materials and course activity. Content generated through course activities (such as discussion board posts, group assignments, student presentations, class discussions, guest speakers, expert interviews, etc.) can be referenced in assignment and exam questions. When students create written responses or products, restrict their cited sources to only those materials provided by the instructor or generated within the digital or physical classroom.
• Shift learning outcomes to focus on information literacy and referencing skills. Generative AI can often produce well-written, well-structured, and factually oriented information, but it struggles to accurately output properly cited quotations, paraphrasing, and images. Additionally, it may cite sources that do not exist and has been known to misrepresent research findings. Evaluating students’ ability to find and appropriately reference research sources as well as accurately situate research, critically assess methodology, and identify shortcomings can help students appreciate the limitations of generative AI, the importance of disciplinary knowledge, and discourage inappropriate or unproductive future use.
• Incorporate oral assessments. Oral assessment, on its own or as a component of assessment, encourages student engagement in the learning process. Students can present their research, analysis, summary, etc., orally instead of in written format.  Students can be asked questions about their written answers/projects or to expand upon written answers/projects during an in-person or virtual one-on-one meeting with the instructor.

• Avoid assessing tasks generative AI can do well in an unsecured assessment environment. Generative AI is particularly good at defining terms, answering fact-based questions, summarizing, producing content (text, image, video, audio) that responds to prompts. When asking students to do something independently that you know (or suspect) generative AI can do well, make the learning goal clear to students and consider low-stakes assessments or no-stakes (ungraded) activities that support the development, practice, and mastery of the learning outcome.
• Conduct some assessments in a secured (in-person, supervised) environment. This could include flipping some or all of your classroom, with students reading/watching/listening to assigned learning materials outside of class hours and working on assignments/assessments in-class. If teaching online, consider implementing some synchronous activities and assessments. When learning outcomes are appropriately assessed via tasks that generative AI can do well, consider including in-person tests, assignments, and activities in your course assessment plan.

General Assessment Design Strategies for Engaging Responsibly with GenAI

Some faculty would like students to embrace GenAI (in fact some at UWaterloo are at the forefront of developing aspects of AI and GenAI technologies). Others feel that GenAI literacy ought to be taught along with other forms of literacy in order to equip students with requisite skills for an uncertain future. Certainly, where students are asked to use GenAI or at least permitted to do so in some way, GenAI literacy should be a piece of the explicit skills building. Here are a few things to keep in mind when contemplating advice in this guide:

• responsible use of GenAI includes acknowledging how and where tools are used, with proper citation and/or acknowledgement where necessary
• University data should never be entered into GenAI apps or tools other than authenticated Copilot (not MS 365’s Copilot). See IST’s Guidance on Artificial Intelligence use
• GenAI’s tendency to sound authoritative does not eliminate, and in fact promotes, the need for human fact and accuracy checks
• harms, bias, climate impact, and other ethical issues are serious and should be part of the decision to use or not use particular tools. See Generative Artificial Intelligence (GenAI) Overview.

Immediately below we provide general advice related to engaging responsibly with GenAI, and then under each assessment type you’ll find some more specific suggestions to consider if you are thinking about inviting students to engage with GenAI.

• Explore bias, harms, and ethics issues inherent to generative AI by having students interrogate GenAI outputs.  This has the potential to broaden and deepen learning experiences, create rich connections, activate higher order thinking, support problem-solving, increase awareness about the capabilities and limits of GenAI, and expand GenAI literacy.
• Illuminate the importance of skill and expertise by having students interrogate generative AI outputs. Give students a GenAI output or have them produce a GenAI output on a specific topic, questions, process, etc. and have them fact-check the output for accuracy, depth, breadth, completeness, gaps, etc. At the end of the process have students reflect on the process with such questions as “What skills and knowledge are necessary to effectively interrogate GenAI outputs?,”  “How are such skills and knowledge acquired?,” “Can GenAI outputs be trusted?”
• Unpack assumptions about “neutral” academic writing and its inherent biases by having students prompt GenAI to respond to the same query in different styles (academic, informal, enthusiastic, etc.) and for different audiences (experts, novices, 5 year olds, first year university students, women, etc.). Help students develop their own unique voices and value their own lived experiences by crediting authentic work over perfect work, process over product, and input over outcome.  
• Generate non-traditional examples to support learning. Generative AI can be engaged as a resource in lesson planning and to support student learning by harnessing its ability to provide a wide range of examples (e.g., “Give me an example of a two-legged omnivore” or “Give me an example of a prototype that was difficult to develop”) including examples that can extend the knowledge reach of the instructor and students, engage different perspectives, and spark new ways of approaching course topics.
• Quickly access and/or summarize basic information and factual content. When students are permitted to use GenAI for such purposes, they can devote more time to higher-order thinking, writing, and problem solving in their courses, labs, and research assistant work. This permitted use of GenAI should be paired with a fact-checking/verification process and works best when students have at least a foundational understanding of the subject matter at hand.
• Probe topics from different perspectives. Have students use provided or self-selected keywords to generate summaries or overviews focused on specific topics or areas of interest. Have students compare and interrogate what GenAI outputs when different keywords are used, included, omitted, paired, etc.
• Generate structural outlines or literature summaries that students can edit and incorporate into their final products. Students can use track changes to indicate where and why they edited or expanded the AI-generated content, explain how GenAI influenced the final product, and reflect on the impact of GenAI on their learning experience.
• Generate a case study, scenario, problem, poem, etc. for students to analyze, comment on, interrogate, or solve. To add complexity, students can be asked to do this from a particular framework, point of view, era, etc. or using instructor-created prompts.
• Place a historic figure or a character from a book, film, or play in a different era or setting using generative AI and have students respond to questions, engage in a conversation or debate with the character, solve an era or setting specific problem, etc. The same student, a peer, a peer group, or the class can then react to and discuss the strengths and weaknesses of the GenAI output.
• Collaborate with generative AI. GenAI can be an instant partner for ideas, examples, and feedback. It can “talk” through ideas with students as they ideate, contribute to producing small amounts of written text (when permitted), format citations and bibliographies (although citation/reference management tools like Zotero tend to be more accurate), generate graphics and images that can be incorporated into finished products (not advised if work might be published), suggest illustrative examples, counterpoints, and rebuttals, and give feedback on drafts leading up to the finished product. 
• Help students get past the blank page. Generative AI is good at summarizing, clarifying, organizing, and responding to information. It can help students ideate, organize thoughts, draft an outline, etc. to get them past the blank page.
• Debate and refine arguments and ideas. GenAI outputs can surface debate points, counterarguments and rebuttals as students develop and shape their own arguments. Students can be asked to submit the longform AI-assisted discussions used during ideation and development along with the final product to make clear the role this technology played in the process of augment development and learning.
• Gamify learning. Students can compete with generative AI (and each other) to provide the most complete or compelling responses to instructor-assigned questions. Students can strive to “beat” GenAI by exposing gaps, fallacies, and errors in AI-generated output.  In class or during synchronous online sessions, students can compete (individually or in groups) to accurately respond to questions posed by the instructor in the least amount of time using GenAI and their own knowledge.  
• Check, give feedback, and validate drafts and prototypes. For example, a language generation model like ChatGPT can provide feedback on argument strength and organization, edit for grammatical and sentence structure errors, customize language style for a variety of audiences, “talk” through development problems, debug computer code, or provide case study recommendations that students can check against their own work. Remind students that GenAI can be wrong while also sounding authoritative and that policies and approaches to GenAI use will vary from course to course.
• Discover actual or potential AI applications. Generative AI’s outputs can be fascinating for their accuracy and potential as much as for their inaccuracy, biases, and harms. Discovering applications for GenAI is an important way in which the world is being changed by AI. Challenge yourself and your students to find the most creative, beneficial, ethical, and/or potential uses of generative AI in a discipline, field, or problem area.
• Integrate reflection into assessment. Reflection can support generative AI literacy. Consider having students respond to such prompts as: “What are the merits and drawback of engaging with GenAI in this way or to assist with this task?,” “What was productive and what was unproductive about this use of GenAI?,” “Did you notice or can you identify any (potential) biases, gaps, ethical concerns, etc. related to GenAI, using GenAI in this way, or using GenAI for this purpose?” Keep in mind that reflections can be completed in writing, in a video, in a voice memo, in a class or small group discussion, in an online discussion forum, or in a one-on-one or small group meeting with the instructor or a TA/Lab Assistant.