From: David DeVidi, Associate Vice-President, Academic
To: Instructors and TAs
Date: April 10, 2023
- Turnitin results will include AI detection tool results starting lastweek
- Instructors who use Turnitin should treat these results with caution until we have better evidence of reliability
- Your associate deans are available to discuss cases of concern
Colleagues,
Turnitin released an artificial intelligence detection tool on April 4, and it went live at Waterloo on April 6. The tool has been trained on GPT 3 and GPT 3.5 (which includes ChatGPT) and Turnitin claims that it has impressive accuracy.
Instructors who use Turnitin will start receiving an AI detection score (entirely independently generated from Originality score) as part of its Similarity report. The report on artificial intelligence will only be visible to instructors and administrators, and only from within the Originality report. Of course, if you do not have a statement in your course outline notifying students that you will be using Turnitin you will not have access to these reports (because Turnitin is not activated for your course).
We are writing to urge instructors to exercise caution if considering the results of the AI detection score.
- As Turnitin itself notes, “we must emphasize that the percentage on the AI writing indicator should not be used as the sole basis for action or a definitive grading measure by instructors.” A high percentage of “likely AI content” does not in itself warrant a determination of academic misconduct. The tool has not been widely tested by anyone besides Turnitin. Other existing AI detection tools have a high rate of false positives (i.e., incorrect determinations that human-produced prose was generated by AI). False negatives (i.e., suggestions that AI generated text is likely human) are also possible.
- If you are using Turnitin, you’ll have a statement in your course outline to that effect. We may modify the wording of the recommended statement in the future, but for now we will consider that statement to also cover the Turnitin AI Detection tool. However, your associate dean and appeal committees should treat its reports with skepticism until we have better evidence of reliability.
- A high score from the AI detection tool may warrant a conversation with the relevant associate dean in your Faculty. They may suggest approaches for gathering other relevant evidence (e.g., meeting with the student to assess their ability to engage with the assignment’s subject matter).
In early February, we sent out a memo encouraging instructors to, when possible, use pedagogical approaches that avoid the need for detection tools in this rapidly changing landscape. See related FAQs or consult the other resources linked to in that memo. We also hope for a more thorough investigation on the usefulness of detection tools, among other topics, by the Standing Committee on New Technology, Pedagogy, and Academic Integrity that is also mentioned in the memo. We are still soliciting expressions of interest for this committee, which will have its first meeting (we anticipate) in early May. Please let me know if you are interested in being on that committee.