Research from the University of Waterloo shows that Artificial Intelligence (AI) can improve academic writing and facilitate peer reviews. For researchers, particularly beginners and non-native English speakers, using AI can help them produce better manuscripts faster with more time for teaching and supervising.
The study, led by Hilda Hadan, a PhD Candidate from the Department of Systems Design Engineering, found that reviewers perceived AI-paraphrased abstracts in research papers as more honest and compelling than human-written ones.
In a blind evaluation, reviewers were asked to rate three versions of research abstracts — human-written, AI-generated, and human-written but AI paraphrased. The latter got the highest rating, but overall reviewers prioritized valid methods, robust results and the significance of the research — its contributions to academia, society and industry.
“The findings come with important caveats,” Hadan said. “The study shows that AI can improve language and readability, but the reviewers noted that AI-generated abstracts often had “generic statements” and lacked insights born from human expertise.”
The study emphasized the need for the “human touch” in academic writing. Using AI can help make the content clearer, but researchers must have the final say — reliance on AI will reduce the validity of their work and the arguments they present.
The study also provides helpful suggestions to publishers and journal editors on how to create rules and make sure everyone is open and honest about using AI in research publications. Specifically, to educate reviewers on biases in human and algorithmic AI detection methods and emphasize evaluating manuscripts based on scientific quality, because neither method can reliably detect AI use in manuscripts. And to establish clear ethical guidelines for AI use in research and mandate detailed AI disclosures, including AI prompts, example outcomes and specific sections where AI was used.
“The future of research writing isn’t about choosing between human or AI,” Hadan said. “It’s about finding the right balance where technology enhances rather than replaces human expertise.”
The study, recently published in the Journal of Computers in Human Behavior: Artificial Humans, was funded by the HCI Games Group and explores views gathered from peer-reviewers at HCI conferences.
The research team discusses the study and its results in this podcast.
Researchers in the photo, from left to right: Hilda Hadan; Dr. Eugene Kukshinov; Lydia (Lili) Choong; Dr. Lennart Nacke (supervisor); Federica Gini; Joseph Tu; Sabrina Sgandurra; Dr. Reza Hadi Mogavi; Dr. Sebastian Cmentowski.
Photo credit: Derrick Wang.
Hilda Hadan (far left), a PhD candidate in the Department of Systems design Engineering, with her fellow researchers.