A recent study led by HCI Games Group investigates how AI tools shape the perceptions of peer reviewers. Led by Hilda Hadan, a PhD Candidate from the Department of Systems Design Engineering, the study found that reviewers perceived AI-paraphrased abstracts in research papers as more honest and compelling than human-written ones.
Taking a unique approach, the study used a blind evaluation to analyze reviewers’ opinions of three versions of research abstracts: human-written, AI-paraphrased, and fully AI-generated. For researchers, particularly beginners and non-native English speakers, using AI can help them produce better manuscripts faster with more time for teaching and supervising.
Peer reviewers often noted that AI-generated abstracts contained generic statements and lacked the depth born of human expertise. “We need to consider that while AI enhances language clarity and readability, the importance of methods, results, and contributions remain paramount in academic reviews”, Hadan cautions.
The research underscores that human researchers must lead the writing process, using AI as a tool to refine and clarify their work while accurately reflecting their original arguments. “The future of research writing isn’t about choosing between human or AI,” Hadan continues. “It’s about finding the right balance where technology enhances rather than replaces human expertise.”
Peer reviewers in the study highlighted the importance of subjective human expressions in writing. They viewed the peer-review process as a collaborative effort to guide authors toward excellence rather than a cold, mechanistic task. “The human touch,” Hadan emphasizes, “remains a vital element of academic writing”.
The study also offers practical recommendations for publishers and journal editors. It calls for transparent guidelines and clear rules to promote openness about AI usage in research publications.
Hadan’s work not only sparks important dialogue about AI in research but also invites the academic community to rethink and embrace technology’s evolving role in scholarly communication.
The study was recently published in the Journal of Computers in Human Behavior: Artificial Humans.
For a more in-depth discussion, listen to the accompanying podcast.