Cheriton School of Computer Science Professors Charlie Clarke and Gordon Cormack, Management Sciences Professor Olga Vechtomova, along with their former graduate students Maheedhar Kolla, Azin Ashkan, Stefan Büttcher and Ian MacKinnon, have received the 2019 ACM SIGIR Test of Time Award.
Their paper, titled Novelty and Diversity in Information Retrieval Evaluation, demonstrated the feasibility of using novelty and diversity as the evaluation measures that are optimized by an information retrieval system.
L to R: Professors Charles Clarke, Gordon Cormack and Olga Vechtomova
For a given query, an information retrieval system should return a ranked list that represents both the breadth of information and any ambiguity in the query. For example, if a user were to search for “jaguar,” the information retrieval system might return a mix of documents discussing the British sports cars, the large cat species found in the Americas, and the classic Fender guitar favoured by surf musicians.
Ideally, the document ordering would account for the interest of users. For example, if the cat species were more popular than the sports car, it might be appropriate to devote the first few documents to those discussing cats, before switching to those discussing the cars or guitars. However, creating an information retrieval system that accounts for redundancy and ambiguity presents many challenges.
“Congratulations to Charlie, Gordon, their colleague and graduate students on receiving the prestigious 2019 SIGIR Test of Time Award,” said Mark Giesbrecht, Director of the Cheriton School of Computer Science. “These awards really reflect what has been transformative in computer science research over time. Their paper expands on a thread of related ideas in information retrieval, codifying these ideas into a foundation that accounts for redundancy and ambiguity, and results in a framework that allows them to make a precise distinction between novelty — the need to avoid redundancy — and diversity — the need to resolve ambiguity.”
The ACM Special Interest Group on Information Retrieval — or SIGIR — Test of Time Awards recognize research that has had long-lasting influence, including impact on a subarea of information retrieval research, across subareas of information retrieval research and outside the information retrieval research community. Since the award’s inception in 2014, each year’s winning paper is selected from the set of full papers presented at the main SIGIR conference 10 to 12 years prior.
Professors Clarke and Cormack’s paper was originally presented at SIGIR ‘08. Their 2019 SIGIR Test of Time Award was presented at the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, which ran from July 21 to 25, 2019 in Paris. SIGIR is the premier international forum for presentation of new research results and for demonstration of new systems and techniques in information retrieval.