University of Waterloo COVID-19 update

Please see the University of Waterloo’s frequently asked questions for information about COVID-19 and how it has affected university operations.

Please visit the list of modified services if you have questions about university services.

Although the University of Waterloo is closed for in-person events until further notice, many virtual events and presentations of interest to computer scientists are taking place each week at the Cheriton School of Computer Science. Please check out what our students and faculty are doing.

Seminar • Artificial Intelligence | General Interest — Towards Fair and Accurate Peer ReviewExport this event to calendar

Thursday, December 5, 2019 — 2:30 PM EST

Ivan Stelmakh, PhD candidate
Machine Learning Department, School of Computer Science
Carnegie Mellon University

Peer review is the backbone of scholarly research and fairness of this process is crucial for the successful development of academia. In this talk, we will discuss our two recent works on fairness of peer review. In the first part of the talk, we will focus on the automated assignment of papers to reviewers in the conference setup. We will show that the assignment procedure currently employed by NeurIPS and ICML does not guarantee fairness and may discriminate against some submissions. In contrast, we will present the assignment algorithm that simultaneously ensures fairness and accuracy of the resulting allocation. 

In the second part of the talk, we will continue the long-standing debate on single- vs. double-blind peer review. We will discuss the methodology of prior works on testing for biases in single-blind peer review and show that strict assumptions made therein put their tests at risk of being unreliable. We will then present our novel test that accounts for various idiosyncrasies of peer review and provably controls for the Type-I error while having non-trivial power. 

This talk is based on joint work with Nihar Shah and Aarti Singh. No specific background is required to enjoy the talk.


photo of Ivan StelmakhBio: Ivan Stelmakh is a Ph.D. student in the Machine Learning Department at Carnegie Mellon University, advised by Nihar Shah and Aarti Singh.

His research interests lie at the intersection of statistics and the field of learning from people with a current focus on studying and improving the process of scholarly peer review.

The goal of his research is to design tools and methods that help to assess and improve the fairness, reproducibility, and accuracy of peer review. Before coming to CMU, he received a B.S. in Physics from the Moscow Institute of Physics and Technology.

Location 
DC - William G. Davis Computer Research Centre
1302
200 University Avenue West

Waterloo, ON N2L 3G1
Canada

S M T W T F S
26
27
28
29
30
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
1
2
3
4
5
6
  1. 2020 (104)
    1. June (3)
    2. May (17)
    3. April (20)
    4. March (17)
    5. February (25)
    6. January (22)
  2. 2019 (255)
    1. December (21)
    2. November (25)
    3. October (16)
    4. September (20)
    5. August (18)
    6. July (12)
    7. June (23)
    8. May (23)
    9. April (32)
    10. March (25)
    11. February (16)
    12. January (24)
  3. 2018 (220)
  4. 2017 (36)
  5. 2016 (21)
  6. 2015 (36)
  7. 2014 (33)
  8. 2013 (23)
  9. 2012 (4)
  10. 2011 (1)
  11. 2010 (1)
  12. 2009 (1)
  13. 2008 (1)