Thursday, June 25, 2020 — 4:00 PM EDT
Fairness through Experimentation: Inequality in A/B testing as an approach to responsible design
As technology continues to advance, there is increasing concern about individuals being left behind. Many businesses are striving to adopt responsible design practices and avoid any unintended consequences of their products and services, ranging from privacy vulnerabilities to algorithmic bias. We propose a novel approach to fairness and inclusiveness based on experimentation. We use experimentation because we want to assess not only the intrinsic properties of products and algorithms but also their impact on people. We do this by introducing an inequality approach to A/B testing, leveraging the Atkinson index from the economics literature. We show how to perform causal inference over this inequality measure. We also introduce the concept of site-wide inequality impact, which captures the inclusiveness impact of targeting specific subpopulations for experiments, and show how to conduct statistical inference on this impact. We provide real examples from LinkedIn, as well as an open-source, highly scalable implementation of the computation of the Atkinson index and its variance in Spark/Scala. We also provide over a year's worth of learnings -- gathered by deploying our method at scale and analyzing thousands of experiments -- on which areas and which kinds of product innovations seem to inherently foster fairness through inclusiveness.
Please note: This seminar will be given online through Webex. To join, please follow this link: Virtual seminar by Guillaume Saint-Jacques.
Thursday, June 4, 2020 — 4:00 PM EDT
Can the reported COVID-19 data tell us the truth? Scrutinizing the data from the measurement error models perspective
The mystery of the coronavirus disease 2019 (COVID-19) and the lack of effective treatment for COVID-19 have presented a strikingly negative impact on public health. While research on COVID-19 has been ramping up rapidly, a very important yet overlooked challenge is on the quality and unique features of COVID-19 data. The manifestations of COVID-19 are not yet well understood. The swift spread of the virus is largely attributed to its stealthy transmissions in which infected patients may be asymptomatic or exhibit only flu-like symptoms in the early stage. Due to the limited test resources and a good portion of asymptomatic infections, the confirmed cases are typically under-reported, error-contaminated, and involved with substantial noise. If the drastic effects of faulty data are not being addressed, analysis results of the COVID-19 data can be seriously biased.
In this talk, I will discuss the issues induced from faulty COVID-19 data and how they may challenge inferential procedures. I will describe a strategy of employing measurement error models to address the error effects. Sensitivity analyses will be conducted to quantify the impact of faulty data for different scenarios. In addition, I will present a website of COVID-19 Canada (https://covid-19-canada.uwo.ca/), developed by the team co-led by Dr. Wenqing He and myself, which provides comprehensive and real-time visualization of the Canadian COVID-19 data.
Please note: This seminar will be given online through Webex. To join, please follow this link: Virtual seminar by Grace Yi.