Department Seminar by Lucy Gao

Please Note: This seminar will be given online.

Beyond sample-splitting: valid inference while “double-dipping”

As data sets continue to grow in size, in many settings the focus of data collection has shifted away from testing pre-specified hypotheses, and towards hypothesis generation. Researchers are often interested in performing an exploratory data analysis in order to generate hypotheses, and then testing those hypotheses on the same data; I will refer to this as ‘double dipping’. Unfortunately, double dipping can lead to highly inflated Type 1 errors. In this talk, I will mainly focus on hierarchical clustering. First, I will show that sample-splitting does not solve the ‘double dipping’ problem for clustering. Then, I will propose a test for a difference in means between estimated clusters that accounts for the cluster estimation process, using a selective inference framework. Finally, I will show an application of this approach to single-cell RNA-sequencing data. This is joint work with Daniela Witten (University of Washington) and Jacob Bien (University of Southern California).