Scientists should be mindful of their limitations
Pandemic impacts on people everywhere brought a unique opportunity for social scientists to study and forecast changes in society. The Forecasting Collaborative was founded in 2020 by Dr. Igor Grossmann, associate professor of psychology at the University of Waterloo, to evaluate the accuracy of social and data scientists' approaches in predicting social change.
What was the major finding of your study?
The study revealed that social scientists were overall no better at predicting how major social issues – well-being, prejudice toward minorities, or political polarization – would unfold over the first year of the pandemic than the general public or statistics such as the historical average. We did find that consideration of prior data and multidisciplinary research teams were somewhat more accurate.
Since no one can predict the future, how should social scientists engage in social commentary?
Scientists should be mindful of their limitations. Along with consideration of base-rate information and an appreciation of temporal dynamics in their studies, social scientists would benefit from more intellectual humility and consider if their lab studies scale up to what happens "in the wild." In addition, they should avoid predictions without necessary caveats. For instance, a researcher said at a major scientific conference that their theory "predicted everything in the pandemic." It is statements like this that damage the credibility of social scientists.
Is there a risk of eroding public trust in social science research?
Yes, especially if one makes predictions about things one has little knowledge about –one may know how to explain a particular phenomenon but not know how to predict the next steps in this phenomenon. Social scientists should be aware of the limits of their knowledge, and junior scientists should receive better training in prediction methods to mitigate this risk.
Insights into accuracy of social scientists' forecasts of societal change by Grossmann et al. is published in the journal Nature Human Behaviour.
