AI seminar: Mechanism design using scoring rules

Speaker: Enrico Gerding, University of Southampton

Scoring rules are originally used to reward probabilistic estimates such as weather forecasts, where the score or utility that an agent receives depends on the materialized outcome of the prediction. Strictly proper scoring rule are a class of scoring rule which are designed such that they incentivize utility-maximizing agents to reveal the entire probability distribution truthfully. In this talk I will present a novel two-stage mechanism, based on strictly proper scoring rules, that motivates selfish rational agents to make a costly probabilistic estimate or forecast of a specified precision and report it truthfully to a centre. The mechanism is applied in a setting where the centre is faced with multiple agents who are able to make a prediction at the specified precision, but the centre has no knowledge about the costs of doing so. Thus, in the first stage of the mechanism, the centre uses a reverse Vickrey-like auction to elecit the costs of the agents, and then allocates the estimation task to the agent who reveals the lowest cost. While, in the second stage, the centre issues a payment based on a strictly proper scoring rule. When taken together, the two stages motivate agents to reveal their true costs, and then to truthfully reveal their estimate. We prove that this mechanism is incentive compatible and individually rational, and then present empirical results comparing the performance of the well known quadratic, spherical and logarithmic scoring rules.