Master's Defence | Isaiah Naveed Farahbakhsh, Modeling human-coupled common pool resource systems with techniques in evolutionary game theory and reinforcement learning

MS Teams ( please email amgrad@uwaterloo.ca for the meeting link)

Candidate

Isaiah Naveed Farahbakhsh | Applied Mathematics, University of Waterloo

Title

Modeling human-coupled common pool resource systems with techniques in evolutionary game theory and reinforcement learning

 Abstract

Shared resource extraction among profit-seeking individuals involves a tension between individual benefit and the collective well-being represented by the persistence of the resource. In these systems, the decisions of rational agents have been modeled from a game theoretic, and more recently, a reinforcement learning approach. Within game theoretic models, the mechanisms used for learning dynamics are often assumed, and the influence of the type of learning dynamics are not systematically compared under identical models. Models using reinforcement learning techniques are a relatively recent addition to this field, and the literature on multi-agent systems with spatial structure is very sparse. This thesis presents two common pool resource models, each using one of these two different approaches.

In the second chapter, an evolutionary common pool resource game is simulated on a social network with payoff functions that depend on the state of the resource. Model predictions under two types of learning, best response and imitation dynamics are compared and it is shown that best response dynamics lead to an increase in sustainability of the system, the persistence of cooperation while decreasing inequality and debt. Given the strikingly different outcomes for best response versus imitation dynamics for common-pool resource systems, our results suggest that modellers should choose strategy update rules that best represent decision-making in their study systems.

In the third chapter, an analogous model to the one above is presented, however it uses reinforcement learning techniques to inform the agents' harvesting decisions rather than evolutionary game theory. Here, the harvesting strategies of the agents are learned, rather than prescribed a priori, and the payoff function is the weighted sum of a profit goal and a social conforming goal. Preliminary results show that an increased cost of harvesting increases the mean resource level and sustainability of the system. Additionally, the effect of the weight of the conforming goal shows contradictory outcomes, which are highly dependent on the cost of harvesting.

Results from both chapters demonstrate the profound effect human learning models can have on common-pool resource systems, as well as the potential for sustainable outcomes to emerge among a non-hierarchical system of self-interested agents.