MASc Seminar: Stackelberg Multi-Agent Reinforcement Learning for Hierarchical EnvironmentsExport this event to calendar

Monday, October 7, 2019 — 2:00 PM EDT

Candidate: Sahil Pereira

Title: Stackelberg Multi-Agent Reinforcement Learning for Hierarchical Environments

 

Date: October 7, 2019

Time: 2:00pm

Place: EIt 3142

Supervisor(s): Crowley, Mark

Abstract:

We explore multi-agent domains containing asymmetric information between agents resulting in a hierarchy of leaders and followers. Leaders are agents that have access to follower agent policies and the ability to commit to an action before the followers. Followers can observe actions taken by leaders and respond to maximize their own payoffs. Since leaders know the follower policies, they can manipulate the followers to elicit a better payoff for themselves. In this paper, we focus on the problem of training agents in a multi-agent setting with continuous actions at different levels of hierarchy to obtain the best payoffs at their given positions. To address this problem we propose a new algorithm, Stackelberg Multi-Agent Reinforcement Learning (SMARL) that incorporates the Stackelberg equilibrium concept into the multi-agent deep deterministic policy gradient

(MADDPG) algorithm, for training agents at different levels. Since maximization over a continuous action space is usually intractable, we propose a method to solve the Stackelberg formulation for continuous actions using conditional actions and gradient descent. We evaluate our algorithm on multiple mixed cooperative and competitive multi-agent environments and the agents trained using our method show promising results in hierarchical domains.

Location 
EIT
Room 3142
200 University Avenue West

Kitchener, ON N2L 3G1
Canada

S M T W T F S
27
28
29
30
31
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
  1. 2019 (246)
    1. December (2)
    2. November (10)
    3. October (19)
    4. September (26)
    5. August (26)
    6. July (40)
    7. June (24)
    8. May (23)
    9. April (35)
    10. March (25)
    11. February (9)
    12. January (10)
  2. 2018 (150)
    1. December (13)
    2. November (25)
    3. October (12)
    4. September (13)
    5. August (7)
    6. July (23)
    7. June (9)
    8. May (6)
    9. April (9)
    10. March (16)
    11. February (10)
    12. January (7)
  3. 2017 (212)
  4. 2016 (242)
  5. 2015 (242)
  6. 2014 (268)
  7. 2013 (192)
  8. 2012 (31)