PhD Comprehensive Exam | Yanming Kang, Multi-level TransformerExport this event to calendar

Monday, December 12, 2022 10:00 AM EST

MC 5417 and Zoom (please email amgrad@uwaterloo.ca for the meeting link)<--break->

<--break->Candidate

Yanming Kang | Applied Mathematics, University of Waterloo

Title

Multi-level Transformer

Abstract

Transformer based models have shown strong performance on natural language tasks. However, the quadratic complexity of the self-attention operation limits the maximum input length can be handled. Our proposed model reduces the computational complexity to $O(n\log{n})$ by grouping tokens according to their distance to the target, and summarizing them using strided convolution. In this presentation I will review prior work focusing on the efficiency of Transformers, describe our method, and introduce some preliminary results.

Event tags 

S M T W T F S
25
26
27
28
29
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
1
2
3
4
5
6
  1. 2024 (39)
    1. May (1)
    2. April (2)
    3. March (8)
    4. February (14)
    5. January (14)
  2. 2023 (96)
    1. December (6)
    2. November (11)
    3. October (7)
    4. September (8)
    5. August (12)
    6. July (5)
    7. June (6)
    8. May (5)
    9. April (14)
    10. March (7)
    11. February (8)
    12. January (7)
  3. 2022 (106)
  4. 2021 (44)
  5. 2020 (32)
  6. 2019 (86)
  7. 2018 (70)
  8. 2017 (72)
  9. 2016 (76)
  10. 2015 (77)
  11. 2014 (67)
  12. 2013 (49)
  13. 2012 (19)
  14. 2011 (4)
  15. 2009 (5)
  16. 2008 (8)