Broadband optoelectronic synapse for low-power neuromorphic machine vision

Background

Conventional computing is based on von Neumann architecture where processor and memory regions are separated and bridged by data buses. This makes image processing in Artificial Intelligence (AI) and Machine Learning (ML) very lengthy and power intensive. Convolutional Neural Networks (CNNs) that are widely used in AI, ML and computer vision tasks (including image classification, object detection, semantic segmentation, video analysis and medical imaging) require dot-product calculation during the convolution process which is very energy and resource-intensive. Therefore, it is anticipated that performance of von Neumann computing architectures will soon reach their physical limits given excessive need for processing data in machine vision applications such as image sensing, feature extraction and neuromorphic computing.

While various types of Memristor arrays have been developed to implement low energy convolution operations, they suffer from issues including sneaky currents (which adds to fabrication costs to avoid it), low accuracy due to non-linearity of I-V curve and a big footprint for realizing convolutional operations. 

Description of the invention

A two terminal, optoelectronic device has been invented by researchers at the University of Waterloo that enables low power convolution computation without the need of shuttling data among different units. The device uniquely deploys lasers with different wavelengths to effectively implement and control image filtering convolution that is required in CNN operations. For the first time, the same device can be used as a broadband optoelectronic synapse for front-end retinomorphic image sensing, convolutional processing and back-end neuromorphic computing. A 3x3 array of the device has been fabricated and successfully tested in a CNN for image processing.   

Advantages

• Low power (37pJ per character detection)
• Fast (enables direct hardware convolution)
• Low fabrication cost (2 terminal – low footprint/high density, optoelectronic device controlled by photons/lasers with no need for avoiding sneaky currents and/or build separation measures)

Potential applications

Fast, accurate and low-power image classification, object detection, semantic segmentation, and video analysis in:

• Robotics and Autonomous vehicles
• Remote sensing and Defense systems
• Augmented Reality (AR) and Virtual reality (VR)
• Medical Imaging
• Natural Language Processing (activities such as text classification, sentiment analysis, and language translation)