The development status and trend of event-based visual sensor

[Introduction]Traditional cameras use a frame-based vision sensor, which obtains scene information frame by frame by presetting a certain exposure time. This frame-based vision sensor has many shortcomings. The event-based vision sensor is a new type of bionic vision sensor, which is more similar to the working mechanism of the human eye, which makes it widely concerned.

Different from the working mechanism and output mode of traditional frame-based cameras, the pixels of event-based vision sensors can individually detect the logarithmic change of light intensity, and output event information including position, time, and polarity when the amount of change exceeds a certain threshold. , has the advantages of low latency, high dynamic range, and low power consumption. Its unique output mode and working characteristics make it especially suitable for occasions with high-speed motion, large changes in lighting conditions, and low energy consumption.

According to Memes Consulting, researcher Xu Wei’s research group from Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences published a review article with the theme of “Development Status and Trends of Event Vision Sensors” in the journal “Liquid Crystal and Display“. Researcher Xu Wei is mainly engaged in the research work on the overall design of new high-resolution space payloads.

This research introduces the development history, classification and working principle, advantages and disadvantages of event cameras, as well as the application of event cameras in the fields of fast motion tracking and monitoring, target recognition, real-time localization and map construction (SLAM) in recent years. Finally, the challenges that still exist in different application fields of the event camera are summarized, and its future development is prospected. The wide application of event cameras will provide new solutions for high-speed motion and high dynamic range occasions where traditional cameras are still difficult. With continuous updating and development in the future, it will be able to play a role in more complex application scenarios.

The development status and trend of event-based visual sensor

Architecture Overview of the Stereo Event PTV Framework

Currently widely used event cameras can be roughly divided into three categories: Dynamic Vision Sensors (DVS), Asynchronous Time-Based Image Sensors (ATIS), and Dynamic Active Pixel Vision Sensors (DAVIS). DVS is the most basic and the first to develop an event camera. The ATIS pixel structure is divided into two parts (A and B), including two photoreceptors, which can provide certain grayscale information while providing event information. The DAVIS camera combines a DVS camera with a traditional Active Pixel Sensor (APS) camera, capable of simultaneously outputting scene events and grayscale information.

The development status and trend of event-based visual sensor

DVS pixel circuit structure

The development status and trend of event-based visual sensor

ATIS pixel circuit structure

The development status and trend of event-based visual sensor

DAVIS pixel circuit structure

The shortcomings of traditional cameras limit the application of some special occasions, and the unique performance of event cameras can provide new solutions to these problems. According to the characteristics of the event camera, it can be applied to some suitable visual application fields, such as fast motion tracking and monitoring, target recognition with small data volume, real-time positioning and map construction (SLAM), and there have been major breakthroughs. .

Event cameras have very prominent advantages: low latency, high time resolution, high dynamic range, and low power consumption. However, in different application fields, there are different defects hindering its development. In many applications, the low bandwidth communication, low weight, low power and high speed of the event camera make it fully meet the demanding requirements for sensors in the aerospace field. If the event camera is applied to the tracking of military targets such as missiles, space shuttles, ultra-fast fighter jets, and naval warships, because these targets have significant appearance characteristics and their moving speeds are generally fast, it may be possible to obtain more accurate information than traditional remote sensing cameras. Good results. In addition, the application of event cameras to space-based space situational awareness also provides a new solution for monitoring satellites and stars. In the future, there will be more rapid development of event cameras in the aerospace and military fields.

This project is supported by the Innovation Workstation Open Fund of Qian Xuesen Space Technology Laboratory (NO.GZZKFJJ20220003). The first author of the study is Fang Yinghong, a master student at the Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, mainly engaged in research on intelligent imaging.

Related Posts