Automotive DMS Research: DMS installations shoot up, with a year-on-year upsurge of 360% in Q1 2020

DMS (Driver Monitoring System) is bifurcated into active DMS and passive DMS. Passive DMS judges the driver's state based on the steering wheel turning and driving trajectory. Active DMS, generally enabled by cameras and near-infrared technology, detects the driver's state from eyelid closure, blinking, gaze direction, yawning, and head movements. Only active DMS is studied in this report.

In 2006, the Lexus LS 460 was packed with active DMS for the first time, and the camera was mounted on the top of the steering column cover with six built-in near-infrared LEDs. Automakers are not interested in active DMS, because they believe that it increases the cost of the vehicle and consumers may be reluctant to pay for it. Yet, a train of accidents over the recent years highlight the importance of DMS in ADAS, especially L2/L3. Active DMS started to soar from 2018, with the massive availability of the L2 system and the to-be-spawned L3 system.

Euro-NCAP issued a roadmap for 2025, which requires that new cars must be equipped with DMS from July 2022. China has legislated the mandatory installation of DMS for commercial vehicles, and similar stipulations for passenger cars are just around the corner. The active DMS market is growing prosperous as relevant chip, software and algorithm vendors are vigorously promoting the development of DMS technology.

10,170 units of active DMS were installed in new passenger cars in China in 2019, surging by 174% on an annualized basis. In 2020Q1, the installations skyrocketed 360% year-on-year to 5,137 units amid the wide use of active DMS in the models priced between RMB150,000 and RMB200,000 and the adoption by WEY, Xpeng, Geely, to name a few.

Most Tier1 suppliers have launched total DMS solutions, including Valeo, Bosch, Continental, Denso, Hyundai Mobis, Visteon, Veoneer, etc. Among Chinese companies, the DMS of Hikvision, SenseTime, Baidu, and Dahua Technology have been found on various brand models.

Rise of DMS algorithm developers
DMS is used mainly to monitor drivers’ fatigue and distraction. Yet a larger number of sensors, vision + infrared cameras, and even radars mean availability of more functions, e.g., face recognition, age and gender recognition, emotion recognition, seat belt detection, posture, position and forgetting detection, cabin abnormality detection, and infant detection. Face, gender and expression recognition helps with identity authentication and offers richer interaction between human and vehicle. DMS that is now used just for early warning will enable personalized body control and more functions once it is coupled with ADAS/AD systems.

In Valeo’s case, its AI-driven DMS uses convolutional neural network (CNN) algorithms for emotion recognition and allows unique settings by recognizing the driver’s face.

The versatility, however, cannot be met only by Tier1 suppliers in the short run. That’s why DMS is often co-developed by Tier1 suppliers and algorithm companies.

With years of efforts on development of computer vision technology, ArcSoft Corporation Limited has developed DMS, ADAS and BSD, among others, to meet automobile industry’s needs for vision-related capabilities. The developer’s intelligent driving algorithm business expanded rapidly in 2019, contributing revenue of RMB16.0566 million, a substantial increase as compared to 2018 (RMB2.2021 million), and rang in RMB17.0076 million in the first quarter of 2020, more than the full-year revenue of 2019.  

ArcSoft’s peers such as EyeSight, Smart Eye, FotoNation and Seeing Machine all have been working on development of software algorithms for years. Even so, use of DMS technology still faces plenty of tough challenges:

1. The highest technical barrier before application of vision-based DMS technology is its performance in high or low light which may lead to all white or all black images, respectively. In this case, even the most advanced algorithms won’t work.
2. How to quantitatively define fatigue and drowsiness also creates a bottleneck to use of DMS. The most effective way to measure fatigue and define the relation between fatigue and temperature, dermal resistance, eye movement, respiratory rate, heart rate and brain activity, is measurement of pulse and heart rate variability (HRV), but the technology is still not mature enough.
3. Widely varying signal-to-noise ratio and contrast, obscured and dithered images, and light difference in different weathers and time periods will issue in images of differing brightness.
4. Face detection challenges: in-plane rotation of face; out-of-plane rotation of face; existence of cosmetics, beard and glasses; expression (happy, cry, etc.); obscured face; real-time processing requirements.
5. DMS latency caused by insufficient computing power, communication, etc..
   Interference to users caused by too many false alarms.
6. Few sample databases for training algorithms.

Companies are trying to add more sensors and more powerful chips to solve the problems that vision cameras fail to address in multiple scenarios.

Hardware improvement and cockpit monitoring
In May 2019, OmniVision Technologies and Shanghai Fullhan Microelectronics announced a joint solution for capturing and processing high quality color (RGB) and infrared (IR) automotive interior images, day and night, with a single camera. This solution combines OmniVision’s OV2778 2MP RGB-IR image sensor which offers high sensitivity across all lighting conditions with Fullhan’s FH8310 image signal processor (ISP). The result is high quality video for both machine vision in-cabin monitoring systems (IMS) and viewing applications, simultaneously, providing integrated solutions for facial recognition, the detection of objects and unattended children, remote monitoring, and recording for ride-hailing and robotaxi vehicles. 

In June 2019, Vayyar, an Israeli startup, announced the launch of the first automotive 4D point cloud application on a single radar chip. The single chip with point cloud displays the dimension, shape, location and movement of people and objects, enabling the complete classification of the car’s environment, regardless of bad lighting or harsh weather conditions. In-cabin solutions include seat belt reminders, optimized airbag deployment, gesture control, driver drowsiness alerts, and infant detection alarms.

In September 2019, ON Semiconductor rolled out a complete in-cabin monitoring system inclusive of driver monitoring and occupant monitoring functions. The demonstration includes three ON Semiconductor 2.3-megapixel RGB-IR image sensors. This multi-camera system utilizes Ambarella’s CV2AQ System-on-Chip (SoC) and integrates Eyeris’ AI software performing complex body and facial analytics, occupant activity monitoring, object detection, among others. 

In February 2020, Analog Devices, Inc. (ADI) announced a collaboration with Jungo to develop a Time-of-Flight (ToF) and 2D infrared- (IR) based camera solution to enable driver- and in-cabin monitoring in vehicles. The combination of ADI’s ToF technology with Jungo’s Codriver software is expected to enable the monitoring of vehicle occupants for levels of drowsiness and distraction by observing head and body position as well as eye gaze.  

Valeo has also rolled out its cockpit monitoring technologies, such as vision-based rear seat monitoring solution and radar-based living beings detection solution.

EyeSight Cabin Sense in-car Occupancy Monitoring System (OMS) detects occupants, their posture or whether they wear seatbelts or not, as well as whether any child or object like a bag is forgotten in car or not.

In a nutshell, DMS will be thriving with wide use of L2/L3 automated driving in the upcoming several years. In future, it will evolve into cockpit monitoring system which will play a crucial role even in the era of L4 autonomy.