In-cabin Monitoring Research: In the first nine months of 2022, the installations of DMS+OMS swelled by 130% yr-on-yr with visual DMS/OMS as the mainstream solution

Local manufacturers are keen to deploy DMS and OMS simultaneously

According to the statistics of ResearchInChina, 259,000 sets of DMS+OMS were installed from January to September 2022, a year-on-year spike of 130.1%; the installation rate hit 1.8%, up about 9 percentage points year on year. Among them, 116,000 sets of DMS+OMS were installed by emerging automakers, accounting for 44.6%.

Trumpchi GS8, NIO ES6 and AION Y are the top three models by DMS+OMS installations. Representative models equipped with DMS and OMS simultaneously include AITO M5, SAIC MAXUS MIFA 9, JETOUR X70, Neta U, NIO ET7, Changan UNI-K, Li L9, Voyah FREE, Changan COS Z6, HAVAL Shenshou and so on.

The OMS of OEMs is mainly based on visual solutions, of which radar solutions account for a rising proportion

92.1% of OEM OMS solutions adopt visual solutions, of which radar solutions account for a rising proportion. From January to September 2022, the proportion of OMS solutions using radar increased from 1.9% a year ago to 2.3%. The representative models include Latte, Mocha and Lynk & Co 01, mostly fuel and hybrid models.

For example, Mocha New Energy is equipped with the rear life monitoring system with a radar solution. When a life is sensed to stay in the car within 90 seconds after the car is locked, the system will make the car whistle, and send text messages and APP notifications to remind the car owner. Even if the car owner cannot return in time, the car will automatically open the sunroof to dissipate the heat inside when the temperature exceeds 23°C.

In addition, some manufacturers have introduced in-cabin monitoring solutions combining radar and cameras, both of which can exert their advantages. In October 2021, Continental developed an integrated solution for cabin sensing. The cockpit sensing system combines camera data with radar sensors and intelligent algorithms to cover the entire vehicle interior. After the car is locked, the cabin sensing technology can detect the child left behind through the radar sensor and the stored object classification algorithm, and then issue an alarm. In the future, cabin sensing will be able to measure and evaluate not only object movements but also health parameters such as pulse, breathing rate, and body temperature as well as emotions of drivers and passengers.

In March 2021, PathPartner, an expert in product development and engineering, developed an occupant monitoring sensor (OMS) that utilizes 4D radar technology combined with camera fusion. It's designed to supplement a camera-based vehicle occupancy monitoring system. PathPartner's camera sensor with near-infrared illumination enables reliable performance under different lighting conditions. PathPartner's 4D radar technology can be used to detect subtle movement or even a person's breath in a vehicle's interior, such as a sleeping child under a blanket or the presence of pets.
 
Current R&D focuses on DMS/OMS with a single camera
 
At present, DMS and OMS usually require two cameras. If a single RGB-IR wide-angle OMS camera is used to cover all functions of DOMS (driver and occupant monitoring system), the cost will be cut down on the premise of ensuring the algorithm accuracy.

For example, in September 2022, Eyeris introduced a monocular 3D sensing AI solution which obtained additional depth data about drivers, passengers, objects and surfaces through a single 2D RGBIR image sensor. Eyeris uses proprietary technology that accurately regresses depth information with 3D output from 2D image sensors, which applies to all in-cabin features. It is achieved through rigorous collection of naturalistic in-cabin 3D data to train compute-efficient depth inference models that run on AI-enabled processors. The data generated can be used to map the interior of a car, for example, and accurately determine in three dimensions the location of occupants' face, body, hands, objects and everything else inside the car.

In January 2022, emotion3D and onsemi announced a joint reference design for driver and occupant monitoring system (DOMS). Based on emotion3D’s CABIN EYE AI software stack and the award-winning AR0820AT 8.3 MP image sensor from OnSemi, this new DOMS solution not only replaces the single-task driver monitoring mono/IR camera, but also enables multiple use cases for safety and convenience by employing a single color/IR camera.

The integration of DMS and ADAS promotes the integration of DMS algorithms and chips
 
DMS is gradually merging with ADAS to detect whether the driver is in the best condition to take over the vehicle at any time. At the same time, it also prompted the DMS algorithm to be embedded in the main chip of ADAS, so as to ensure the driver's safety when using intelligent driving function.

For example, ArcSoft cooperates with Qualcomm to provide DMS directly by using part of the redundant computing power of Qualcomm intelligent driving chip on Qualcomm 8155 or Qualcomm's next generation chip.

In January 2022, Seeing Machines announced a collaboration with Ambarella to bring integrated ADAS and occupant and driver monitoring system (OMS and DMS) solutions to the market. This unique technology combination will bring Seeing Machines’ industry leading OMS and DMS technology solutions to Ambarella’s CV2x CVflow? family of edge AI perception systems on chip (SoCs).

Mobileye plans to mass-produce its next-generation EyeQ6 computing platform in 2023-2024, which will be preset with an embedded vision DMS algorithm which is presumably provided by Cipia, an AI computer vision in-cabin automotive solution provider. Cipia has integrated its DMS algorithm into TI TDA4VM SoC and put it on RX5 MAX of SAIC Roewe, and now it has started mass production.

In addition to the field of intelligent driving, the function extension of DMS/OMS is reflected in intelligent cockpits.
 
In addition to the basic fatigue and distraction detection, the in-cabin monitoring system based on a vision solution has expanded functions in the field of intelligent cockpits, such as personalized IVI interface login, personalized cockpit settings, multimodal interaction with the combination of DMS lip reading and voice system, etc., so as to rapidly improve the user experience.

MIFA 9 is equipped with dual OMS on B-pillar. After passengers take seats at the second row of the car, their faces are automatically recognized. According to the account bound to the current face, the personalized settings are synchronized. When the second-row passengers exchange seats, the original seat configuration will be seamlessly applied to the current seats. In addition, the eye movement control function can recognize the line of sight of the passengers at the second row, and control the sunroof, air conditioning, side windows, etc. by combining the armrest screen gestures.

The DMS of Changan CC PLUS supports Face ID, and automatically completes the settings of seats, lighting, automotive applications and so on according to the memory of the user's previous driving habits. It can monitor whether the driver shows signs of fatigue such as yawning and squinting, and remind the driver through voice, light, music, etc.. Combined with "lip reading” function of DMS, "AI Xiaoan" intelligent voice system can recognize the operation instructions spoken by the car owner through lip reading.

The DMS of ZEEKR 001 features Face ID, fatigue monitoring, distraction detection, eye tracking, eyes on road, special action warning, etc.

In the future, with the improvement of DMS/OMS algorithms and hardware, more application scenarios of the in-cabin monitoring system will be unlocked, and product premium will further jump.