Research on automotive vision algorithms: focusing on urban scenarios, BEV evolves into three technology routes.
1. What is BEV?
BEV (Bird's Eye View), also known as God's Eye View, is an end-to-end technology where the neural network converts image information from image space into BEV space.
Compared with conventional image space perception, BEV perception can input data collected by multiple sensors into a unified space for processing, acting as an effective way to avoid error superposition, and also makes temporal fusion easier to form a 4D space.
BEV is not a new technology. In 2016, Baidu began to realize point cloud perception at the BEV; in 2021, Tesla’s introduction of BEV draw widespread attention in the industry. There are BEV perception algorithms corresponding to different sensor input layers, basic tasks, and scenarios. Examples include BEVFormer algorithm only based on vision, and BEVFusion algorithm based on multi-modal fusion strategy.
2. Three technology routes of BEV perception algorithm
In terms of implementation of BEV technology, the technology architecture of each player is roughly the same, but technical solutions they adopt are different. So far, there have been three major technology routes:
Vision-only BEV perception route in which the typical company is Tesla;
BEV fused perception route in which the typical company is Haomo.ai;
Vehicle-road integrated BEV perception route in which the typical company is Baidu.
Vision-only BEV perception technology route: Tesla is a representative company of this technology route. In 2021, it was the first one to use the pre-fusion BEV algorithm for directly transmitting the image perceived by cameras into the AI algorithm to generate a 3D space at a bird's-eye view, and output perception results in the space. This space incorporates dynamic information such as vehicles and pedestrians, and static information like lane lines, traffic signs, traffic lights and buildings, as well as the coordinate position, direction angle, distance, speed, and acceleration of each element.
Tesla uses the backbone network to extracts features of each camera. It adopts the Transformer technology to convert multi-camera data from image space into BEV space. Transformer, a deep learning model based on the Attention mechanism, can deal with massive data-level learning tasks and accurately perceive and predict the depth of objects.
BEV fused perception technology route: Haomo.ai is an autonomous driving company under Great Wall Motor. In 2022, it announced an urban NOH solution that underlines perception and neglects maps. The core technology comes from MANA (Snow Lake).
In the MANA perception architecture, Haomo.ai adopts BEV fused perception (visual Camera + LiDAR) technology. Using the self-developed Transformer algorithm, MANA not only completes the transformation of vision-only information into BEV, but also finishes the fusion of Camera and LiDAR feature data, that is, the fusion of cross-modal raw data.
Since its launch in late 2021, MANA has kept evolving. With Transformer-based perception algorithms, it has solved multiple road perception problems, such as lane line detection, obstacle detection, drivable area segmentation, traffic light detection & recognition, and traffic sign recognition.
In January 2023, MANA got further upgraded by introducing five major models to enable the transgenerational upgrade of the vehicle perception architecture and complete such tasks as common obstacle recognition, local road network and behavior prediction. The five models are: visual self-supervision model (automatic annotation of 4D Clip), 3D reconstruction model (low-cost solution to data distribution problems), multi-modal mutual supervision model (common obstacle recognition), dynamic environment model (using perception-focused technology for lower dependence on HD maps), and human-driving self-supervised cognition model (driving policy is more humane, safe and smooth).
Vehicle-road integrated BEV perception technology route: in January 2023, Baidu introduced UniBEV, a vehicle-road integrated solution which is the industry's first end-to-end vehicle-road integrated perception solution.
Features:
Fusion of all vehicle and roadside data, covering online mapping with multiple vehicle cameras and sensors, dynamic obstacle perception, and multi-intersection multi-sensor fusion from the roadside perspective;
Self-developed internal and external parameters decoupling algorithm, enabling UniBEV to project the sensors into a unified BEV space regardless of how they are positioned on the vehicle and at the roadside
In the unified BEV space, it is easier for UniBEV to realize multi-modal, multi-view, and multi-temporal fusion of spatial-temporal features;
The big data + big model + miniaturization technology closed-loop remains superior in dynamic and static perception tasks at the vehicle side and roadside.
Baidu’s UniBEV solution will be applied to ANP3.0, its advanced intelligent driving product planned to be mass-produced and delivered in 2023. Currently, Baidu has started ANP3.0 generalization tests in Beijing, Shanghai, Guangzhou and Shenzhen.
Baidu ANP3.0 adopts the "vision-only + LiDAR" dual redundancy solution. In the R&D and testing phase, with the "BEV Surround View 3D Perception" technology, ANP3.0 has become an intelligent driving solution that enables multiple urban scenarios solely relying on vision. In the mass production stage, ANP3.0 will introduce LiDAR to realize multi-sensor fused perception to deal with more complex urban scenarios.
3. BEV perception algorithm favors application of urban NOA.
As vision algorithms evolve, BEV perception algorithms become the core technology for OEMs and autonomous driving companies such as Tesla, Xpeng, Great Wall Motor, ARCFOX, QCraft and Pony.ai, to develop urban scenarios.
Xpeng Motors: the new-generation perception architecture XNet can fuse the data collected by cameras before multi-frame timing, and output 4D dynamic information (e.g., vehicle speed and motion prediction) and 3D static information (e.g., lane line position) at the BEV.
Pony.ai: In January 2023, it announced the intelligent driving solution - Pony Shitu. The self-developed BEV perception algorithm, the key feature of the solution, can recognize various types of obstacles, lane lines and passable areas, minimize computing power requirements, and enable highway and urban NOA only using navigation maps.
End-to-end Autonomous Driving Industry Report, 2024-2025
End-to-end intelligent driving research: How Li Auto becomes a leader from an intelligent driving follower
There are two types of end-to-end autonomous driving: global (one-stage) and segmented (two-...
China Smart Door and Electric Tailgate Market Research Report, 2024
Smart door research: The market is worth nearly RMB50 billion in 2024, with diverse door opening technologies
This report analyzes and studies the installation, market size, competitive landsc...
Commercial Vehicle Intelligent Chassis Industry Report, 2024
Commercial vehicle intelligent chassis research: 20+ OEMs deploy chassis-by-wire, and electromechanical brake (EMB) policies are expected to be implemented in 2025-2026
The Commercial Vehicle Intell...
Automotive Smart Surface Industry Report, 2024
Research on automotive smart surface: "Plastic material + touch solution" has become mainstream, and sales of smart surface models soared by 105.1% year on year
In this report, smart surface refers t...
China Automotive Multimodal Interaction Development Research Report, 2024
Multimodal interaction research: AI foundation models deeply integrate into the cockpit, helping perceptual intelligence evolve into cognitive intelligence
China Automotive Multimodal Interaction Dev...
Automotive Vision Industry Report, 2024
Automotive Vision Research: 90 million cameras are installed annually, and vision-only solutions lower the threshold for intelligent driving. The cameras installed in new vehicles in China will hit 90...
Automotive Millimeter-wave (MMW) Radar Industry Report, 2024
Radar research: the pace of mass-producing 4D imaging radars quickens, and the rise of domestic suppliers speeds up.
At present, high-level intelligent driving systems represented by urban NOA are fa...
Chinese Independent OEMs’ ADAS and Autonomous Driving Report, 2024
OEM ADAS research: adjust structure, integrate teams, and compete in D2D, all for a leadership in intelligent driving
In recent years, China's intelligent driving market has experienced escala...
Research Report on Overseas Layout of Chinese Passenger Car OEMs and Supply Chain Companies, 2024
Research on overseas layout of OEMs: There are sharp differences among regions. The average unit price of exports to Europe is 3.7 times that to Southeast Asia.
The Research Report on Overseas Layou...
In-vehicle Payment and ETC Market Research Report, 2024
Research on in-vehicle payment and ETC: analysis on three major application scenarios of in-vehicle payment
In-vehicle payment refers to users selecting and purchasing goods or services in the car an...
Automotive Audio System Industry Report, 2024
Automotive audio systems in 2024: intensified stacking, and involution on number of hardware and software tuning
Sales of vehicle models equipped with more than 8 speakers have made stea...
China Passenger Car Highway & Urban NOA (Navigate on Autopilot) Research Report, 2024
NOA industry research: seven trends in the development of passenger car NOA
In recent years, the development path of autonomous driving technology has gradually become clear, and the industry is acce...
Automotive Cloud Service Platform Industry Report, 2024
Automotive cloud services: AI foundation model and NOA expand cloud demand, deep integration of cloud platform tool chainIn 2024, as the penetration rate of intelligent connected vehicles continues to...
OEMs’ Passenger Car Model Planning Research Report, 2024-2025
Model Planning Research in 2025: SUVs dominate the new lineup, and hybrid technology becomes the new focus of OEMs
OEMs’ Passenger Car Model Planning Research Report, 2024-2025 focuses on the medium ...
Passenger Car Intelligent Chassis Controller and Chassis Domain Controller Research Report, 2024
Chassis controller research: More advanced chassis functions are available in cars, dozens of financing cases occur in one year, and chassis intelligence has a bright future. The report combs th...
New Energy Vehicle Thermal Management System Market Research Report, 2024
xEV thermal management research: develop towards multi-port valve + heat pump + liquid cooling integrated thermal management systems.
The thermal management system of new energy vehicles evolves fro...
New Energy Vehicle Electric Drive and Power Domain industry Report, 2024
OEMs lead the integrated development of "3 + 3 + X platform", and the self-production rate continues to increase
The electric drive system is developing around technical directions of high integratio...
Global and China Automotive Smart Glass Research Report, 2024
Research on automotive smart glass: How does glass intelligence evolve
ResearchInChina has released the Automotive Smart Glass Research Report 2024. The report details the latest advances in di...