On April 25th, UniAD (Unified Autonomous Driving), Shangtang’s true end-to-end self-driving solution for mass production, made its debut at the Beijing Auto Show.
After putting forward the industry’s first general self-driving model of integrated perception and decision-making at the end of 2022 and winning the best paper at the International Conference on computer Vision and pattern recognition (CVPR) in 2023, Shangtang took the lead in achieving a key breakthrough in China’s end-to-end self-driving solution from technological innovation to vehicle deployment.
In the first show of the real end-to-end scheme, the UniAD starts in the urban area and runs smoothly on the country road.
as the landing scene of smart driving enters the urban area from high speed, the complexity of the road environment increases sharply.
For the traditional smart driving scheme, the complex urban scene with unprotected left turn is no small challenge, which requires multi-sensor fusion perception and a lot of resources to solve a variety of long tail problems.
UniAD began to learn how to drive on urban roads after it was deployed.
Today, vehicles equipped with UniAD end-to-end self-driving solutions only rely on the visual perception of the camera and do not need high-precision maps.
Through data learning and driving, they can observe and understand the external environment like people, and then based on sufficient perceptual information, UniAD can think and make decisions and drive like people.
Smooth unprotected left turn, fast pedestrian-vehicle mixed traffic lights, independently solve a variety of difficult urban complex driving scenes.
Not only that, but on the non-center-line rural road where it is difficult to break through in the traditional scheme, UniAD can also drive freely, completing a series of difficult operations, including turning left to the bridge at large angles, avoiding occupied vehicles and construction areas, and making a detour of running pedestrians, so as to truly “drive like people”.
In the boarding demonstration of the Beijing Auto Show, there is a rather complicated scene: on the narrow unmarked rural road near Hong Kong, there is a car coming and pedestrians running in front, and UniAD judges that there is enough space ahead to operate, so under the condition of ensuring safety, choose to quickly bypass pedestrians to the left and return to the normal route to complete the meeting.
Solve this complex scene smoothly, just like an old driver.
(UniAD flexibly bypasses pedestrians and completes a ride, really driving like a human.
) Shangtang Jiuying demonstrated the strength of China’s mass production-oriented end-to-end smart driving program with stunning UniAD test results.
UniAD true end to end: the integrated model of perception and decision is the optimal solution.
at present, the mainstream architecture scheme of autopilot algorithm is based on the handwriting rules defined by engineers, and relies on the cooperation of different modules such as perception, decision-making and planning to realize autopilot.
However, because the data between each independent module is transmitted step by step, there is bound to be information loss and error, and the error of the previous module will affect the next, and the information error between multiple modules will continue to accumulate, thus affecting the overall effect of the autopilot scheme.
And after all, the limited rules can not fully cover the infinite complex scenes and long tail problems, the ceiling of traditional smart driving has begun to emerge.
In order to realize the lossless transmission of information from the beginning of perception and break the ceiling of traditional intelligent driving, there must be a new algorithm paradigm, and the end-to-end model is opening up a new technical route for autopilot.
Different from the traditional intelligent driving algorithm, the end-to-end autopilot scheme takes the final driving performance as the goal, deals with the autopilot task in an integrated way, and the whole process from perception to decision-making to control depends on the Transformer neural network model.
With the proposal of UniAD and Tesla carrying FSD V12 version of the real car on the road, more and more companies also began to launch their own “end-to-end” solutions.
At present, many end-to-end solutions in the market build a large model framework in the two modules of perception and decision, respectively, which makes it easier to land.
However, the information transmitted between the two models of perception and decision-making of the “two-stage” end-to-end solution is artificially defined explicit information, and data transmission will still be filtered and lost.
Shangtang Shadow UniAD scheme is the first in the industry to integrate perception, decision-making, planning and other modules into a full-stack Transformer end-to-end model, realizing the integration of perception and decision-making, without the need for abstract and step-by-step transmission of perceptual data, “WYSIWYG” inputs the original information directly into the end-to-end model, and then outputs instructions based on self-vehicle trajectory planning to achieve true end-to-end self-driving.
(true end-to-end is the integration of perception and decision) in the future, the efficient paradigm of end-to-end solutions relying on continuous computing input and high-quality data learning will replace the time-consuming and inefficient mode of relying solely on manpower, and become the key capability of autopilot in the AGI era.
First of all, both the traditional intelligent driving scheme and the “two-stage” end-to-end scheme rely on artificially defined rules to transmit explicit information, and there are errors and loss of information, so it is difficult to restore the external scene completely and accurately.
the most obvious advantage of the end-to-end self-driving model lies in the lossless transmission of information.
the end-to-end model can learn, think and reason based on the original information, and finally understand the complex traffic environment comprehensively like human beings.
And can continue to grow, with a higher limit of capacity.
Second, the data-driven end-to-end solution can generalize the driving ability and skills learned to other scenarios, with faster iterative efficiency, and help car companies achieve the goal that can be driven all over the country more quickly, and the UniAD can now drive easily on both urban and rural roads.
Finally, the end-to-end self-driving model perceives and understands the external environment like human beings.
Pure vision and non-high-precision maps are the inherent talents of UniAD.
It only needs navigation information to drive the car to its destination, which can naturally help car companies to reduce the cost of software and hardware.
The perceptual decision-making integrated model with higher capability upper limit, faster iterative efficiency and lower system cost is the optimal solution of true end-to-end intelligent driving.
Shangtang’s real hard core strength: strong model performance, high-quality data and rich computing power.
Compared with traditional rule-based smart driving schemes, the core advantage of end-to-end self-driving schemes is the strong learning, thinking and reasoning ability of large models, especially the “emergence” ability, while the ability of UniAD end-to-end solutions requires strong model performance, high-quality data and rich computing power.
Resource support.
At the level of model performance, Shangtang Jueyu put forward the industry’s first general model of self-driving with integrated perception and decision-making at the end of 2022.
Driven by high-quality data, UniAD scheme has gone through many rounds of iterations and continuous performance optimization, and is in a leading position in the industry.
Tesla’s FSD V12 version deleted more than 300,000 lines and eventually shrunk to thousands of lines, but this end-to-end smart driving solution is still powerful and growing.
The same is true of UniAD.
Relying on Shangtang’s rich experience in lightweight deployment, the Shangtang Shadow UniAD solution has been deployed since the second half of 2023, and continues to iterate and grow rapidly with the support of abundant computing power and high-quality data.
Not only that, integrated end-to-end solutions such as Tesla FSD V12 are based on an undecoupled model.
UniAD integrates multiple modules into an end-to-end model architecture, and each module can still be monitored and optimized separately.
Compared with the pure black box end-to-end technology, UniAD scheme has stronger interpretability, security and continuous iteration.
At the data level, end-to-end self-driving training requires high-quality video data, mainly a variety of long-tail scenarios, such as retrograde vehicles, cross-crossing non-motorized vehicles, “ghost probe” pedestrians, etc.
, which is very difficult to collect in the real world.
now many enterprises have a lot of data, but most of them are low-quality data of normal driving.
Through real car collection, data pipeline cleaning and screening ability and powerful simulation technology, Shangtang Shadow can create complex scenes by adding obstacles and provide nutrients for continuous evolution and commercial landing for UniAD.
Relying on the world model, Shangtang Jiuyin can continuously generate video data for more refined and complex scenes in the autopilot environment, and then use these data to train UniAD models.
For example, the world model can generate complex urban scenes such as mixed traffic between people and cars, around the island, and even replicate the “8D” urban structure.
(the Shangtang device provides a solid foundation for the efficient training and learning of UniAD and the deployment of real vehicles).
At the computing level, Shangtang has been laying out the construction of AI infrastructure since 2018.
Today, the Shangtang device has laid out a nationwide integrated intelligence network, with an overall computing capacity of 12000 petaFLOPS (gigabit floating-point operations per second, hereinafter referred to as “P”).
With the leading domestic computing resources of Shangtang device, the efficient training, learning and real vehicle deployment of UniAD autopilot program have a solid foundation.
DriveAGI: a smarter and more powerful end-to-end is on its way.
Shangtang Juyu unveiled a forward-looking preview of the smarter and more powerful next-generation self-driving technology, DriveAGI, which is based on a multi-modal large model to improve and upgrade the end-to-end smart driving scheme.
DriveAGI is the evolution of the large model of self-driving from data-driven to cognitive-driven, which transcends the concept of driver, deepens its ability to understand the world, and has stronger reasoning, decision-making and interactive ability.
it is the most close to human mode of thinking, the best understanding of human intentions and the strongest ability to solve difficult driving situations in self-driving, and takes an important step towards complete self-driving.
(the new generation of autopilot large model DriveAGI: perceptible, interactive, trustworthy), not only that, DriveAGI is based on multimodal large models, with strong interactive ability, so that users can use natural language instructions in the cockpit for interaction and driving control, and further achieve perceptible, interactive and reliable experience.
From UniAD to DriveAGI, Shangtang has been leading the trend of end-to-end autopilot, but we will not stop there.
Shangtang Shadow is breaking the boundary between intelligent cockpit and intelligent driving, promoting the structural change of the integration of cabin and driving, and accelerating the smart car to enter the new future of AGI.
, return to the first electric network home page >.
