Source : Mouser Electronics
According to a 2021 study by the National Highway Transportation Safety Administration (NHTSA), over 38,000 automobile accidents occurred in the United States in 2020 — and approximately 95% of those accidents were caused by human error. Advocates for automated driving systems have long argued that automated driving technology will be central to reducing the impact of human error and making driving safer. However, experts agree that fully automated driving systems are still years, if not decades, away, making the human element a lingering part of the equation. As semi-automated driving systems become more and more ubiquitous in passenger vehicles, a relatively new form of human error has reared its ugly head – distracted driving. Concern over distracted drivers with their eyes on their phones, tablets, etc., failing to pay enough attention to avoid preventable collisions and other accidents continues to increase as automated driving systems have become more common. Although confirmed malfunctions in automated driving systems have been sporadic, there are still legitimate concerns about their safety, mainly regarding the element of human error.
How do car manufacturers and regulators ensure that drivers operating vehicles with automated driving systems pay attention to the road? One method is by utilizing technology that has been implemented in commercial transportation truck fleets for years, driver monitoring systems (DMS). DMS uses infrared sensors to monitor the driver’s facial position and eye movements to determine whether they are alert and looking out the front windshield. If the system detects that the driver is distracted or inattentive, it will sound an alert or flash interior lights to warn them of any potential dangers on the road and apply the brakes if the driver doesn’t respond. Legislation has been introduced in the United States, the European Union, and China to make DMS a mandatory feature of new automobiles by the end of this decade. Let’s look at the current state of DMS and where it might be headed.
DMS – Past and Present
As previously mentioned, DMS has been used in commercial vehicle fleet management in the US, parts of Europe, and Asia. DMS has been found to reduce accidents in the commercial shipping and transportation industry and has helped make drivers more time-efficient and confirm their adherence to the stringent safety protocols that driving large commercial vehicles entails. Until recently, however, those systems relied primarily on human oversight of drivers, meaning video captured by DMS would need to be uploaded to the cloud and then reviewed by human watchdogs. The lag time this created between the driver showing signs of distraction and the human monitor activating the driver alert system was a significant drawback in the functionality of those early systems. By utilizing AI, DMS can alert a driver in real-time if their attention is perceived to have wandered.
Most DMS technologies work along similar lines. A camera equipped with infrared LED sensors is mounted on the steering column that monitors the position of the driver’s head (are they looking up at the road or down at their phone?). The movement of the driver’s eyes and the frequency with which they blink (people blink more often when they’re drowsy than when they’re awake and alert). When the DMS determines that the driver isn’t responsive enough to possible road dangers or unsafe behaviors, like drifting in and out of lanes, it activates the driver alert system. The DMS then lets the automated driving system take the appropriate action if the driver doesn’t intervene in time (Figure 1). Lexus was the first auto manufacturer to implement DMS in a passenger vehicle back in 2006, quickly followed by Lexus’s parent company Toyota and Volvo. American car manufacturers like Ford and Cadillac and European companies like BMW have also developed and deployed DMS in some of their models. The expectation is that virtually all automobile companies will be investing in DMS in the next decade.
Figure 1: Driver Alert System Being Activated
DMS: System Implementation and Safety
The primary concern when implementing a driver monitoring system is safety. The system architecture needs to provide real-time responses to various environmental cues that indicate potential danger or unanticipated obstacles. These cues are observed and recorded by multiple sensors that are combined and/or compared to identify outlying measurements in a process often designated as “sensor fusion.” A specialized processor with the required computational power and key features is an excellent place to start this kind of system implementation. The NXP Semiconductors S32V234 Vision and Sensor Fusion Processor provide the platform necessary to build out the rest of a state-of-the-art DMS solution.
The NXP Semiconductors S32V234 Vision and Sensor Fusion Processor provide a fully integrated approach to safety-critical designs, like DMS, by using a heterogeneous mix of a general-purpose CPU, particular purpose processing via a 3D GPU, dual APEX-2 vision acceleration cores enabled by OpenCL™, APEX-CV, and APEX graph and embedded Image Sensor Processing (ISP) to speed HDR and color processing. The CPU includes a quad-core 1GHz Arm® Cortex®-A53 for compute-intensive algorithms and a 133MHz Arm Cortex-M4 microcontroller that offloads the more minor computing-intensive system management functions.
As an added safety and reliability feature, the processors all have error-correcting codes (ECC) to protect data and instructions to the processor caches, the internal 4MB scratchpad memory, and the DRAM memory. The S32V234 also includes an embedded Crypto Security Engine (CSE) with 16KB of on-chip secure RAM and ROM as security is fast becoming a critical factor in protecting internal IP and protecting against external intrusion attacks. The S32V234 Processor is part of the NXP SafeAssure™ Program. This program supports ISO 26262 ASIL B ADAS applications and includes several key DMS elements: pedestrian detection, object detection, lane departure warning, intelligent head beam control, and traffic sign recognition. The S32V234 Processor is also engineered for automotive-grade reliability, functional safety, and security to support vehicle and industrial automation.
DMS: The Present and Future Regulatory Environment
The transition to making DMS mandatory in new vehicles is already underway in the EU. In 2019, the EU’s Council of Ministers enacted a general safety regulation that requires all newly manufactured automobiles available in the EU to include enhanced safety features, including driver monitoring technology. The new rules begin to take effect this year. While they currently only apply to new automobiles with some level of automated driving capability, they will apply to all newly manufactured cars in the EU by 2026. The council estimates that this legislation will prevent over 100,000 automobile accidents in the following decade. Legislation isn’t the only way the EU is investing in DMS. The EU’s New Car Assessment Program, otherwise known as Euro NCAP, now requires new cars to include driver monitoring capabilities to receive a five-star safety rating.
In China, where over 4 million automobiles were sold in 2018 before the pandemic drove down sales, DMS is also taking hold as a required safety feature, albeit more gradually than in the EU. Jiangsu became the first Chinese province to make DMS mandatory in long-haul trucks and vehicles transporting hazardous materials back in 2018 — a similar national mandate is expected shortly. With the US and European markets leading the way, it makes sense for the most populous country to make significant investments in driving safety.
Congress is also taking legislative steps to make driver monitoring systems mandatory in new automobiles. In July of 2020, the House of Representatives passed the Moving Forward Act, a bill to make driving safer by investing in infrastructure and road safety. The bill includes a stipulation like the EU’s safety regulation-making driver monitoring systems mandatory in all newly manufactured vehicles sold in the US. While the Moving Forward Act still needs to be approved by the Senate and the President, similar legislation has been introduced in the Senate by members of the Senate Commerce, Science, and Transportation Committee. The Stay Aware for Everyone Act, or SAFE, directs the US Department of Transportation to research the effect automated driving systems have on driver distraction and the efficacy of DMS in reducing automobile accidents associated with inattentive driving. The research outcome could require regulators to enact and enforce rules that require car manufacturers to include DMS technology in their new model. SAFE would mandate that the research be completed within two years of the bill’s passage and determine DMS’s effect on driver safety within four years.
If DMS is determined to make a substantive difference in driver safety, manufacturers will have two more model years to comply with the mandate. The legislation is designed to be “agnostic” according to its proponents, ” meaning it won’t play favorites with particular technologies or developers. Both pieces of legislation have been held up in the Senate due to the pandemic and other factors, and although no firm deadlines have been set for either bill’s passage, both are expected to receive bipartisan support when they come up for a vote at some point this year.
The Future of DMS
Considering the international interest in instituting checks and balances on the mass implementation of (semi-)automated driving systems and improving driving safety, DMS seemingly has a bright future in tomorrow’s automotive market. Implementing AI has already made DMS more efficient and provided enhanced functionality. The next stage in DMS’s development will likely be incorporating facial recognition software – Subaru already employs FR in its DriverFocus system. The system alerts the driver if they’ve looked away from the road for more than three seconds at a time while simultaneously putting the automobile’s automated driving system into a heightened state of alert in case the system needs to take over for the driver. There are privacy concerns centered around cameras constantly capturing video while vehicles are in operation. Those concerns will probably have to be addressed before facial recognition software becomes a widely accepted feature of DMS. Despite those concerns, if the EU’s safety regulations and legislation in the House and Senate eventually pass in some form, DMS figures to proliferate into passenger vehicles at a substantial rate over the next decade. Get ready for your car to pay more attention to you when you’re not paying attention to the road.
To learn more, visit www.mouser.com
