Aptiv & AI Specialist Affectiva Want to Know How Drivers Feel

Understanding a driver’s state of mind is becoming more important as technologies for sensing it grow more advanced. Aptiv, the vehicle technology vendor powering Lyft’s robotaxi trial in Las Vegas, has now partnered with artificial intelligence company Affectiva to bring deeper analysis to the problem.

Systems to detect sleepy or distracted drivers with cameras and other sensors are starting to emerge as key tools for reducing crashes. They are also a component of driver assistance systems, such as Cadillac’s Super Cruise, to make sure drivers keep paying attention when automation is switched on.

Affectiva’s algorithms take it further. Through a commercial partnership agreement the companies announced on January 3, Affectiva will provide Aptiv with software they say can unobtrusively detect how drivers and passengers are thinking and feeling. In addition to increasing safety, this capability will allow Aptiv to enhance the experience of riding in a self-driving car, the two companies claim.

Affectiva grew out of the MIT Media Lab with the goal of making technology understand “all things human.” Its AI platform can recognize complex emotional and cognitive states in people, according to the company. Five automakers plan to deploy it in the next few years, Affectiva CEO Rana el Kaliouby told TU Automotive.

The software complements Aptiv’s own products, which include components like onboard domain controllers that collect and process data from multiple sensors in real time, said David Paja, Aptiv’s vice president of advanced safety and user experience, in an interview. As part of the partnership, Aptiv will acquire a stake in Affectiva.

Affectiva’s software uses machine learning to interpret inputs from multiple sensors, so the software can even tell how many passengers are in the car, what mood they’re in, and what kinds of things they’re doing, el Kaliouby said.

All this eventually could help make trips in self-driving cars more enjoyable, el Kaliouby said. The car could turn off the radio and dim the lights when passengers were falling asleep, for example. But there are more practical applications in the short run, primarily for privately owned and driven vehicles.

For example, the AI can improve on drowsiness detection systems, sensing the problem sooner and with more certainty by looking at multiple factors, such as head pose, facial expressions and tone of voice. This can trigger the car to alert the driver through sound and vibration and warn them to take a break.

Affectiva’s software can also detect when a driver is experiencing a heavy “cognitive load” – a set of conditions such as heavy traffic, children making noise, and complicated driving tasks that could make it harder to drive well. Under those conditions, the car’s software might put off giving non-essential alerts and change the way it communicates with the driver, el Kaliouby said.

The company takes privacy seriously, and nothing that the cameras and microphones inside the cabin pick up is recorded or transmitted, she said. All the processing and decision-making is done onboard, something that is possible with in-car computers now, el Kaliouby said.

For commercial fleets with paid drivers, the software could operate differently, she added. For example, a supervisor could be alerted in real time if a driver is drowsy or distracted and could take action immediately. Fleets could also use the AI to aid driver training.

Aptiv isn’t using the technology in its fleet of Lyft robotaxis, though it does use interior cameras to capture riders’ reactions to the service — with their consent, Paja said. But the company will look at incorporating some of these types of features in future AVs, he said.

Stephen Lawson is a freelance writer based in San Francisco. Follow him on Twitter @sdlawsonmedia.