Training Self-Driving AI Takes Humans & Automation

Stephen Lawson

---

7th August 2018

Autonomous driving is a multifaceted engineering challenge but, among other things, it’s a massive object-recognition problem.

Scale, a San Francisco start-up that helps train artificial intelligence platforms for many online services, is now addressing the steep demands of self-driving AI systems.

AI and machine learning are making strides in performing the complex planning and decision-making functions involved in getting from one place to another safely. Companies such as Nvidia and Tesla are building ever more powerful hardware to handle those tasks. However, interpreting sensor data to recognize roads, cars, pedestrians and more is at the heart of the problem and AI can’t do that alone.

When cameras, radar and LiDAR deliver readings to an AI system, the computer can only recognize objects if it’s been trained to generalize, Scale CEO Alexandr Wang told TU-Automotive. If the system encounters a car with a slightly novel design element, the AI may not know it’s a car at all.

For example, a model that had a rear turn signal with three lights in sequence instead of one flashing light was a challenge for AI systems. So were the dockless electric scooters that began to show up in some cities recently, Wang said. Even the slightest change in a car’s design from one model year to the next can make the AI’s recognition a little bit less accurate.

Scale takes in sensor data from AV development vehicles operated by its customers, such as GM Cruise, Uber, Lyft and nuTonomy, and labels the objects that the vehicles encountered. Software does some of the work, but it needs to be double-checked. “Every piece of data gets at least looked over by a human,” Wang said. (Most of the work is done by contractors, he said. Scale doesn’t say how many there are, or where they work.)

Then Scale sends the labeled data back to the customer, which uses it to train the AI to better recognize objects. AV companies need new data every time they modify the array of sensors on a vehicle, as well as regular updates to keep up with new kinds of cars and other objects. People know what they are, but AI often doesn’t.

“Ultimately, what we’re providing them is the insights that humans have for free on top of the imagery and other kinds of sensor data,” Wang said.

Automated driving is one of the most demanding uses of AI training data that Scale works on, he said. The company also labels things like couches in images on Pinterest and brick walls in Airbnb photos, which helps those companies put relevant content and ads in front of users.

Unlike those companies’ machine-learning systems, automotive AI platforms need to do image recognition in real time. Also, driving systems need to achieve greater accuracy because the stakes are that much higher. “They can never be accurate enough,” Wang said.

Also, driving systems aren’t hidden in the background like the AI behind most web-based services.

“The way the models and the automated systems and the AI performs is front and center. If you’re sitting in a car, you’re going to see how it ends up driving,” Wang said.

AI isn’t even close to correctly identifying every important object by itself, said Navigant Research analyst Sam Abuelsamid. “I don’t think that is going to happen any time in the near future,” he said. So companies like Scale, as well as others with similar objectives, such as MightyAI, will have work to do for years, he said.

Even small changes like putting bits of tape on a stop sign can throw off an automated recognition system, so the software needs to be resilient enough that it can recognize things anyway, Abuelsamid said.

“It’s actually surprisingly easy to break these systems, so it’s crucial that you train them with a very careful data set,” he said.

— Stephen Lawson is a freelance writer based in San Francisco. Follow him on Twitter @sdlawsonmedia.