Think soccer when keeping your eye on the autonomous ball

Soccer holds the key to autonomous driving, according to the University of Nevada.

Raúl Rojas, visiting professor of computer science, mathematics and statistics, says soccer is harder than driving – although the stakes are lower. And he should know, having developed robots, called FUmanoids, while he was a professor of artificial intelligence at Freie Universität Berlin.

Driving is more ordered than soccer, Rojas points out. Cars stay on the road and, for the most part, only move forward. Soccer players, on the other hand, need to be able to move in any direction. When playing on a team, he says: "The robots have to recognise each other and move fast."

However, in soccer, there are no fatal errors. "If a computer vision system is able to recognise the position of the ball 95% of the time, that would be very good for soccer but not for driving."

The University of Nevada is building on the soccer robot work for its autonomous driving research. The team wants to reduce costs by relying on video cameras instead of lasers and radars. "If humans can drive with two eyes, a computer should be able to drive with four or five video cameras," Rojas says.

But that will require more advances in computer vision, for example, being able to recognise a traffic light or pedestrian from up to 300 feet away.

Making self-driving cars more trustworthy

At the Georgia Institute of Technology, autonomy and robotics projects are spread throughout the university, including in computer science and electrical engineering departments, while the Georgia Tech Research Institute (GTRI) works with the US government and industry partners on projects. Most of GTRI's research ends up in the defence sector and later may trickle down into consumer vehicles.

Donald D. Davis, principal research engineer for GTRI, says there are three major research areas for robotics: perception, trust and evaluation.

One of the assumptions of GTRI's work is that there will be a combination of manned and unmanned vehicles in military operations. "Object recognition has been done in our world for a long time; now we are in the stages of combining different types of sensor input and moving into recognition of scenes," Davis says. For example, "We'd like to be able to say, 'That's a white car.' It's better if we can say, 'That's a white Nissan Maxima' – and even better would be, 'That's a white Nissan with a guy making a call on his cellphone.'"

GTRI is working on allowing disparate types of sensors to share information at the proper level, while creating autonomous systems that can make inferences about objects and what's going on. While many researchers train autonomous systems by showing them images or video, or else driving them over courses, Davis thinks there's a limitation to this approach. "You will inevitably encounter objects you haven't seen before and need to be able to recognise without training. Then, we want to interpret not just objects in a scene but what's going on.

“An example would be, am I on a highway or in a supermarket parking lot?"

The goal is to be able to have a completely autonomous team of equipment go out into the field, recognise what is going on and make intelligent decisions about what to do. That goal is still a couple of years out, according to Davis.

An autonomous system that could make those inferences is, Davis says, "the last piece of the puzzle."

When it comes to trust, autonomous vehicles need to know what other equipment is trustworthy while proving their own trustworthiness to humans.

When a driver sees another car swerving on the road or cutting off other cars, it immediately makes a judgment about that vehicle's safety and takes defensive action. "We want to teach autonomous vehicle to use observed behaviour of other vehicles to assign a level of trust," Davis says.

This requires creating robust models of human behaviour. For example, if a robot that carries supplies for a combat unit sees all the humans take cover, it needs to understand what to do. In the case of autonomous cars, if a passenger suddenly slumps to the floor, it would be ideal if the car did not simply proceed to its destination.

These are more than perception problems, Davis says. "They're behaviour problems."

GTRI has psychologists with PhDs in computer science on its team, and their studies have shown that humans are quite willing to trust machines; however, if that trust is broken, it's difficult to win it back.

That’s still another reason why autonomous systems need to be so reliable. GTRI's third area of research, testing and evaluation is a big focus. "We've developed tools to stimulate and simulate an autonomous system in faster than real time. We can create situations to throw at it and see how it reacts," Davis says. "We can stress the system by throwing situations you wouldn't expect in real life, looking for the edge cases."

The interface that GTRI uses for testing is JAUS, or Joint Architecture for Unmanned Systems, a standard used by the US Department of Defense that allows the research institute to apply its testing and evaluation tools to other autonomous systems, as well as its own.

In fact, the system can compress real time by factors of hundreds or even thousands. This is highly useful, according to Davis, because, in real-world simulations, most of the time, nothing important happens. Only very occasionally is there an aberration that's worth looking at. "You might run 1,000 hours of simulation and only be interested in looking at 15 minutes of results that you might want to dig in a little further," Davis says.

Prior maps

While the University of Nevada is working on camera-only perception for autonomous driving, and GTRI hopes to create systems that can make inferences about the environment, the Perceptual Robotics Lab (PeRL) at the University of Michigan is taking a top-down approach to sensor fusion, aiming to fuse data very early to develop a coherent, 3D map of the world. Its Next Generation Vehicle, employing a 2014 Ford Fusion hybrid, is a collaboration with Ford that goes back to 2006.

"We approach, from a top-down perspective, how can we fuse data as early as we can. We pull in raw data and think about trying to develop an algorithm so that we are fusing data very early on into a coherent picture of the world," says Ryan Eustice, a University of Michigan associate professor and the lab's founder. "We think that's a more powerful way to interpret confusing situations."

PeRL also is looking at greater use of cameras, as opposed to LIDAR and radar, to reduce costs. Central to the lab's approach is using "prior maps," previously generated 3D maps of the environment, for navigation instead of asking the individual vehicle's systems to parse it in real time. As Eustice and his colleague Ryan Wolcott explained in a paper, "…Rather than using the vehicle’s sensors to explicitly perceive lane markings, traffic signs, etc., metadata is embedded into a prior map, which transforms the difficult perception task into a localization problem."

In practice, an individual's self-driving car might have on-board a prior map for the region in which he usually drives; he could download additional maps as needed. Or, maps could be streamed into the car via a 3G or 4G cellular connection.

The lab's engineers are working to understand what data and attributes need to be represented in these prior maps so that, ideally, third-party vendors could produce them. Says Eustice, "Think of it as Google Maps on steroids."

Find out how to crack the autonomous code at this year's TU-Automotive Detroit (June 3-4).