Driverless Tech Testers Playing ‘Games’ With People’s Lives

Autonomous technology testers are risking the lives of thousands of members of the public because they trust gamification over military and aerospace simulation systems.

That’s the stark warning of former aerospace systems engineer Michael DeKort speaking exclusively this week to TU-Automotive. DeKort has previously slammed the use of on-road shadow or safety testing of driverless technology where a human test pilot is entrusted to step in to correct any errors made by the robot car. This issue was ramped up further with recent data released by the lead testers using this process, including Waymo, General Motors Cruise, Mercedes-Benz and Uber, revealing the high levels of intervention required to override issues with the current autonomous technology.

Now DeKort claims the autonomous engineers’ obsession with gaming technology is needlessly placing lives at risk. He said: “When people realize they have to train their systems on thousands of accident scenarios by experiencing them thousands of times and the public is going to be a guinea pig to that level, they are never ever going to get close [to proving the technology].

“The odds are on the death of the first child or family. This is not just a possibility, this has to happen. According to their plan, it’s completely unavoidable which is why Elon Musk and Mark Rosekind, the former head of NHTSA and now the safety officer of Zoox, wrote articles saying that people have to understand that the deaths that will occur are necessary for the greater good, which is absolute nonsense.”

Engineers’ egos

He believes much of the problem lies in a mixture or ego and not wanting to rock the boat while big bucks from investors are still pouring into driverless trials. DeKort explained: “These people are clearly intelligent but they have an extremely low exposure and, therefore, an experience set in proper systems engineering, simulation or anything remotely close to this. This is not Twitter.

“What you have is a massive echo chamber that everybody is in on, the autonomous vehicle makers, their investors, government officials at every level. It’s just leveraged trust and it’s going to wind up being the largest case of intelligent people doing the worst possible thing since Columbus.

“I can give you a list of examples, the most important one being they do not do intensive ‘what if?’ or negative testing. Except for the operating system people, nobody else does and that’s not remotely close to what is needed here.

“They are desensitized by games and their wealth and everything else. They have just got in thinking they could innovate everything and that nobody else has innovated a thing and they can’t look to any other industry and so they have got in over their heads and now they are stuck.”

DeKort says he was so frustrated by the lack of understanding of how simulation should be operated that he was driven to set up his own consultancy, Dactle. He admitted: “I clearly have a conflict of interest which I tried to avoid for a year and a half by getting other companies, especially the majority of the simulation companies in this space to understand what they needed to do. The ones that did understand it chose not to make the change needed because they wanted to wait for their customers to figure it out and pay them for it.”

So, what should autonomous testing companies be doing? DeKort said: “The answer is simulation, proper simulation and what’s available to the industry right now is gaming based and not remotely close to the quality necessary to replace the majority of public shadow and safety driving.

Military grade simulation

“Aerospace and defense technology that has been around for more than 20 years and they have done every bit of this and there is absolutely nothing here that they haven’t done and been doing well for decades. It’s just that these people don’t get it – they immediately think, when I’m talking about this, that I’m talking about air travel which is non-chaotic. However, DoD (US Department of Defense) simulated wargames are every bit and, more, intense than these folks would do in an urban environment. That’s because of two reasons, one is that they drive off the roads at some point and, two, that there is fighting going on and shelling and all the rest of it.

“The [current industry] problems are in several areas and one of them is the real-time action of these systems are based on gaming engines and that means they don’t run fast enough. Number two is their architecture is restrained by the way they write their operating systems and they cannot load up the high-fidelity models that need to be loaded up to run in real time. So, the models they are using are imprecise which will cause false confidence.

“Let me give you an example: if you don’t use the exact car, the exact tires and the exact road surface in the exact weather, when you are in a scenario that pushes the performance curve of any of those objects or some combination, what will happen is the machine learning will assume it has certain capabilities in that car in that situation that it will learn and apply in the real world. So, what will happen is there will be accidents that shouldn’t have occurred or there would be more injuries than needed to occur. That’s because the simulation wasn’t precise enough for the machine learning to have a proper plan.”

On the up side, DeKort claims that when autonomous testing companies do see the light and begin using military and aerospace grade simulation, “You can eliminate 99.9% of the shadow and safety driving and, therefore, 99.9% of casualties. I’m not going to say that with the methods I prescribe there will be zero accidents because that would be unreasonable but it would be extremely rare that somebody would wind up getting hurt.”

— Paul Myles is a seasoned automotive journalist based in London. Follow him on Twitter @Paulmyles_


2 comments

  1. Margaret Steciuk 25th March 2019 @ 8:22 pm

    I’m happy to see someone who is NOT on the AV bandwagon without a critical eye. The concept of automated vehicles is so sexy it has overwhelmed many smart people’s good senses. Thanks for being bold enough to publish this article. The Emperor is perhaps naked.

  2. Patrick Sullivan 8th April 2019 @ 4:09 pm

    Mike’s comments are right on target. Coming from a military background with simulation testing he has the right approach. I am wondering why the industry has not asked for his assistance and support. Why risk lives.

Leave a Reply to Margaret Steciuk Cancel reply

Your email address will not be published. Required fields are marked *