For over a year now, safety officials have been looking at a tragic accident that killed a woman in Phoenix when an Uber self-driving SUV failed to detect her pedestrian activity.
Now, we have documents coming out from the NTSB showing more on the details of some of what was behind the failure of the technology.
“The most glaring mistakes were software-related,” wrote Aarian Marshall at Wired yesterday. “Uber’s system was not equipped to identify or deal with pedestrians walking outside of a crosswalk. Uber engineers also appear to have been so worried about false alarms that they built in an automated one-second delay between a crash detection and action. And Uber chose to turn off a built-in Volvo braking system that the automaker later concluded might have dramatically reduced the speed at which the car hit (the pedestrian), or perhaps avoided the collision altogether.”
This sad episode uncovers some of the liabilities of new self-driving technologies related to what many call the ‘value learning problem’ – the difference between human intuition and machine-driven strategy.
The value learning problem suggests that machines don’t inherently know what to value, making their cognitive strategic role simpler then a human decision-making process.
In this case, while a human driver would intuitively know, when faced with the scenario, that visually that a pedestrian was crossing outside of a designated crosswalk, the machine doesn’t know that, unless it’s programmed to – and in this case, it apparently wasn’t.
“Uber didn’t tell its car to look for pedestrians outside of crosswalks,” Marshall wrote, citing the following language from NTSB documents: “The system design did not include a consideration for jaywalking pedestrians.”
The omission of jaywalking activity from strategic self-driving operation is kind of astounding, and very lamentable, and represents big liabilities for these new technologies in the future.
We’ll have to see whether planners can include these kinds of additional safety measures in successive versions of self-driving vehicle platforms. If not, we can expect that autonomous vehicles will not become part of the normal scene on the American roadway anytime soon.