How Safe Are Self-Driving Cars After the First Fatal Accident?

[256bits]

Comparing apples to oranges.
According to Virginia tech, self driving cars crash rate is less than national average, per miles driven.
https://www.vtti.vt.edu/featured/?p=422

When compared to national crash rate estimates that control for unreported crashes (4.2 per million miles), the crash rates for the Self-Driving Car operating in autonomous mode when adjusted for crash severity (3.2 per million miles; Level 1 and Level 2 crashes) are lower. These findings reverse an initial assumption that the national crash rate (1.9 per million miles) would be lower than the Self-Driving Car crash rate in autonomous mode (8.7 per million miles) as they do not control for severity of crash or reporting requirements. Additionally, the observed crash rates in the SHRP 2 NDS, at all levels of severity, were higher than the Self-Driving Car rates. Estimated crash rates from SHRP 2 (age-adjusted) and Self-Driving Car are displayed in Figure 1.

 

[mfb]

These rates are for Google's cars aka Waymo. As rough guideline: Level 1 are serious crashes (involving airbags, injuries, require towing, ...). Level 2 accidents have property damage only, level 3 means no/minimal damage.
If we combine the first two we get about 6 per million miles for human drivers and 3 per million miles for Google (but only with 1.3 million miles driven - so just 4 accidents in total).

 

[NTL2009]

These rates are for Google's cars aka Waymo. As rough guideline: Level 1 are serious crashes (involving airbags, injuries, require towing, ...). Level 2 accidents have property damage only, level 3 means no/minimal damage.
If we combine the first two we get about 6 per million miles for human drivers and 3 per million miles for Google (but only with 1.3 million miles driven - so just 4 accidents in total).

If those are Waymo only, there is a huge difference in the approach Waymo is taking and the approach Tesla is taking.

For example, Tesla has no Lidar, something that Waymo considers a necessity.

 

[nsaspook]

Interesting details but I think we all know the root issue here is human decision making on all sides.

I don't know the level of misclassification of objects but it must have been serious for the system to be disabled even for human sized object detection at close range. Too many false positives or even emergency braking when the impact danger is minor compared to possible reactions to the stop (small animal in the road with a semi behind you) can also be dangerous so it should have been a top priority fix for Uber. Waymo seems to have designed systems that handle most cases.

This accident was caused by humans on all sides. The car detected the illegally jaywalking intoxicated human (as an unknown object) in the dark 6 seconds before impact and would have triggered emergency braking almost two seconds before hitting her. Humans made the decision to disable automatic braking instead of fixing the classification problem first, humans decided to put a single 'safety' driver in the car with a impossible job and that human failed as expected from every study in human reaction time to monitor the need for emergency braking while not actively driving. The least responsible entity in the accident IMO is the car, computers and software.

 

[mfb]

The car detected the object and didn't slow down. What exactly did the car expect the object to do? Magically disappear? Turn around and leave the street? While the latter is a likely outcome, it is not guaranteed, and every responsible driver would prepare to slow down as soon as they see the object, and slow down long before an emergency braking is necessary.

If those are Waymo only, there is a huge difference in the approach Waymo is taking and the approach Tesla is taking.

For example, Tesla has no Lidar, something that Waymo considers a necessity.

Tesla's approach is different in several aspects.
- No Lidar
- Build the hardware into a big number of cars
- Start with a driving assistant, but make it available to the public. Collect data from all the drivers using it.

Their software doesn't do as much as Waymo at the moment, although we don't know how much it could do. But they have tons of traffic data even for extremely rare traffic conditions - exactly these unknown unknowns where Uber and Waymo never encountered many of them.

 

[nsaspook]

The car detected the object and didn't slow down. What exactly did the car expect the object to do? Magically disappear? Turn around and leave the street?

Classification of the object early is the key. The computer needs to know some sort of generic physical description of size, mass and direction to calculate the physics of motion for a possible future intercept path. The type of object is also important as bikes are very hard to judge movement orientation wise for computers and move unpredictably at slow speeds. Drivers don't always slow for every object (even people) at the edge of the road or for bikes in the official bike-lane on roads a few feet away from the side of a passing car.

I would ask the same thing of the unfortunate jaywalking victim here about expections. How can you not see an approaching cars headlights in darkness on a lonely street?
Human error on all sides, contributes to 94 percent of traffic fatalities, according to U.S. regulators.

 

[mfb]

Read the report please. The object was at no point anywhere close to anything that could have been a bike lane. As soon as it was on the street it was an object that doesn't belong there: A potential risk.

I would ask the same thing of the unfortunate jaywalking victim here about expections. How can you not see an approaching cars headlights in darkness on a lonely street?

Well, it was clearly primarily the fault of the pedestrian - but it was still an accident the car could have avoided. The report gives some insight why the pedestrian might have missed the car:

The videos also show that the pedestrian, once visible, did not look in the direction of the vehicle until just before impact. [...] Toxicology test results for the pedestrian were positive for methamphetamine and marijuana.

 

[nsaspook]

Read the report please. The object was at no point anywhere close to anything that could have been a bike lane. As soon as it was on the street it was an object that doesn't belong there: A potential risk.

The report lacks important details about first detection position, direction and speed to 1.3 seconds before impact.
Bicycle position at 4.7 seconds after first detection as a unknown object.

(This Uber self-driving system data playback from the fatal, March 18, 2018, crash of an Uber Technologies, Inc., test vehicle in Tempe, Arizona, shows when, at 1.3 seconds before impact, the system determined emergency braking was needed to mitigate a collision. The yellow bands depict meters ahead of the vehicle, the orange lines show the center of mapped travel lanes, the purple area shows the path of the vehicle and the green line depicts the center of that path.)

The report says the cars flawed systems still could have executed the correct procedure of an emergency brake sequence once the potential risk was a high probability of actual impact in time to at least mitigate the severity of the accident at 1.3 seconds and ~25 meters away from the victim at the current speed.

The report states data obtained from the self-driving system shows the system first registered radar and LIDAR observations of the pedestrian about six seconds before impact, when the vehicle was traveling 43 mph. As the vehicle and pedestrian paths converged, the self-driving system software classified the pedestrian as an unknown object, as a vehicle, and then as a bicycle with varying expectations of future travel path. At 1.3 seconds before impact, the self-driving system determined that emergency braking was needed to mitigate a collision. According to Uber emergency braking maneuvers are not enabled while the vehicle is under computer control to reduce the potential for erratic vehicle behavior. The vehicle operator is relied on to intervene and take action. The system is not designed to alert the operator.

NTSB News Release

 

[Vanadium 50]

e. I can't fathom that someone would think it should be acceptable for Uber to be violating basic safe driving rules - and again, it disturbs me that I think I'm seeing people arguing that what Uber did is acceptable.

I agree.

They didn't have a safety driver. They had someone sitting in the driver's seat charged with other tasks. Furthermore, they used to have a second human operator for those tasks, but they went to one operator - and assigned her the non-safety tasks. That probably saved them a buck a mile. And now someone's dead.

 

[mfb]

Bicycle position at 4.7 seconds after first detection as a unknown object.

Or 1.3 seconds after the first detection. An unknown object on the street that does not move notably in the direction of the traffic. I see the following options:
- it is nothing. I would expect this probability to be quite small. Slowing down will delay the car a bit but does not harm otherwise.
- it is a vehicle that broke down or stopped on the street (but not on the right side) for other reasons. This is a dangerous situation, a human might walk around, maybe even in a disoriented state if it was the result of an accident or similar. Slow down.
- it is some other stationary object like parts of a construction site. Slowing down might not be necessary, but it is not a bad idea either.
- it is a pedestrian or similar who has no business being on the street while a car approaches. Slow down.
- it is some slow object that does have a reason to be there (construction work or whatever). Slow down.
- something else? Is there any situation where slowing down could be a bad idea if a stationary or slow object is on the left side of the street?

 

[nsaspook]

https://www.reuters.com/article/us-...driving-car-crash-police-report-idUSKBN1JI0LB

SAN FRANCISCO/WASHINGTON (Reuters) - The safety driver behind the wheel of a self-driving Uber car in Tempe, Arizona, was streaming a television show on her phone until about the time of a fatal crash, according to a police report that deemed the March 18 incident “entirely avoidable.”

A report by the Tempe Police Department said the driver, Rafaela Vasquez, repeatedly looked down and not at the road, glancing up a half second before the car hit Elaine Herzberg, 49, who was crossing the street at night.

The report said police concluded the crash, which has dealt Uber Technologies Inc a major setback in its efforts to develop self-driving cars, would have been “entirely avoidable” if Vasquez had been paying attention.

Vasquez could face charges of vehicular manslaughter, according to the report, which was released late on Thursday in response to a public records request.

According to a report last month by the National Transportation Safety Board, which is also investigating the crash, Vasquez told federal investigators she had been monitoring the self-driving interface in the car and that neither her personal nor business phones were in use until after the crash. That report showed Uber had disabled the emergency braking system in the Volvo, and Vasquez began braking less than a second after hitting Herzberg.

https://www.azcentral.com/story/new...crash-tempe-police-elaine-herzberg/724344002/

 

[nsaspook]

https://www.theverge.com/2018/12/20...iving-car-return-public-road-pittsburgh-crash
Uber’s self-driving cars return to public roads for the first time since fatal crashLet's hope the auto emergency braking function is working this time.

 

[russ_watters]

https://www.theverge.com/2018/12/20...iving-car-return-public-road-pittsburgh-crash
Uber’s self-driving cars return to public roads for the first time since fatal crashLet's hope the auto emergency braking function is working this time.

Perhaps PA could give the car a written driving test to see if it feels like following basic safety procedures now?

Still no manslaughter charges have been filed for the "driver" or car in the incident earlier this year, and presumably both still have their licenses.

 

[mfb]

Self-driving car hits self-driving robot Edit: Seems to be a PR stunt without accident.
_{The robot wars have begun.}

A robot got lost following other robots and ended up on a street where a Tesla pushed it to the side. The driver said he was aware of the robot but didn't brake - probably because no human was in danger. The robot fell over and got damaged.

While it is interesting to learn how the robot made it to the street: The car should have avoided the robot, and it will be interesting to see Tesla's reaction to it.

 

[seazal]

Self-driving car hits self-driving robot
_{The robot wars have begun.}

A robot got lost following other robots and ended up on a street where a Tesla pushed it to the side. The driver said he was aware of the robot but didn't brake - probably because no human was in danger. The robot fell over and got damaged.

While it is interesting to learn how the robot made it to the street: The car should have avoided the robot, and it will be interesting to see Tesla's reaction to it.

looks like a robot suicide.. lol

 

[nitsuj]

Perhaps PA could give the car a written driving test to see if it feels like following basic safety procedures now?

Still no manslaughter charges have been filed for the "driver" or car in the incident earlier this year, and presumably both still have their licenses.

Do you think that manslaughter charges are due and for who?

 

[nitsuj]

looks like a robot suicide.. lol

If you look at the screen shot the robot is grinning a suspicious amount for a solo stroll on the road; i suspect it was under the influence and wandered.

 

[seazal]

Say, what kind of sensors do these self driving cars used in navigations? Is it Ultrasonic? Heartbeat or Exhaust sensors? Shapes detection? I'd like to understand why it missed the robot.

What if you walk like a robot or wear Darth Vader costumes. Maybe when I see any self driving cars in the street, I should just run as fast as I can to the sides or look for any cover.

 

[russ_watters]

Do you think that manslaughter charges are due and for who?

Yes, certainly for the driver - vehicular homicide/manslaughter due to distracted driving is a fairly standard/common charge.

And if the prosecutors are feeling frisky, the supervisor(s) of the test as well. This one is a lot less likely and justified now that we know the driver was watching a video at the time of the collision. Still, the driver was evidently explicitly instructed to violate safe driving principles/laws while safety features on the car were purposely disabled. This set of conditions was created by the test supervisor(s).

 

[russ_watters]

Say, what kind of sensors do these self driving cars used in navigations? Is it Ultrasonic?

Google yields an awful lot of information on this if you search. The short answer is some combination of one or more of video, lidar, radar and ultrasonic.
https://www.sensorsmag.com/components/three-sensor-types-drive-autonomous-vehicles

 

[seazal]

Google yields an awful lot of information on this if you search. The short answer is some combination of one or more of video, lidar, radar and ultrasonic.
https://www.sensorsmag.com/components/three-sensor-types-drive-autonomous-vehicles

I mean, why did it miss the walking robot? What sensors can miss it?

 

[mfb]

Tesla doesn't use lidar but the cameras, the forward radar and (shortly before impact) the ultrasonic detectors should have seen it. Here is an overview over the sensors.

 

[Vanadium 50]

Yes, certainly for the driver

Legally, who was the driver? If I were the person in the driver's seat, I'd be arguing "Hey, I wasn't the driver. My job was to sit up here and fill out forms."

 

[russ_watters]

Legally, who was the driver? If I were the person in the driver's seat, I'd be arguing "Hey, I wasn't the driver. My job was to sit up here and fill out forms."

Sure, but unless the law was rewritten (doubt it), the person with the steering wheel in front of them is a "driver" regardless of what the company tells them their job is.

 

[nitsuj]

Yes, certainly for the driver - vehicular homicide/manslaughter due to distracted driving is a fairly standard/common charge.

And if the prosecutors are feeling frisky, the supervisor(s) of the test as well. This one is a lot less likely and justified now that we know the driver was watching a video at the time of the collision. Still, the driver was evidently explicitly instructed to violate safe driving principles/laws while safety features on the car were purposely disabled. This set of conditions was created by the test supervisor(s).

I lean more towards the person, in this case Uber i guess, that instructed to violate safe driving principles. presumably the driver assumed that is because the car can drive itself and not hit things on the road. which turned out to not be the case.

I guess the question is would a reasonable person follow those instructions.

The test set up doesn't seem to be done with malicious intent, so I see it as "eing done by Uber", the supervisor is just an "agent" of uber. Same as if I have authority to bind my employer to a contract. My employer cannot back out because they don't like it.

 

[Borg]

I think that people are forgetting the first few pages where it seems pretty clear that the pedestrian stepped into the path of the car at night, nowhere near a crosswalk and at the last second in the darkest section of the road - such that it would have been difficult to avoid even if the driver had been paying full attention. If this had happened to me, I have a dash cam that would have shown it as unavoidable just as the forward-looking camera did on the Uber car. But, because there was an internal camera pointed at the driver showing that he wasn't paying attention, suddenly he's guilty of manslaughter? Yes, he should have been paying attention but I don't think that it would have made any difference because it's doubtful that he could have reacted fast enough anyway.

About an hour ago, the Tempe police department released a video of the accident:

Correction on the last second part. Based on Om's video link, she was crossing from the left but was in the section between the lights. I still think that this is far more the fault of the pedestrian and would be a coin flip as to whether it would have been avoidable or not.

 

[russ_watters]

presumably the driver assumed that is because the car can drive itself and not hit things on the road. which turned out to not be the case.

I would hope the driver was instructed that the collision avoidance system had been turned off, but if not that would shift culpability to Uber management.

 

[russ_watters]

I think that people are forgetting the first few pages where it seems pretty clear that the pedestrian stepped into the path of the car at night, nowhere near a crosswalk and at the last second in the darkest section of the road - such that it would have been difficult to avoid even if the driver had been paying full attention. If this had happened to me, I have a dash cam that would have shown it as unavoidable just as the forward-looking camera did on the Uber car. But, because there was an internal camera pointed at the driver showing that he wasn't paying attention, suddenly he's guilty of manslaughter?

This thread is getting old and long and stale, but we've discussed before that according to the preliminary ntsb report, the car detected and tracked the pedestrian for six seconds before impact, meaning the pedestrian was in the clear, and the driver would have had an unobstructed view for about 330 feet - a football field. The lighting is difficult to prove, but a number of people shot stills and video to debunk the idea that it was very dark. That is tough to know though because of varying camera settings. But sure, if it can be demonstrated - and it should be easy for police or the ntsb to do so - that darkness precluded seeing the pedestrian until too late, that would reduce the driver's culpability.

I'm not sure if the pedestrian's jaywalking matters legally or not. There are a number of sites that discuss the civil side (yes a jaywalker can successfully sue a driver who hits them):
https://www.injuryattorneyofdallas.com/jaywalker-hit-car-whos-fault/

 

[Vanadium 50]

Sure, but unless the law was rewritten (doubt it), the person with the steering wheel in front of them is a "driver" regardless of what the company tells them their job is.

I looked it up. The legal term is "operator" and the legal definition is . "Operator" means a person who drives a motor vehicle on a highway, who is in actual physical control of a motor vehicle on a highway or who is exercising control over or steering a vehicle being towed by a motor vehicle.

I think the phrase "actual physical control" is subject to lawyering, and I think Uber should be terrified of the question. If the answer is "the check driver is in actual physical control", then they need to explain why they added job tasks that would require her attention to be elsewhere. This makes them negligent. If the answer is "no, the check driver is not in physical control", who or what is? And the only answer to that is "the car's programming", which is again, Uber's responsibility.

 

[Vanadium 50]

As a PS, I expect a lot of legal wrangling over the difference between "physical control" and just plain old "control".

 

[nitsuj]

As a PS, I expect a lot of legal wrangling over the difference between "physical control" and just plain old "control".

Wouldn't the law certainly say the person in the drivers seat was driver? What's ambiguous is if employee, employer or person who set up the test with anti collision disabled is to blame.

 

[nitsuj]

Yes, he should have been paying attention but I don't think that it would have made any difference because it's doubtful that he could have reacted fast enough anyway.

How can something that didn't happen be used as justification for something that did?

Peeps can be charged for intent alone. I see the logic in your comment the same as saying "Yes they should not have been conspiring a terrorist attack, but they didn't succeed so it doesn't matter anyway."

 

[mfb]

Wouldn't the law certainly say the person in the drivers seat was driver? What's ambiguous is if employee, employer or person who set up the test with anti collision disabled is to blame.

Where is the driver seat in the law (there are cars which have them on the other side) and who thought about defining that explicitly at a time driverless cars didn't exist?

How can something that didn't happen be used as justification for something that did?

"Failing to do X" is typically not illegal if doing X wouldn't have made a difference. If you don't give CPR to a person who is clearly dead it is okay - giving CPR wouldn't make a difference.

 

[nsaspook]

How can something that didn't happen be used as justification for something that did?

Peeps can be charged for intent alone. I see the logic in your comment the same as saying "Yes they should not have been conspiring a terrorist attack, but they didn't succeed so it doesn't matter anyway."

I think the slight difference here is the person hit was breaking a traffic law, jaywalking at a location the city explicitly put a "Do No Cross" sign. I think that's usually a mitigating factor in the blame calculation even if the driver was distracted but otherwise a reasonable (not impaired) driver. An offsetting 'foul' type of situation for a criminal charge or civil liability due to Contributory or Comparative Negligence.

It's not an excuse for UBER disconnecting the emergency auto-braking system because of operational 'false alarm' problems.

 

[nitsuj]

Where is the driver seat in the law (there are cars which have them on the other side) and who thought about defining that explicitly at a time driverless cars didn't exist?"Failing to do X" is typically not illegal if doing X wouldn't have made a difference. If you don't give CPR to a person who is clearly dead it is okay - giving CPR wouldn't make a difference.

It doesn't have to be in the law to discuss how the law interprets itself. I don't find calling the person in the drivers seat the driver difficult. Regardless of what technology was assisting the person in driving; a conscious task in sum. To your point the law isn't written for driverless cars. I find it silly to debate who was the driver in this situation.

The person was breaking the law, only reason it's against the law is because risk of injury and damage. It's okay for me not to give cpr due to not knowing how to give cpr.

ignoring duty of care is against the law. not paying attention while driving a car is ignoring duty of care; because the activity can result in injury / damage.

I'm starting to think uber assumed duty of care in operating the car as a good way to word it. If they did actually gives instructions that moreless mean the same.