> The problem is that a false positive causing emergency braking is also dangerous. So you'd be trading one instance of the car not emergency braking when it should against however many instances of it dangerously doing so when it shouldn't. That isn't an obvious call even before you consider that the human driver can cause the car to brake when it should but can't cause it to not break when it shouldn't.

Yes, building a self-driving car is hard. But if you don't even have emergency braking working, then your technology is too underdeveloped to drive on public roads. Keep trying on the test track.

False positive emergency braking seems less dangerous than running over obstacles. Wouldn't the expected accident be a car crashing into the Uber's back? If the false positive rate is so high that the number of such accidents outweighs the lesser risk, then perhaps Uber shouldn't let the cars drive autonomously with that software.

People can die from rear-end collisions as well, though. It's possible that Uber did the calculations based on the rates of each occurrence and concluded it would be more likely for a fatality to occur from a false positive emergency brake than from running over an obstacle.

Personally, I doubt that they really considered that; I believe that their software was rushed, flawed, and totally not ready for real-world driving. But I don't think it's a certainty that false positive emergency braking is always better than running over obstacles, especially if "running over obstacles" is determined to be a sufficiently rare event.

False positives are only a tradeoff when someone is following. The single most important advantage that driving computers have over humans to make up for their inherent deficiencies is that they don't have to split focus. The computer has no need to check the rear view mirror before going all in on the brakes, it should at any time know the current risk of a rear collision and tune thresholds accordingly.

You hit the nail on the head. People often jump to reactions without looking at bigger picture.

It is entirely likely that Uber engineers evaluated this scenario, and (correctly) decided that it was safer overall to turn off reactionary breaking - and have that function performed by a human in the driver seat. If they hadn't, these cars might instead have been brake-checking people at 100x the rate.

It's easy to say things like "Uber should have waited until better safety features were available", such as the eye tracking suggestions mentioned elsewhere in this thread. But features like that take time - especially if they're development-only features that would have no place in the final product. Every additional safety feature pushes FSD deployment back.

Globally, 1.25 million people die from car accidents annually. That's over 3,000 people per day. For every day that you delay mass adoption of FSD, you are accruing massive amounts of fatalities that could have been avoided.

FSD does not have to be perfect, it's development will cost innocent lives - but if you're optimizing for minimal loss of life, it's the correct thing to do. Reactionary policies do the opposite of what you intend them to do - they cost more lives in the long run.

Uber was nowhere near ready for live tests. It's that simple. Relying on human intervention for split-second decisions where no actions are necessary for long stretches of time is pure insanity.

This accident alone demonstrates just how flawed the system is. This wasn't even an emergency situation to start.

>Relying on human intervention for split-second decisions where no actions are necessary for long stretches of time is pure insanity.

This is literally how all human driving works right now.

Seriously, look around at other drivers while you're on a freeway or interstate sometime.

They're not driving, they're singing along with the radio, or shaving, or putting on makeup, or sending text messages.

They're not Luftwaffe aces with eagle eyes and steely nerves monitoring the fuel mix and oil pressure while scanning the skies for the silvery glint of the sun off the wings of a P-51 that may herald their last few moments on earth. They're bored people doing their boring commute and even when they have their hands on the wheel they're not really paying attention.

They still require some attention all the time. A sense of responsibility and for all our faults we are att least selective about when we aren't paying attention.

Yes, humans suck. But that's still orders of magnitudes better than uber.

What's your criterion of being ready for live tests? Under what conditions is loss of innocent life okay?

A self-driving car is ready for live testing under the same conditions in which it could pass a driver's test, which is the minimum standard for anyone to be allowed to drive legally on public roads.

My road test consisted of driving out of a dead mall parking lot (careful to obey the posted 15 MPH speed limit -- this is actually the hardest part of the test!) turn right onto a street, turn left onto a residential cul-de-sac, make a three-point-turn, and return to the mall parking lot.

These cars could absolutely pass that test.

Globally 1.25 million people die from car accidents annually, but a vanishingly tiny proportion of them would have been saved by the proprietary tech of a taxi company that doesn't even operate in many of their regions, even assuming the technology is ultimately capable of delivering a net improvement on commercial driver fatal accident rates.

Uber isn't a philanthropic research endeavour, and it can't rationalise killing people based on lives it couldn't or wouldn't have saved in the event of its technology actually working. The reason they put tech that reportedly hit things every 15k miles and had a near miss every 100 miles on the road as soon as possible has nothing to do with optimizing for minimal loss of life and everything to do with optimizing for a unicorn valuation.

Which weights off different peoples lives. I dont see how developing a technology factoring in unrelated people getting killed could ever get past a ethics board. Especially if the victims are unrelated to the technology to start with and didnt volunteer to be test subjects.

People getting killed by drunk drivers didn't volunteer for it either. It doesn't matter how you spin it, the metric that matters (there is nuance, but put bluntly) is "How many innocent lives are lost in traffic related accidents from now until [year X]?".

If, given two scenarios:

1. We develop FSD very carefully. 0 lives are lost during FSD development. Ten million lives are lost by the time that FSD sees 100% adoption.

2. We develop FSD less carefully. 100,000 lives are lost during FSD development due to suboptimal performance before it is perfected. Five million lives are lost due to human drivers by the time FSD sees 100% adoption.

Would you really choose the former option? If your ethics board refuses option 2 in favor of option 1, it is they that are mistaken.

> 2. We develop FSD less carefully. 100,000 lives are lost during FSD development due to suboptimal performance before it is perfected. Five million lives are lost due to human drivers by the time FSD sees 100% adoption.

How can you know the future in this case, I would not believe Tesla/Uber (the others) predictions , "trust us, let us kill 1 million people in the next 10 years but after that the number will go down to 100, trust us, we software developers are very reliable at predicting things and our mathematical skill are so great we can be sure even when we use NN that are unpredictable. And if we fail then the next startup will appear, promise the same thing and you have no choice then to give them also a check for 1 million lives to not discriminate between startups.

The difficulty is, you don't know the 100 thousand number is lower than the 10-5 million number in advance. An estimate can be made, but humans have a not unreasonable status quo bias in case like that. Come to think of it, status quo bias on risk is the opposite of the planning fallacy.

That is, without proof the new way will be safer in future by a specific proportion, how many excess deaths can the ethics committee sign off?

(You can make a similar, better founded argument for mass train transit. It is about 10 times safer than driving per passenger mile. So, reductions in safety that moves people from car commuting to train commuting (I imagine safety is expensive and cheaper transit would be more popular) might be worth it. No-one seems to advocate them though.)

>People getting killed by drunk drivers didn't volunteer for it either.

Its not a numbers game. Those are likely two different sets of people. You are saving some people by calculating in, that others get killed instead. It is different to for example, using an experimental treatment option on terminal patients. That would be, saving some of the dying patients but some might die regardless. The scenario is much more similar to the trolley problems cousin the transplant problem. Having someone else, uninvolved killed to save a larger group of people.

Seeing as the automatic emergency breaks were disabled, it seems the product wasnt good enough to detect false positives and was rushed by cutting the safety measure, accepting that putting it in live traffic the car would likely kill people unrelated to the research. It was a trade off between slower development and killing people.

I dont think you will find any ethics board that will green light you killing uninvolved people to save how ever many people. Of those i can think of who did that, quite a few ended up being wanted for crimes against humanity. We have ethics boards for a reason. Without them the ends can quickly justify the means.