Here’s the video.
The Uber driver did not seem to be paying much attention. I wouldn’t have either. Nor would you. The point is, and it’s obvious, there was no time for a human to react.
However, one might easily have expected radar to perform better. In this instance, self-driving fared no better than humans. Not a great performance.
Would a Waymo vehicle have done better? I suspect so, but I do not know so.
Looking ahead: Radar will get better. Humans won’t.
That’s the all-important bottom line.
Will this slow down self-driving deployment?
Mike “Mish” Shedlock
medical and aviation software contains hundreds to thousands of lines of source code running an algorithm that specifies a specific action clearly. Self-driving car code is thousands to tens of thousands of lines of code, where the user has no idea of how the software is making decisions, because the decisions are based on tens of megabytes to gigabytes of training data. There is a chasm of risk between a few well defined steps in a medical/aviation algorithm and a cloud of data whose interpretation is known only to the machine learning algorithm. Very, very different. BTW I used to work on the Honeywell team that maintained the space shuttle engine controllers in addition to other similar mission critical software. My past experience in eliminating failure in complex systems is likely what compels me to be so vocal in carefully considering when high risk technology is “ready”.
@JeffD – Yes, you’re asking the standard questions for device control software. For example, medical and aviation are fields with software in such high-reliability environments. Car companies have built systems for such environments for some time now. Check the Internet for answers to how these things are done.
To give a flavor of this sort of engineering, think redundant systems, often of completely different tech. E.g. Use both LIDAR and camera. One dies? The other is used to gracefully fail (pull to the side of the road in the case of a car). Also, reliability/testing is one reason Waymo talks of rolling their driver’s 10-plus digit odometer.
And this is the real problem, Let’s say it is a simple bug. How often will software be updated, introducing *new* bugs? How many simple bugs exist in the system? How often does technology have a ‘shutdown’ level failure event compared to a human (i.e. humans passing out for no reason). Technology is in *total* control here, and technology fails all the time. How often have you had to cycle your computer, cell phone,, because it mysteriously locked up? How can one insist that the computer software will, at some distant point in the future, work *unfailingly* when the set of possible inputs is an undefined open system, and not a well-tested closed system? These are the questions that have to be answered in order to make the claim, with a straight face and honesty in one’s heart, that self-driving cars (with no steering wheel!) will have a better safety record than human drivers.
Just noodling some more about this Uber kill: I’d not be surprised if it’s found to be a simple bug – not related to AI or the sensors or anything particularly groundbreaking in the tech. Instead, just some edge case that fell through the cracks because a loop missed the first or last item in a list. The kill seems a classic single-point-of-failure bug.
It’s interesting that so little information has come from Uber on this. They should have published something explaining what went wrong by now. The engineers can’t still be trying to find out!?! They would have replayed this event plenty of times by now, and it’s not like anyone on their team has more important things to do.
More stupidity mixed with wishful thinking, lack of sensitivity, greed and arrogance. The driver was the BACKUP! Which means, he/she is expected to react AFTER it became clear that the automatic system had failed. So, his/her reaction will necessarily be SEVERAL HUNDRED MILLISECONDS to A FEW THOUSAND MILLISECONDS TOO LATE.
“Let Uber build it’s own testing grounds with it’s own money. Oh, I forgot Uber isn’t a profitable enterprise. What gives Mish? I thought you were the bastion of free enterprise? Why should publicly funded roads be used for Uber’s experimentation? ”
It’s straight out of the neoliberal playbook – gut regulations, corporations are people, all people are equal but some are more equal than the others, wages quashed with anti-American so-called free trade deals etc. The whole show is kept going by expanding the credit bubble. Which they pretend to oppose, thereby adding hypocrisy to a mix that already consists of arrogance, callousness and dishonesty.
Last night driving home I reached the same conclusion – the camera lens was not open enough. Look at the adjacent left lane – little light from the headlights. The victim was wearing reflective sneakers yet they don’t show up until we can see her.
Someone drove the street the video is from using a better camera:
https://www.youtube.com/watch?v=CRW0q8i3u6E&feature=youtu.be&t=32
What the video shows is irrelevant. Its a camera looking straight ahead. It does not show what radar and lidar were supposed to detect. It does not show everything that a human driver would see and respond to in the 10 seconds prior.
If the backup driver is supposed to be there for safety of the public, why was the driver looking in his lap?? It wasn’t a stick was it?
If Uber needs to test on public roads, then who gets to decide what the qualifications of the backup driver are? (You need a special license to drive a truck, to drive a motorcycle, to drive a bus). If the driver is there to protect the public and needs to be able to take over from a computer that might have made a poor decision or a system bug, they are going to need excellent reaction times and serious training.
Mish you stated that “Where Uber failed a driver failed as well”. But the truth is more like Uber failed in its technology and in its deployment of human backup.” I really hope that you have invested a lot of your and your client’s money in this technology sector and are just pumping it to raise the stock prices. At least then your rather startling jump to the conclusions you have drawn would be understandable….
Sorry Mish, respect your work but have many doubts as to whether this dashcam video represents what a human eye would have seen.
Let Uber build it’s own testing grounds with it’s own money. Oh, I forgot Uber isn’t a profitable enterprise. What gives Mish? I thought you were the bastion of free enterprise? Why should publicly funded roads be used for Uber’s experimentation? Why should public safety be jeopardized because Uber can’t afford to build it’s own real life testing grounds?
Believe it or not there is a deeper issue here and it concerns something called freedom but I digress.
For fun I ran thru Gimp an image from the video slightly before when the pedestrian shows up.
One can *clearly* see the human. (insert sense of humor apology) She’s the red below the round light at pixel 618×417. This is image 77 of 649 in the 720p YouTube video.
https://s3-us-west-2.amazonaws.com/maven-user-photos/mishtalk/economics/ChlohB3OekqgEIhOtbeWYw/-D2pNLw6OUKIyOmAk8zSrw
Apparently Uber has a less than stellar reputation of good citizenship. So it will be interesting how quickly we see *all* the sensor data from this incident at Github for use by other driver development teams.
The Tempe police chief was probably trying to do her best when commenting on the video, but her statement was unfortunate. As Mish has shown, cameras can make HDR images that can exceed human eye capabilities, but as others here have pointed out, this video does *not* come from such a camera system. The video makes it appear that if the Uber driver were solely using this video feed for obstacle detection, it was overdriving its headlights. And others have pointed out that LIDAR (or radar) data is probably what wants to be examined, not this video.
My own wild guess is that the driver saw but effectively ignored the pedestrian. This makes Uber, a company, liable for whatever a human driver in such a case would be liable for. That the pedestrian was one bulb short of a light, so to speak, may be relevant or not. Lawyers, speak to us on this subject.
No one here has mentioned the road builders apparent goof in constructing purely decorative “walkways” in the median. I’m sure a really good Uber defense lawyer could use them tactically, if not strategically.
If you cannot test, then you cannot have the technology because idiots will preach “it’s never been tested”. I really wish people would think before posting stupidity. Alas, I ask too much. We had controlled testing, with a driver behind the wheel. Is there any other way to test? The technology failed but the person J-Walking at night, in dark clothes gets about 90% of the blame. People do stupid stuff, get hurt, idiots scream.
Public roads should not be used to test technology!!!!!!!!!!!!!!!!!!!!!!
How the F do we prove the technology if not on the road? And by the way, where Uber technology clearly failed, a driver failed as well. Fools don’t want this technology to succeed, they do not even want it tested. The track record so far – other than Uber, has been exemplary. That is the indisputable bottom line.
https://youtu.be/t0ogGxJi8fI
Stupid argument. The driver was the backup and therefore, his reaction time would be necessarily be slower than if he was not the backup. In such situations, literally every millisecond counts, and as the backup, he would necessarily be a few hundred milliseconds too late.
But the on-board computers were there. And they can have perfect memory and insight into what happened if programmed correctly. Unfortunately, what the media will never ask is “what inputs did the computer get, and where on the decision tree did it fail?” Was the sensory input incorrect? Why? Can it be fixed? Affordably? Did the computers get the correct input? Was the processing time just too long to react in time? Can that be fixed? Affordably? For those who think computers can react faster than humans in this type of situation, you’re horribly wrong. What humans are incredibly good at is detecting and assessing anomalies and taking action instantaneously. It’s inate in anything that evolved in an eat or be eaten world. This was an instantaneous flick of the wrist to the left situation for a human.
What I love is how well this story illustrates *why* the whole “no-privacy-cameras-everywhere” society is a really bad idea.
Not one of us was actually there – in any capacity. Not one of us knows exactly what either individual in this case saw, or was this thinking, or what else may have been going on nearby that had either of them thinking about something else.
And yet all of us think we can look at a few snippets of videos and know for certain what the truth is. We all claim to know with certainty what *we* would have done, or how *we* would have reacted – and that allows us to, in many cases, point our finger of shame at whomever we believe to be the culprit – the self-driving car; the distracted drive; the pedestrian crossing the street out of a crosswalk and in dark clothing at night.
The reality is that everyone who watches the videos has their own thoughts, experiences, and biases in play when they do. And all of that influences what they see – and what they don’t. And how they feel they would have reacted or what they would have done. This is the same reason eyewitness testimony is some of the *worst* testimony.
It was an accident… They happen. Distractions happen – and not just from cell phones. Drivers can be distracted by any number of things that existed long before cell phones.
So with one brief glint, in the dark, in the first second, you would have postively identified the threat, determined it’s path, and taken evasive action early enough to avoid a collision with something which is only visible again directly in front of your vehicle after second two, and colliding with it in second three?. Do you fly fighter jets? Like all the time when you are on the road too? As for the rest of us mere mortals, this person made a deadly decision to cross a road in the dark in front of an oncoming car (with functioning headlights). Nor did this person attempt any evasive action of their own. But you, you could have averted everything. Only if everyone was just more like you.
Hang in, Mish Shedlock, The worst moment of my life was calling my mom and dad from a phone booth on Janurary 10, 1969 to tell them my brother had died. Driving back to LA from Ventura late at night, he dozed off and his Volvo flipped. Computers don’t doze. F*** the Luddites. This ladies’ death is an horrific nightmare. The only worse thing imaginable to me is my brother’s death times 20,700 annually because armchair, Monday morning assholes call their congressmen and lawyers and ambush AV.
The technology failed. I said so. A driver would have and indeed did fail too. And so did a person wearing dark clothes at night walking across a road, likely illegally. And to idiot “Clintonstain” when did I say anything with “total certainty”. The only thing certain is stupid comments. I gave reasons for my beliefs, and like it or not, the industry is following almost exactly the path I laid out, with truck hubs etc.
Hi, the computer can react much faster than a human, which is exactly why autonomous vehicles are so much safer. That car had a relative eternity to decide to swerve and brake compared to a human. Furthermore, if you watch the (low resolution poor contrast) video, you can clearly see the shadow of the pedestrian crossing the street about a second before the lights are shining directly on her. Please, be honest in your assessment. The technology is nowhere near up to snuff. BTW also consider that teens cross mid-street all the time, and no matter how hard you try to legislate away bad judgement by youth, it will never go away.
“The contempt and disdain that Mish has for working people, working American people, is immeasurable.”
Mish reminds me of the Intellectual Yet Idiot mentioned in Nassim Taleb’s writing. More than happy to give you predictions with total certitude on an industry they’ve never worked with or in.
The thing that is wrong with these people is money and profits. To them, everything is dispensable in the name of making profits. The contempt and disdain that Mish has for working people, working American people, is immeasurable. It comes as absolutely no surprise to me that he and his ilk would treat a homeless person as a totally expendable commodity and an obstacle to be literally run over in the frenzied pursuit of the filthy lucre.
Total BS. There was enough time for the car to veer to the left but it never happened.
Public roads should not be used for failed experimentation. The all important bottom line? You amaze me MISH. A human being is dead because of a failed system and all you can say is that the system will improve. What is wrong with people? What does being homeless have to do with any of this? WAKEUP! SOMEONE IS DEAD! The desensitivity training in America is working well.
I’ve watched this til I’m sick, the lady still dies. Key points: She’s clearly visible in second 3. We have no time sync with the interior video, so can’t say when the operator registered that look of horror delta to second 3, BUT it’s at second 21 and, critically important, the operator is not moving forward into the shoulder harness restraint. I.e., Lidar had not detected the bike and caused the code to load the most aggressive braking algorithm available to it. It should have. Most humans would not have done better here, but that’s not the key issue: The computer should have done better. Fix it guys! You’ve got something obviously wrong in integrating the lady & bike laser bounce and differentiating it from that car in the distance at her head.
No way that car should hit that person no matter who is driving, the first story was that the cyclist stepped in front of the car WRONG. the cyclist was almost clear of the car on the passenger side. the law is written in such a way that you must be able to stop ahead of any unmoving, parked or disabled vehicle, regardless. This person was moving across the road and was in plain sight ahead of the vehicle and was in the line of sight in time for any reasonable operator to stop. The guy was looking DOWN. Do not be fooled by the lack of headlight illumination in the video, there was really more than enough to activate a response. if you were driving you would have, should have seen this person and tried to stop. remember the difference between life and death is often a few miles an hour, reaction time. i was a professional driver and have seen lots of bad s*&T, and most if not all of it comes from distracted drivers
lidar picks up reflected light at various wavelengths. The beams of light are fairly narrow at close range. The lady is wearing a black jacket which likely doesn’t reflect the light very well. She is also wearing bluejeans. But if the lidar is aimed at the height of her jacket she might just appear as nothing. Humans will see the whole picture instantly. The computers picking up the lidar data will never process everything that fast. Interesting issues here. What if the lidar had simply been jostled slightly out of place from hitting a pothole?
Regardless of who’s at fault. This is going to five ammo to the anti-driverless car crowd and will be a setback.
I’ve seen homeless people get hit by people drivers a few times in broad daylight. It was always the homeless persons fault. I’ve twice hit deer, One time a group of maybe ten crossed the road right in front of me during a snowstorm. I hit the brakes, but couldn’t completely stop and swerving would have just changed the deer I hit. Not avoided an accident. Another time a deer ran into the side of my driver side front quarter panel. All I saw was a deer head appear right in front of me. Took out my side view mirror at maybe 40 mph. No way i could have stopped.
Self Driving Car Wins Daytona 500!
Show me that headline and I’ll believe these things can handle real world traffic
@xil who said in response to my statement, “I’d have been able to stop unless I were distracted.”:
“right, just like the COUNTLESS people who’ve hit cars, deer, dogs, people… that have suddenly appeared in their path.”
Nearly two seconds elapsed from the time the person’s feet were clearly visible to the vehicle striking the pedestrian. My reaction time is under a half-second (despite my 65 years) and I am on the brake in well under 3/4 second. I have had many years driving experience (most with a stick-shift), driven through major cities in the US (and London) and across the US 20 times. I also rode a motorcycle for several years and had to learn to anticipate every possible thing that could kill me, including deer.
And, I am definely not like ‘COUNTLESS people’.
*”George_Phillies: One might wonder how well the LIDAR does against a black surface, such as the one being worn…”*
Sounds like the right idea. I think we have a winner.
Who knows? Maybe the artificial intelligence module in the Uber self-driving car decided that there was no real need to avoid the pedestrian… Terminator style… Score: AI 1, Human 0 – 500 points… Ka-ching!
Levandowski and the “Move Fast and Break Things” strategy of Uber Automated. He was pissed the first death was in a Tesla and not an Uber.
https://www.theverge.com/2018/3/20/17144090/uber-car-accident-arizona-safety-anthony-levandowski-waymo
And, all the security folks are having S#!T hemorrhages over the electric utilities being hacked. That’s a minor annoyance. Imagine 40,000,000 NETWORKED autonomous vehicles simultaneously going full throttle for 10 seconds, then making a sharp left. The only secure computer is one that is not connected to the internet. BAD, BAD IDEA!
One might wonder how well the LIDAR does against a black surface, such as the one being worn. As there is another vehicle dead ahead…the decedent’s head blocking line of sight to that is an early warning of an issue..hi beam would not be acceptable. Someone with very good situational awareness might have known if there was another car in the next lane, so that ‘swerve” was possible, but swerve might have del to multicar crash. …a computer that can multitask willknow the next lane better than a person. You do see the bicycle, because it was not black, and a very cautious driver might have concluded unknown, close, moving sideways, foot off accelerator.
We are constantly being told that Uber uses LIDAR and it will not run into things. Watching this video I have to wonder if LIDAR is all it is said to be. It did not even seem to swerve to avoid collision. I don’t want to be on the same road with an unmanned vehicle. I’ll take a real human at the wheel.
(After the inevitable tragedy) Mike: “Sure it’s sad, but just look at what the correct people are Tweeting.”
It’s arrogant, callous, sadistic, vicious and sociopathic. How many people here have essentially said that the homeless person deserved to be run over? Sheesh! And then we wonder why things are going to dogs in this country?!
One of these days, a self-driving truck will hit a school bus. And guess what? Mish et al will be saying the show should go on. They will be run over (figuratively speaking) by the crowd.
“I’d have been able to stop unless I were distracted.”
right, just like the COUNTLESS people who’ve hit cars, deer, dogs, people… that have suddenly appeared in their path.
There are several issues here. First, why did the cyclist choose a dark part of the road to cross. Second, why was she not looking both ways for vehicle headlights (this car’s lights should have given her ample warning. Third, the driver was sitting there quite unlike a driver, relaxed, and looking down at something for disconcertingly long periods of time. Perhaps it is okay in the US, but this invites disaster in a country like India. In my country, one passes cyclists and pedestrians like this every few 100 meters (especially on highways passing villages with little light where many people randomly cross on foot, cycles and motorcycles – and you also have cows, dogs, goats and other creatures ambling/running around to ramp up the fun element).
As a driver, I would have had my high beams on and would have seen that lady. The Uber didn’t because it’s engineers are as arrogant as the author of this blog.
Well I have no doubt that if the car was fitted with an infrared vision system it would have seen the woman faroff enough to stop, but the fact of the matter is she was crossing the part of the road with no street light on it in a black jumper.
There is absolutely no excuse for this vehicle to not have been aware of the pedestrian crossing, and to take evasive action. The fact that one specific, cheap or poorly calibrated, dashcam managed to not show a good image, means nothing more than that it’s possible to set up a dashcam shitty enough to show nothing. If the sensors available to AV builders weren’t more acute than this cam, there would be no attempts at building AVs in the first place.
Also, the “driver” in this scenario is not driving. He is, rightfully so, being a passenger. Who can, theoretically, reach across and grab the wheel should the driver have a heart attack. But isn’t really expecting to have to. Hence isn’t exactly on high alert.
Having a “driver” on board, may make sense in order to help the car solve higher order problems, like the AV being shut out of opportunities to merge onto a busy freeway due to overcautiousness, thus piling up half of Phoenix behind it. But pretending that such a “driver” will in any way be primed to react with normal human driver quickness to an impending accident, is just sleight-of-hand folly aimed at assuaging the hopelessly clueless.
Heck, this scenario is pretty much a poster case of situations in which AVs should have an advantage over humans. Wide open space, poor light, noone else around that the algos have to pay attention to…. Heck, my vacuum cleaner hopped up to do 40mph, would likely have fared better than this car. If this is even remotely close to the best Uber can currently do, they really ought to get off the road, and go back to practicing evasive maneuvers and situational awareness in Uber shareholders’ and employees’ children’s schoolyards.
Hilarious how few people point out the obvious stupidity of the “victim”….god rest her soul….Who on earth would casually cross a dimly lit road at this time of night in a largely industrial area without looking for oncoming traffic and perhaps even hustling a bit to get across. Survival of the fittest. Self driving tech will far surpass our ability to avoid these situations in short order. Additionally, take a look at the number of vehicle/pedestrian fatalities for 2018 so far in Phoenix metro to put this in perspective.