mastodon.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
The original server operated by the Mastodon gGmbH non-profit

Administered by:

Server stats:

297K
active users

I'm thinking about getting a "new" used car. Unfortunately, tests of driver assistance / "self driving" systems don't seem to be any better than they were three years ago when I wrote danluu.com/car-safety/.

No one's doing anything resembling what a programmer would consider serious benchmarking and it doesn't look like anyone is going to either even though driver assistance / ADAS systems are becoming a more important part of safety over time.

danluu.comHow do cars do in out-of-sample crash testing?
Dan Luu

Since writing that post, I've spent a lot of time talking to people who work on ADAS systems about the processes they use; how they handle bugs when they're found; when they use a shared ADAS vendor, which companies find the bugs in the vendor's system and which companies never notice, etc.

I believe this is the only way you can reasonably compare these systems today and I doubt this will change within a decade. For obvious reasons, no one who has the information is going to write up a ranking,

so it's now nearly impossible for the general public to make judgments about automotive safety except in cases where the company repeatedly ships extremely shoddy software, e.g., Tesla with mastodon.social/@danluu/109514, twitter.com/kenklippenstein/st, etc., but Uber was the only other company putting something so obviously bad on the roads, so this doesn't give you much information.

FWIW, I'd say that BMW has the best methodology I know of among cars you can buy (I updated the post to reflect that), but

TwitterKen Klippenstein on Twitter“I obtained surveillance footage of the self-driving Tesla that abruptly stopped on the Bay Bridge, resulting in an eight-vehicle crash that injured 9 people including a 2 yr old child just hours after Musk announced the self-driving feature. Full story: https://t.co/LaEvX9TzxW”

every company I know of that's selling a "shipping" car has a methodology that you'd consider "move fast and break things" if they were a software company; the spectrum runs from "nearly completely untested, relies heavily on vendor's components working when connected (essentially never verified) while pushing vendors to add unsafe features to reduce cost or increase velocity" to "moves quickly and cuts corners that create significant risk".

Just for example, a semi-typical story is

the car company pushes for [redacted] to save eng effort, which accidentally results in a large fraction of cars they ship having AEB disabled. Years later, a journalist takes one to a test track to try it out and they get one of the many cars with no AEB. It plows through simulated test pedestrians with no attempt at stopping and this results in the car company issuing a recall.

Given how testing is done, it's somewhat lucky that anyone ever found this bug via testing at all.

Emergency braking literally did nothing for years and no agency that rates safety could've found the bug since they test cars early in a model's life and the bug was triggered by an update.

The absolute most basic and easy to find bugs are not being caught by most car manufacturers or any testing agency. This will not change in the foreseeable future.

And of course comprehensive testing of unusual circumstances generally isn't happening either, outside of "testing in production".

@danluu yes. Aircraft did the same thing for decades and air safety didn't improve until the jet age and modern NTSB/FAA certification practices worldwide. Self driving cars will speedrun regulatory issues.

@danluu not at all surprising. My Tesla has no functioning ADAS system to speak of as far as I can tell. Oh, it’s advertised to be in there. It’s a line item on the car’s feature list. It’s supposed to have blind spot warning and emergency braking and all that. But it doesn’t work. I’ve had a number of close calls that should have triggered some kind of at least beeping noise in the car and……… nothing. I just assume it doesn’t work and never will.

@danluu fwiw, there’s another EV maker (an offshoot from a well known safety focused car company) that has worse software than Tesla (they had to buy back 2 cars from me in one month, and I’m legally prohibited from stating their name publicly).

I think the “disruption focused” companies do a bad job at software quality. For example theverge.com/2022/6/29/2318808

The Verge · Vehicle quality drops to lowest level ever in new JD Power surveyBy Umar Shakir

@danluu I would absolutely read a book about orgs who continuously run their test regime on a daily driver

@danluu Do you know of any country's equivalent of NTSB that would manage to learn about and investigate accidents where at first glance some safety feature failed to operate?

@danluu Having worked in and adjacent to that world for awhile, yeah. Testing quality will be vendor-specific, and generally opaque, until there are regulatory requirements. Until then the standard answer will be "the brakes and the driver are the primary line of defense, ADAS is just best-effort".

Happy to chat more in DM if you want more thoughts.

@danluu not only is their software flawed but the way they explain what their cars do and don't is flawed.
Many people think their driver assistance systems do more than they actually can. And doesn't even have to be a Tesla with FSD promises but even automatic braking where people believe it can operate at far greater speeds than it actually can. Bet there's an opportunity there for studying how big is the gap between the confidence people have on their car systems and their capabilities.

@danluu my 2017 Prius Prime's adaptive cruise control seems to struggle with determining good following distance in very rainy dark conditions, especially on curves. I can't tell if it's due to water interfering with the ultrasonic sensors or obscuring the camera or both but it was kinda scary to uncover.

@danluu who was the goat asking for monthly subscription to enable heated seats? 🥹

@danluu it's not available here yet but the Mercedes drive pilot system should be considered. Mercedes will take liability while the system is being used so they have a lot of confidence in it.