This is how NASA writes software:
https://www.fastcompany.com/28121/they-write-right-stuff
I want the software driving cars around my family to be held to the a similar standard of quality. In the context of self-driving cars, "Move fast and break things" means "Half-arse it and kill people."
Humans in the USA manage 1.16 fatalities per 100,000,000 miles travelled. Uber's software couldn't even get to 3 million miles before it killed someone.
We do this properly, or not at all.
> We do this properly, or not at all.
an admirable sentiment, but since feckless megacorporations are in charge now and our nominal systems of governance have largely abandoned all but the hollow pretense of exercising restraint over their actions, we probably just do whatever they decide we're going to do.
@ifixcoinops And the NASA software still screws up even though it's doing something a zillion times simpler than a self-driving car!
@ifixcoinops What's really stunning about that article is that it was published in 1996. More than twenty years, and we're still making the same mistakes.
@Terrana @ifixcoinops I'm handing out copies of The Case Of The Killer Robot like hotcakes lately. These aren't new problems, we just keep deciding to be irresponsible.
@ifixcoinops
at some point I feel like the strategy of 'continue writing fortran/c/c++ but be really careful' is not long-term viable
@ifixcoinops “Its standard practice in almost every engineering discipline except software engineering.” This line made me sit down and re-conceptualize a lot of code I’ve written~
@ifixcoinops
1.
>one-third of the process of writing software happens before anyone writes a line of code. NASA and the Lockheed Martin group agree in the most minute detail about everything the new code is supposed to do
How's that different from actually writing the code?
>specificity and precision usually found in blueprints
The source code _is_ the blueprint.
Why'd you have 2 versions of blueprints in 2 different languages?
>serveral peopel who understand the system deeply
If every blueprint review as turned into a code review instead, it would have the same effect IMO.
>Error responsibility becomes easily tracked
But they have only team responsibility and process responsibility. A singel person is never blamed.
Also, you can do that with source code too - `git blame` is your friend.
Also, new errors can be made when implementing the blueprints in code.
@Shamar @ifixcoinops
Also, I can't type anymore :(
s/as turned/was turned/
s/singel/single/
@Wolf480pl @ifixcoinops
Remember that time one of the modules passed a parameter in imperial units and the other module expected them in metric and it caused a billion dollar rocket to blow up?
That could have been prevented by such a deeply detailed blueprint.
@grainloom @ifixcoinops
It really depends on how detailed the blueprint is.
If it says "the function takes a velocity in m/s as an argument" then it would.
But having a javadoc-style comment on the function saying "velocity - the felocity of Foo in m/s" would do it equally well.
`newtype VelocitySI = VelocitySI Integer` whould've been even better at preventing such a mistake.
@Shamar @grainloom @ifixcoinops
but naming a type in a descriptive way communicates the intent pretty well, and makes it easier for the coder to not make the mistake in the first place, and for the reviewer to spot the problem.
Also, compilers, static code analysis, and formal verification help you check the trivial things that people assume without checking. When reading someone's code, I would rather focus on the logic than manually check whether the types are correct.
@Shamar you still need something that will verify the non-logic.
And you can still make mistakes when implementhing the blueprint into code.
@ifixcoinops I would be very interested to read a similar article, but about SpaceX launch-code :)
@ifixcoinops Would that the people building self-driving car software had this much time, patience, and funding, but I don't expect that will ever be true of a for-profit company with investors breathing down its neck. It's not like developers "haven't learned" these lessons. It's not just a bad choice. They're operating under an entirely different set of incentives.
All of which is not an argument against raising the bar -- it's an argument that this software must be made by someone else.
@relsqui Perversely enough I may actually trust Google to get this right. Their incentive is to keep you hypnotized by your phone on a long, slow, boring journey, whereas Uber's incentive is to get you from point A to point B as quickly as possible to rack up more taxi fares.
@ifixcoinops That accident was the safety driver's fault, not the software's or the programmers'. It was never intended to be driving the car without a human making sure it was safe.
@ifixcoinops Also, few humans who kill someone have driven 3,000,000 miles before doing so ;-)
@freakazoid Nope, it was the programmers' fault. The programmers disabled both their own emergency braking systems and the ones that were already in the car as standard. Furthermore, they demanded the operator take his eyes off the road to monitor the debug log. It's all in the police report.
@ifixcoinops That seems like the problem, then, not that they're not using NASA's approach to developing software.
@ifixcoinops I.e. while NASA's approach would work, so would Google's approach.
@ifixcoinops yesterday Cory Doctorow recommended we should "Be thoughtful and consider human circumstances" rather than "move fast and break things"
@ifixcoinops it's worth noting that that article is more than 20 years old.
people have continued to refine these sorts of processes and generally put more capital-E Engineering into "software engineering", especially for industrial and medical applications.
some of it has even trickled down to consumer companies, albeit in a watered down way.
but I agree, it doesn't feel like autonomous driving efforts are even close with their processes.
@ifixcoinops THIS! They often say "code is poetry", but I'd much prefer the code running vital stuff like power plants, water and gas lines, traffic lights, phone lines, etc. to resemble the boring, steady and solid prose of a peer-reviewed paper, to be honest...
@ifixcoinops when I was an engineering lead I threatened my team with "follow my style guide or we're moving to JPL's" and that put the fear of god in them (I believe it's the guide NASA uses, or similar)
I'm actually shocked self-driving cars aren't required to follow MISRA in general since all embedded automotive systems are
@atootingtwit I don't think it would matter if they were required to do so, since Uber's business model is to break the law until they have enough money to bribe politicians to change the law.
@ifixcoinops
# TEMPORARY, I HOPE I HOPE I HOPE
https://github.com/chrislgarry/Apollo-11/blob/master/Luminary099/LUNAR_LANDING_GUIDANCE_EQUATIONS.agc#L179