mastodon.social is one of the many independent Mastodon servers you can use to participate in the fediverse.
The original server operated by the Mastodon gGmbH non-profit

Administered by:

Server stats:

356K
active users

Fair warning that I’ll be ranty because I hate losers talking about DEI hires.

So why is memory address 0x9c trying to be read from? Well because… programmer error.

So what happened is that the programmer forgot to check that the object it’s working with isn’t valid, it tried to access one of the objects member variables…

This is a huge assumption. The last rumor I’ve read from actual cybersecurity people is that Crowdstrike’s update files were corrupt (update: disproven by Crowdstrike’s blog post). If this is true it’s likely still from programmer error at some level, but maybe not as simple as “whoopsie I forgot an if (data == nullptr) teehee”.

He, like the rest of us that don’t work at Crowdstrike, has no idea what happened. I have seen computers do the weirdest gosh darn things. I know better than to assume anything at this point. I wouldn’t even rule out weird stuff like the data getting corrupted between release qualification and release yet.

It turns out that C++, the language crowdstrike is using, likes to use address 0x0 as a special value to mean “there’s nothing here”, don’t try to access it or you’ll die.

This thread is full of these sorts of small technical inaccuracies and oversimplifications so I won’t point out all of them, but nothing in the C++ standard requires null pointers to refer to memory address 0x0. Nor does it require that dereferencing a null pointer terminates the program.

Windows died not because C++ asked it nicely to, but because a driver tried to access an address which wasn’t paged in.

Crowdstrike should have set up automated testing using address sanitizer and thread sanitizer that runs on every code update.

The funny thing about accessing into non-paged memory in kernel space:

  1. It will crash regardless of if it’s running under Asan or not, sanitizers are literally irrelevant based on what we know so far
  2. The Asan version he linked to is for user-space. In the windows kernel you’d need KASAN instead.

(If this was a simple nullptr dereference on bad input data then perhaps a fuzzer would have helped. Fuzzers are great though I have no idea how hard they are to use with kernel drivers)

C++ is hard. Maybe they have a DEI engineer that did this

Dude would probably call me a “DEI hire”; but I bet I could beat him in a C++ deathmatch so neener neener.

CyberplaceKevin Beaumont (@GossiTheDog@cyberplace.social)The .sys files causing the issue are channel update files, they cause the top level CS driver to crash as they're invalidly formatted. It's unclear how/why Crowdstrike delivered the files and I'd pause all Crowdstrikes updates temporarily until they can explain. This is going to turn out to be the biggest 'cyber' incident ever in terms of impact, just a spoiler, as recovery is so difficult.

@sailor_sega_saturn And given enough time and enough scale even the most improbably weird things will eventually happen. Update file corrupted by a storage controller that flips a couple of bits at random after every 720 hours of uptime but only if it’s 23.682 seconds after the hour? Weirder shit has happened.

@m @sailor_sega_saturn
Builds failing, but only at the new office, and only if you tried to build from scratch.

Funny, the Windows network crew that operated the network and suddenly had to operate NFS over UDP on their network, never really realized that their switches were only capable of half-duplex operation. But announced full-duplex. And these Linux boxes fully used that. And big UDP packages used by NFS under load got corrupted.

@m @sailor_sega_saturn
Took a f%cking nightshift of the CTO (German company, so the CTO had PhD in C.S. and still remembered hacking C++ code) and the resident external IT consultant working on the C++ code getting frustrated with the builds crashing and literally debugging the whole shebang to discover that beside a ton of C++ memory bugs, we also had a network issue.

Andreas K

@m @sailor_sega_saturn And philosophically, I've been now for a decade in "automatic data entry from 3rd parties", ETL (nice phrasing for industry level web scraping and data clean-up).

Literally, what I've seen (and sometimes, as I've also done website development, one wonders what the f%ck the dear colleague was thinking while (s)he developed THAT. Or I want the drugs they were on, that must have been a great trip.), nothing is unthinkable in IT.