Microsoft now blaming the EU for the crowdstrike issue, because the EU made them open their APIs to third party developers in 2009 is hilarious.
If your APIs had any kind of gracious error handling, this wouldn’t be an issue. Fix your bad code, Microsoft, stop pointing fingers.
@thelinuxEXP hahahahaha what.
Sue these fools out of existence.
@peteriskrisjanis @thelinuxEXP Have they threatened to exit the EU yet? Fond memories of facebook repeatedly threatening us with a good time...
@thelinuxEXP So as a Linux user, your contention is that a ring 0 privileged application, with root access to all kernel functions, should never be able to crash the system?
@bouncing No, it’s that the OS should be able to recover gracefully, not go into a BSOD boot loop
@thelinuxEXP @bouncing Recovering from a panic is *really* hard, because it only happens when kernel-level code has done something *bad*. Or unthinkable, sometimes.
Last I checked, Linux still panics when device drivers or kernel modules get page faults (although, as I recall, they do have *some* impressive isolation for that, so they simply shut off the module in some cases; this is not helpful when the filesystem is the thing that panics, though).
@thelinuxEXP @bouncing TIL computers can (re)boot, shut down, crash and recover “gracefully” 
@bouncing The OS should run such apps under certain protections. M$ should display some seriousness for f..k sake, not every app is minesweeper https://nondeterministic.computer/@mjg59/112816011370924959
@muratk5n https://www.pcmag.com/news/why-did-crowdstrike-update-only-hit-windows-blame-the-eu-microsoft-says
> As Microsoft's Chief Communications Officer, Frank X. Shaw, noted on X, a 2009 agreement between the European Commission and Microsoft required Redmond to give security software the same level of access to Windows as Microsoft itself.
By definition, the kernel does not run in a sandbox.
If you’re giving third party apps the same access as Windows, they can crash Windows.
@bouncing @muratk5n antiviruses work as kernel modules because they need complete oversight over the system.
BTW, current most-used Anti-Cheats sadly also have kernel modules just for that reason. Only minor ACs like Valve VAC and games use server-side "Anti-Cheats", probably because it's harder and costlier and bans may be detected later.
Device drivers as well use the same kernel APIs, BTW.
All these kinds of programs HAVE to be coded with the highest standard of security...
@bouncing @thelinuxEXP Windows should definitely auto-disable faulty kernel drivers, to recover on multiple failed reboots. If need be, load Windows with a ”WARNING” banner in the Server desktop GUI and autosend e-mail notifications to sysadmins so that they know which systems to repair.
Sysadmins can and will fix security issues. The problem here was that they couldn’t do so without physical access or at best: resorting to Virtual Machine backup restore, I guess.
@bouncing @thelinuxEXP Preferrably, e-mail would be the last resort because 30,000 server e-mails will cause other unwanted problems. It’s better to use a web-based system monitoring dashboard of choice and then let the monitoring software send one or two e-mails reporting the errors and how many machines are affected.
@bouncing @thelinuxEXP
Yes, otherwise it would be malware
@thelinuxEXP @bouncing I don’t think anyone’s saying that all bad driver behavior should be prevented. This was an accidental out-of-bounds read, which definitely could be handled without a full-system crash.
@bouncing @thelinuxEXP no chance. This kind of error at the kernel level is impossible to sandbox completely. Sure one can think of rebooting the system with another kernel and modules, but someone must push that button. You could have some kind of hw watchdog that defaulted to another kernel, but not without custom hardware. Anyway having a closed driver that updates automatically from a third party on _all_ machines, you are pretty much depending on them. Bonkers...
@bouncing @thelinuxEXP I'm a cyber security consultant and I completely understand the necessity of EDRs, WAFs, IDSs, IPSs, monitoring and auditing, but let me tell you a secret, this isn't the way to make secure systems, just mitigations that can create even more issues like the one we witnessed. You can only achieve total security from isolation, OS simplification, replication, zero external dependencies, really tight controls over user and system actions basically from the ground up.
@bouncing @thelinuxEXP ... basically you must control everything and your system must be defined by a strict set of business processes. You cannot have general purpose OSs for this, even worse at the mercy of your employees and users! You must restrict and control all the inputs and outputs of your system in a very formal manner.
@thelinuxEXP The Linux kernel and all associated api's are open and those developers don't blame the government for mistakes.
?) @thelinuxEXP also if you're going to force drivers to be signed by your key anyway, maybe actually vet the things you're letting people sign (or sign them yourselves)
@thelinuxEXP I saw this in a post on a group on fb, to put you into the context: this is an app that lists job openings
@thelinuxEXP topkek, the people at fault are crowdstrike, end of the line, they should have a proper release pipeline and QA...
@thelinuxEXP I also found that comment to be very immature, to be honest.
@antaeus How would that be immature?
@thelinuxEXP instead of saying it like it is, they blame the EU. It’s a bit childish and immature to blame others for their own shortcomings.
@antaeus Ahh sorry, for some reason I thought you were referring to my own post as immature!
My bad! I totally agree, this finger pointing is just childish.
@thelinuxEXP no worries!
Have a great day, Nicholas!
@antaeus Thanks, you too!
@thelinuxEXP @antaeus I just want to say that your exchange as well as this thread as a whole are so civilised and respectful! I really enjoy reading all those takes regarding a topic of which I only have superficial knowledge, it’s awesome! 
@dom @thelinuxEXP thank you kindly
@thelinuxEXP damn Microsoft is coping hard-
@thelinuxEXP Really? I can only imagine this is kind of a legal thing, to try and avoid a huge fine
@supernov Might be, yeah! It must have broken a bunch of SLAs for some of their customers
@thelinuxEXP they're afraid of losing their BSOD monopoly
@thelinuxEXP Be cautious about wanting them to "secure" their code. They will inevitably find the enshittification way of doing this and simultaneously cooperating with governments to constrict users in awful ways.
@SalemsLot Hahaha possible 
@thelinuxEXP based on the one video I watched they can't afford to recover from any kind of kernel issue in general (and safe mode is there to help prevent it if you know how). CrowdStrike had a boot-start kernel driver apparently and didn't handle "user" errors properly in the kernel.
@thelinuxEXP but blaming the EU is indeed hilarious
@thelinuxEXP open source code fucks up? Open source is the problem. Closed-source code fucks up? Still because of open source somehow
@thelinuxEXP
how likely do you think it is that without this API, CrowdStrike would have used a janky way to get what they want out of the kernel (and still cause this issue)?
This is obviously something they would want to do, but I have no idea if they would think it would be worth the reverse engineering efforts, monkey-patched kernel files, probably need to re-sign everything to be secure-boot-compatible, etc...
@thelinuxEXP my understanding is that in kernel mode you can’t gracefully crash without BSOD. At least based on Garry Explains video and Primeagen interview.
@warriormaster Sure, but after the BSOD, you reboot, look at a log, disable the faulty module, and let your customers go back to work with a warning that something was disabled.
These customers then take the measures they dem necessary.
@thelinuxEXP Kernel extensions on Linux would make the system crash just like Crowdstrike did. You can't really blame Microsoft here.
@k0bin No, the kernel has an ABI to handle this, developed after Crowdstrike crashed Linux systems earlier this year.
@thelinuxEXP they are not able to secure their systems like Apple does. All they are saying is that even if they could they are not allowed anymore 
@SebastienK But that’s not true. Even without the EU they would allow these apps to have the same access, because it’s needed. They even developed an API to handle these errors, that Crowdstrike doesn’t use.
It’s also their fault for not handling the boot loop more graciously.
@thelinuxEXP Well it's true. If Windows was as closed as Apple builds it's infrastructure we would not be here. It’s as simple as that. Hurray for closed eco systems
@thelinuxEXP This is 100% crowdstrike problem. They shipped a kernel driver that loaded external code into the kernel. As I understand it the .sys file that they loaded was all zeros, and their code used pointers in that file without any verification. This sort of file should have integrity checking in it and that should have been checked before loading. This is an enormous hole and I am surprised it hasn't been used before to insert malicious code as yet.
@thelinuxEXP It's amusing to see some people defending Microsoft. I can defend Microsoft to the extent where they weren't responsible, crowdstrike did most of the damage but saying Microsoft is completely blameless is just plain wrong. I've seen plenty of times when linux mess up, it's always linux's fault, never ever the fault of the one actually responsible. But if it's proprietary then nope, how can they ever do wrong. Here's a non-linux user saying the obvious https://www.youtube.com/watch?v=dGKIdGf_8J4
@pikachu_sensei I am quoting from the video linked above timestamp 18:09 to 19:11
"But the most important thing that could've avoided this problem isn't clowdstrike's fault, it's Microsoft's, because why is it okay for a system to make a change to a critical code like drivers, via a mechanism like windows update, which is well understood, and then reboot, and fail, and crash, and then loop crashing over and over and over again."
then he talks about slim bootloader for firmware and it can roll back to a last known working state in case of a boot failure after an update. https://slimbootloader.github.io/security/firmware-resiliency-and-recovery.html
Even Android has something similar like that but apparently that's too much for Microsoft. These days snapshots are common in various linux distros. For once, forget about the servers, at least Microsoft could've done this for desktop version of windows but nope. It may not be their fault directly but blaming EU is plain stupid.
@thelinuxEXP do you have a source for this? I want to share this with friends :D
@thelinuxEXP Are they going to accept liability for every crash and outage before 2009?
@thelinuxEXP #ebpf is an example how the Linux kernel offers safe APIs for this: https://en.wikipedia.org/wiki/EBPF
@thelinuxEXP gosh but has blaming the EU for your mistakes become trendy
@thelinuxEXP EU is probably just the most click-baity of the finger points. I would imagine the MS legal team has produced a substantial list of pointed fingers in an effort to establish a minimal lawsuit dam.
@AngryAnt yeah, that’s the likely explanation