"Experimenting" CS research by sending deliberately vulnerable and/or broken patches to main has gone down about as well as expected


And for context:

@cypnk "As soon as practical" gives a different meaning to "ASAP" 😂

@rune @cypnk yeah but overall good, no-bullshit response from them. No attempts to turn it around or something, at least yet.

@saper @cypnk Absolutely. It's an appropriate response showing they take it seriously.

It's just funny because it subverted my expectations for that exact usage pattern.

@cypnk see, this kinda stuff doesn’t happen with vertebrate research anymore because there are laws now.


kernel devs demonstrated that they have vertebrae by putting a stop to this


@cypnk honestly, I really appreciate the no-bullshit reaction by the Linux project.

The researchers acted in bad faith, and were warned; they continued, so there was no fscking about, Banhammer 40k. 👍

@rysiek @cypnk I think discussion on this issue particularly hits the root of the problem: github.com/QiushiWu/qiushiwu.g

Yes, it's relevant to know if major FOSS projects are vulnerable and how they could possibly improve.

No it's not reasonable to perform scientific experiments on humans who do not consent.

Regardless of the end goal, the intermediary step is performed in bad faith without consent.

@rysiek @cypnk You could absolutely make double blind studies on the Linux kernel after coming to an agreement with the responsible parties, but clearly that wasn't the case here.

@rune @rysiek Yes, indeed. This seems to have many levels of failure in the chain:

- Misidentifying the target of the experiment (intentionally or not) as being actual people

- Failure to predict consequences (this one is honestly surprising)

- Lack of robust checks from "taken for granted" sources like UMN

I only see the last part as something the kernel group can work on. The first two are entirely on the part of the researchers and the school

@cypnk @rune I don't even see the last part too bad. There is a lot to be said about trusting people and institutions once they prove their worth. I assume there were some previous contributions from UMN that were fine.

To me, this is completely on UMN and the researchers.

@rysiek @rune Scrutiny could be limited to individuals rather than orgs somewhat if they already have a history of contributing high-quality patches. But I do think we need to be careful of "regulars" too

Although a totally different project, remember Seggelmann was a well intentioned contributor who accidentally introduced Heartbleed to OpenSSL. It can to anyone

@cypnk @rysiek @rune personally i blame the CVE programming language, but every time i do, people accuse me of being a disruptive radical extremist, and what solution am i suggesting anyway? it's way too impractical to program in anything that isn't explosive hazardous waste

@meena @rysiek @rune I think isolating certain segments and rewriting the interfaces with a safer language might be something worth considering

Firefox is doing this with Rust:

But, of course, is more difficult with kernel code with millions (billions?) of critical systems in operation

@meena I'm gonna steal this quote "it's way too impractical to program in anything that isn't explosive hazardous waste" 😂

@rune please do

and you can claim you invented it too

most things i say are CC0 licensed :P

@rysiek @cypnk @rune I'd agree on that front. Trust is going to be a lossy abstraction at times but it's better than building a system that treats building human relationships and therefore trust as an inconvenience - that way lies social credit programs.

There's certainly some work to be done in this space, but approaches like this throw the baby out with the bathwater - it's pretty similar to how bug bounty hunters keep slamming PyPI with faux-malicious packages in order to prove supply-chain RCE on their targets (companies where people are being paid to triage and defend), which has the side-effect of absolutely slamming PyPI's already belaboured staff with takedown requests who are totally out of scope and it's a breach of their ToS.

@grimmware @rysiek @cypnk Funny, I was thinking of exactly that.

It was neat the first time someone proved that, but it's honestly the same principle. You're essentially attacking thousands of people whether or not it's for research.

@rune @rysiek @cypnk Totally agree, the first report I got for this professionally I was like "fair play" but then the bounty hunter's work got widely shared and we got slammed with people claiming they got RCE on us despite the fact that a) they didn't and b) the bounty hunter's work clearly said that we'd mitigated.

And they were all *less sophisticated* versions of the attack.

The fundamental problem here is people treating themselves like "customers" of systems that are not in fact run for them, they're run for *everyone* and exploiting without taking the time to try to address the underlying problems (in PyPI's case, lack of namespacing support) is just incredibly selfish. UM just want to publish papers, (some) bounty hunters just want to collect bounties, but with no *real* interest in fixing the systems.

@grimmware @cypnk @rune oh, supply chain security is the next big thing that we need to solve in general. It is horrendous.

@rysiek @cypnk @rune IMO this shows that academics should not be trusted.

@be @rysiek @cypnk @rune the whole publish or perish coupled with the way academics have to fight for grants has severely impacted the quality

just look at the covid studies to see what you can get if you don't have to fight for scraps

but a bigger problem is that CS as a field has had no ethical reckoning: types.pl/@hazel/10610431578029

but i think we're just ignorant to the damage that we're already doing, because it's not one Hiroshima, but many, many Dresdens.

@meena @rysiek @cypnk @rune Do computer science students have to take any research ethics classes like everybody in social and biological sciences?

@be @meena @cypnk @rune no. these are often available but not compulsory.

There is some push for Ethics in Mathemathics, which is somewhat related:

Dr Maurice Chiodo, the head of that project, was at and had a lecture about it:

We had a fantastically interesting discussion later at the .

I didn't, I had a professional issues class one semester but that covered code licences not ethics.
@meena @rysiek @cypnk @rune

@be @meena @rysiek @cypnk @rune As a physicist I also didn’t have to take any research ethics classes. No one would invent an atomic bomb or risk creating black holes on earth, right?

(in the latter case people did the required checks and found the risk to be negligible after someone made the worries public and newspapers reported, but only after that)

@cypnk @rysiek @rune I blame the QA process more than UMN. Who accepted the code without thorough review?

@cypnk that's the proper way to respond to this issue: contact the university and let them figure out what happened and take appropriate action

@cypnk obsessed with the audacity of the student in question tbh.

calling accusations of wrongdoing "slanderous" after publishing a paper explaining what it is that you did in great detail? amazing.

@cypnk Why does that article lead with an image of Tux on what looks like the ancap flag?

@aidalgol It's just Tux's beak and back shade. I don't think there's anything more to it than that

Sign in to participate in the conversation

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!