New from 404 Media: Bluesky may have said it won't use user data to train generative AI, but someone else just published a dataset of million Bluesky posts for "machine learning research". Already very popular dataset, your data may be scraped https://www.404media.co/someone-made-a-dataset-of-one-million-bluesky-posts-for-machine-learning-research/
@josephcox I guess you could also pull this together from Mastodon, but Bluesky is going to make readily available data much faster.
@tehstu @josephcox I'm pretty critical of Bluesky (see my timeline) but I don't see why this would be any harder or slower to do from Mastodon/fedi
@tomw @josephcox I meant in the sense that Bluesky has more users and so generates the content faster. I was trying to guess why there haven't been very public datasets to this effect from Mastodon.
@tomw @tehstu @josephcox all Bluesky data is public. Many ActivityPub posts are private or followers-only.
@evan Small clarification as I know you’ve avoided the Bluesky literature: Bluesky DMs are not public because they’re not part of ATProto. They’re a separate service.
@amd thanks. Updated.
@mcv
> Is follower-only a thing on ActivityPub?
Yes. They're called Followers posts in Mastodon now.
> I thought everything was unencrypted and basically accessible to any server that receives it.
Yes. There's been all sorts of research into things like Object Capabilities, that could force receiving servers to do what they're told. But for now, not displaying Specific People posts publicly, or showing Followers(-only) posts only to followers, is an unenforceable handshake agreement.
(1/?)
@mcv
> it only works as intended if all your followers are on servers
...whose admins and software obey the agreement ("protocol"), yes.
For now if you want decentralised *and* E2EE, you need to switch to another network for that. Options include federated networks like XMMP+OMEMO or Matrix, or a P2P one like Jami or Tox.
If you're wanting something noob-friendly, I'd go for Element (Matrix) or Snikket (XMPP).
(Full disclosure: I've done paid contracting for Snikket)
(2/?)
@mcv
> Is it completely out of the question to add some encryption to the protocol for this sort of situation?
Opinions vary. IMHO Mastodon's decision to clone Titter DMs was a mistake. I'm inclined to think it's wiser to clearly separate social publishing from private communications, by having them in separate apps (with some kind of SSO so we can use the same account in both).
I've even suggested using AutoCrypt, so I can check my fediverse DMs in Delta Chat;
(3/?)
But I suspect I'm in the minority. Most people seem to want the fediverse to be a thneed (see The Lorax), and work of various kinds is underway on bringing E2EE to the verse. Apparently there's a taskforce working on it;
https://socialhub.activitypub.rocks/t/end-to-end-encryption-e2ee-task-force-meeting-jul-19-2024/
... which might be this one?
https://github.com/swicg/activitypub-e2ee
Then there's @soatak@furry.engineer's efforts;
https://soatok.blog/2024/09/13/e2ee-for-the-fediverse-update-were-going-post-quantum/
(this covers some policy issues as well as technical ones and is well worth a read)
(4/4)
@helge posted a Fediverse Idea on an E2EE AP messenger last year;
https://codeberg.org/fediverse/fediverse-ideas/issues/3
... and @dansup's of PixelFed and loops.video announced Sup messenger about a year ago;
https://wedistribute.org/2023/08/sup-by-pixelfed-is-coming/
The MIMI working group at the IETF included the possibility of using AP in their investigations;
https://bifurcation.github.io/mimi-aim/draft-barnes-mimi-aim.html
So in summary, a lot is going on all over the place. Maybe we need to get all these folks in a room?
I still think the combination of usenet+email was the perfect integration of public and private communication. They weren't completely separate; usenet posts included the email address of the author.
I'd like to see something modular, like maybe XMPP+ActivityPub, or something like that.
But that still doesn't address semi-public communication, like to all your followers, or to a specific group/circle/aspect, that still guarantees (through encyption) that it's only to that group.
I imagine everybody would automatically publish their public key as part of their profile, and a limited message would be encrypted, with for each authorized recipient an attachment containing the key encrypted with their public key. Of course that could get pretty heavy with posts to lots of users, but servers could throw away attached keys that aren't for any of their own users.
@mcv
> But that still doesn't address semi-public communication, like to all your followers, or to a specific group/circle/aspect
E2EE private groups are the core of Matrix, to the point that DMs are just groups with only 2 members. Delta Chat can encrypt group with AutoCrypt, and I believe XMPP can encrypt private groups too, with MUC+OMEMO.
But from what I've read, the new MLS standard is key to doing E2EE groups efficiently. Devs from all 3 protocol networks are working on implementations.
Sadly nobody can use it, and self host. The installer script is very out of date, and broken.
@SchickeSchickeSchweine
> Sadly nobody can use it, and self host. The installer script is very out of date, and broken
I know @snikket_im are keen to support a range of installation options. But I suspect progress is being slowed by funding challenges. The same thing I'm told is making it harder to deliver Matrix 2.0, MLS support, etc (funding challenges for Element).
@strypey @snikket_im @mcv @evanprodromou It turns out they updated the installer script almost immediately and I tried it and my server installed just like that. I am using it now with my friends and it's really really nice. Thank you so much to the development team.
@SchickeSchickeSchweine
> It turns out they updated the installer script almost immediately and I tried it and my server installed just like that
I love it when this happens! One advantage of being able to @mention projects when we complain about them ; )
@mcv
> There's been all sorts of research into things like Object Capabilities
FYI @cwebber wrote a bit about this, with some links for further reading, towards the end of her insightful analysis of BlueSky/ ATProto;
@tomw @tehstu @josephcox The publicly available firehose makes this so easy it’s not very impressive at all.
Demo: https://firesky.tv/
Just curl and wait.
@amd @tomw @josephcox I'm certainly not impressed, from neither a technical nor ethical perspective.
@tomw @tehstu @josephcox it's the opposite, it is way easier on mastodon. As you don't have to scrape anything, just collect the training set locally on your own instance.