I'm sorry, it took *how* many servers to post a single long message from Ghost to 5k fediverse accounts and handle some replies?
https://activitypub.ghost.org/beta-plans/
(via @fediversereport)
(edited again TA: See replies from @thisismissem pointing to implementation choices + @bengo about pushing to Mastodon)
@kissane @fediversereport I always thought of this as the fundamental technical problem to solve for distributed social networks. And we're just... not solving it. People are talking about everything else besides this. And it's so weird to me.
@polotek @kissane @fediversereport I remember when twitter first launched and everyone in that community were discovering the work needed to get RoR to scale to a global “firehose” size. Feels like we're back there again but in different ways.
@poswald @kissane @fediversereport I should be more clear about which issue I'm referring to. There are many technical issues to solve. I'm talking specifically about an issue that is specific to the decentralized nature of the fediverse. There are many optimizations that a company like twitter can do to scale fanouts, because they control both the source and the destination. Fediverse servers do not have that advantage. And more so, many servers are likely to be under resourced.
@poswald @kissane @fediversereport As far as I understand, mastodon's implementation is particularly naive today. If you have 5000 followers, every post creates 5000 jobs. There are many optimizations they can make today that will help. But I believe the problem of scaling of decentralized message delivery is going to be a huge bugbear if the fediverse keeps growing.
@polotek @poswald @kissane @fediversereport No, that’s not true. A post is only delivered once per domain. And we use keep-alive connections to shave off request setup time for repeat deliveries. If you have 5k followers from 2 domains, 2 requests will be made. Only if you have one follower per domain does it become 5k requests.
@Gargron @poswald @kissane @fediversereport feel free to explain the actual reason this is such a persistent problem. I don't mind being corrected. But please don’t let that be the only reason you pop in.
@polotek @poswald @kissane @fediversereport It’s really easy to make mistakes in computing that make a massive difference in performance. For example a database query that runs inside a loop vs outside. And it’s easy to miss when your code runs once but becomes really obvious once it has to run multiple times per second. I would not make any assumptions about the protocol itself based on Ghost’s work in progress implementation.
@Gargron understood. Thanks for stopping by Eugen. Super helpful as always.