@liaizon Right now, what HTTPS calls with what data go over the server-server and the server-client connections, and what the semantics are; What further actions a server/client should take when sending/receiving those calls. More specifically, some kind of minimal compliant Fediverse server would be nice.
I think I can help you find what you need, or in the worst case figure it out and blog it, but let's start with where you'd like to go with this so we can start you right
It looks like you want a server that implements ActivityPub C2S that you can target with a client. Is this right?
Apart from which part of the spec you'd like to implement (first), it would be helpful to know about your plans - target platforms, preferred toolkit
ActivityPub is very expressive, even Turing complete, but you won't find any discussion of the semantics beyond the primitives in the ActivityStreams Core Vocabulary. The ideal server, which nothing currently approaches, would be a store and forward service that facilitates communication between two or more clients, which may be on different severs, and does so agnostic to the payload. The experience should be that you post an Object to your outbox, the server responds 201 created, and the recipient(s) get copies in their inboxes. Everything else is sugar
@yaaps Thanks. I'm indeed only starting to figure out what I want, and it does approach the ideal server you described. I start the server, it federates with other servers, maintains its corner of the social graph, allows sufficiently authenticated clients to do their thing on it. All social media functionality beyond that would be implemented as Applications.
So, I went through the C2S/S2S specs, I'm brushing up on Flask, I guess the world still needs such a server?
This is a description that includes historical references to fediverse development. These relate to goals in the current discussion without asserting that their goals or decisions weren't correct for their purposes and time (or that I'm qualified to judge)
This is a living community that is iterating away from a group of legacy standards. Initial Mastodon development was happening as the right time to give the federation protocol a necessary push, but did not adopt the client protocol
There weren't any C2S clients or servers in general use until February of this year, and they adopted some shortcuts based on a common history of having implemented the Mastodon API already, so C2S between the two programs works, but OAuth2 dynamic client registration uses Mastodon scopes and endpoints rather than an idiomatic discovery
There's also online tests at activitypub.rocks. Those have been unavailable for awhile, and I'm not sure where to get source code, so the most interesting projects from the perspective of developing infrastructure for a smart client ecosystem would be Pleroma, Andstatus, and Epicyon:
The Pleroma philosophy is "everything and nothing," but it's better at everything than nothing, currently. It's a good target to test a client or federation, with the caveat that backend as well as frontend contain the assumptions implicit with microblogging
AndStatus started as a GnuSocial client. It's added the Mastodon API and the developer has regularly lobbied for AP C2S. You can use it to test a server C2S implementation, with the understanding that AP C2S is still experimental. The developer is eager to help move that along
Epicyon is a basic AP S2S targeted at small deployments. In addition to being in Python, the target environment makes the internal processing decisions easy to inspect. It's the simplest possible AP server that interops with Mastodon
I'm about to resume the blog I started in the summer about how to make a simple AP server and improve it. Here's what's out there so far:
Here's a blog that will help you build a stub to iterate features on:
For basic architecture of a server, the main question is whether you want messaging to delete on delivery, interact with the client about removal, or archive permanently
You will probably want multiple actors even if you end up building for single user, so it's best to keep actor and user as separate concepts
@yaaps Thanks; Looks like I'll either dig into Epicyon, or roll my own.
As far as delivery and routing is concerned, are there some standards / common practices? Should I take care to deduplicate messages that reach me over multiple routes, or may that not happen? What's the best practice on limiting object visibility? Are there supplementary protocols besides OAuth2 that I really should provide?
Webfinger is widely used and services that rely on implicit addressing (using "@" to mention recipients in the body) won't allow users to address messages to actors on your service without it. It's also used to indicate an endpoint for local users to interact with content on a remote
HTTP Signatures is used to sign interactions with remotes. Cache public keys, ignore delete activities unless you have a cache hit, and defederate any server that offers you a key that's over 3kb
JSON-LD is worth looking at for building a general purpose service. The sales pitch that you can just build JSON applications on JSON-LD specs without processing JSON-LD doesn't really pan out beyond very narrow application domains in ActivityPub. Without it, AP isn't extendable. With it, AP arbitrarily extensible
The data structure in ActivityPub is objects, so introducing constraints and mapping it to a relational database is of dubious value. (The ORM in a framework may do more harm than good)
Generally, it's better to flatten objects for random retrieval unless you have a computer science background and toolkit with facility traversing tree structures
Each message will have a unique ID and servers only federate messages from local clients. So you'll only see duplicate messages when a message is quoted in a property of a child object. Quoted objects shouldn't be trusted unless you can verify a linked data signature and one is included. Otherwise, dereference the ID and retrieve the quoted message from source
There's a new standard, OCaps, rolling out for visibility scoping, because current practice is borked. More on that later or do a hashtag search on #ocaps
@yaaps Cool. One thing I haven't seen mentioned in this context so far is websockets; I'd have assumed that people would build on that for push updates?
We've been talking about ActivityPub proper. Web push and the Mastodon streaming API have been out of scope for the conversation so far
The simplest thing to do would be to allow upgrade to WSS on inboxes
It's been over 3 years since I looked at web push and I just learned that pushing to chrome can be done with your own infrastructure, so I need to check that out again even though I'm mostly interested in mobile
@yaaps Thanks. I think my preliminary notes are now complete.
One thing irks me though. AP works by "Everything is visible", and OCAP (now ZCAP?) wants to remove actions by adding a protocol? That seems naive to me, and "We'll just defederate non-compliant servers" is a recipe for data leaks. Is there no approach based on "We'll just encrypt everything"?
Rough night (no tone control)
Encryption adds overhead without changing the fundamental properties of the network. If you have a secret you want to share with a friend and you don't want anyone else to know, use encryption. That's an easy problem and encryption works for all cases where anything will work. It won't stop your friend from being careless or an asshole, but that's not something you can fix. You exercise your judgement and hope for the best
What if you want to keep a secret from someone, but you need to coordinate with other people in the process? You share a decryption key with everyone...and? Or use their public key? Any one of them could be "that" friend, and encryption only makes it harder to do damage control when one of them leaks it. What if you left someone out?
Now supposed you make up magic numbers and share them, then you use your magic numbers to make your secret look like gibberish until someone else with the magic numbers reads it. Then you put the secret in an envelope that's opened by the person with the magic numbers and nobody even sees the gibberish except those who have the magic numbers. Using magic numbers didn't accomplish much in that situation. That's why OAuth2 stopped specifying encryption and required a secure transport instead
OCaps reuses "the numbers don't need to be magic" bearer tokens from OAuth2 because it's a cheaper and more reliable way to permit the actions we want to permit the person who received the message we sent on the secure channel than it would be establishing their identity and looking up what they're allowed to do. As a bonus, the scopes are defined by interactions with our server, not the capabilities of the available algorithms
Finally, there's 2 important things encryption introduces that we need to avoid - complexity and false security. There's a long track record of encryption failing adjacent to the cypher text because of the complexity involved
And the benefits of encryption have been oversold. This is a system for planning a birthday party, not a coup. There are people who should be advised that additional steps should be taken if the scaffolding being delivered for boss' day isn't a dance stage. Securing the message flow without encryption communicates the threat model for the primary use case more accurately and provides a basis for encryption to be an effective add-in
Rough night (no tone control)
@yaaps You do raise a lot of good points. On the last one, though, I vehemently disagree. If this is going to be the next evolutionary step of social media, then there *will* be coups, or at least protests, whistleblowing, communication with abuse victims in need of help and secrecy, being planned and executed on here. And surprise birthday parties.
Rough night (no tone control)
We don't disagree about the ultimate scope of the network, just the initial scope. If we fail to reign in the initial scope, the platform either fails to launch or launches with unknown properties and hidden vulnerabilities
For example, http signature private keys are on the server in most implementations. If those were GPG private keys, you would not want to DM your coworker about the guillotine setup, nor would you have a good reason to encrypt that message with a different key
Separating the scopes gives time to establish a broad user base educated in the characteristics of the network before choosing the interfaces to secure with encryption
Server run by the main developers of the project It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!