Summary of the API changes:

Pleroma emulates the Mastodon REST API to piggyback on the Mastodon apps that already exist (fair enough). Pleroma changes IDs returned by their emulated API, from numbers encoded as base 10 strings, to numbers encoded as base 62 strings (difference being the method by which they can be sorted in typed systems) (1/2)

Follow

Pleroma users send complaints verging on extremely rude to Mastodon app devs, justify it with the fact that Mastodon documentation only specifies IDs being encoded as strings in JSON, not that they are in base 10 or 64 bit. I posit that the Mastodon API documentation is descriptive rather then prescriptive, and the API was never intended to be implemented by non-Mastodon software, and therefore prior assumptions of app devs based on Mastodon API's observable behaviour were not wrong. (2/2)

I do not have any judgement to offer here, except that app devs are definitely not to blame here, and my intent with this summary is to clarify the situation.

It is regrettable that I was only notified about this after the ID change already took place, because we could have clarified things and involved the Mastodon app devs, and have some say in the matter.

Now clarifying an already existing API of ours would be exclusionary towards Pleroma. If you heard about our documentation changing to specify IDs as 64 bit numbers yesterday, it has been changed again today afternoon to keep Pleroma compatible.

unhelpful and probably unhealth sniping that's probably dramatically overstating the drama while also possibly risking increasing it by being this sort of spectator 

@gargron "app devs are definitely not to blame here"

making assumptions about an API is totally not the assumer's fault, not at all

@a_breakin_glass I made assumptions, which I checked with Eugen and the Mastodon devs, and they confirmed they were good and intended assumptions.

@WAHa_06x36 how? the type of integer encoded by the string isn't specified

@Gargron I would never assume an ID string in JSON would ever be a 64 bit number. Twitter went string in their ID (unfortunately they had to have id_string, because they were dumb when they first came out and used a 32 bit number on the key "id"). Also, due to JSON's idiocy on number overflow, a string is best. If someone made an assumption on their client, that is their issue, not Mastodon's, IMHO.

@Gargron "the API was never intended to be implemented by non-Mastodon software"

that says it all; no blame/anger is justified.

@Gargron

there have not been any complaints sent to Mastodon app devs, rude or otherwise.

we have a growing set of relationships with app developers who are committed to supporting both Mastodon and the Pleroma API extensions. if we were hostile to devs, that would not be happening.

it is our policy to only discuss with app devs who have expressed direct interest in supporting Pleroma. that includes some apps that support Mastodon, others that support GNU Social and others that support Misskey as well as Pleroma.

I find your claims that we have been rude and demanding with app devs who have expressed no interest in supporting our project to be disappointing, and as explained above patently false. please correct your statement.

@kaniini @Gargron He said Pleroma users, and let me tell you, you definitely have some seriously rude users and they have been sending their rubbish my way.

@WAHa_06x36

I'm disappointed to hear that you have gotten some hate mail, but as we have explained to the Pleroma community, we only support clients which are certified to work with Pleroma.

those are the ones in the readme file, and Toot! has never been on the list, so it's really their fault for depending on a client we never certified to begin with.

it is unfortunate that Pleroma uses 128-bit IDs, and it is unfortunate that you have no interest in partnering with us.

we solicited input on the FlakeIDs scheme several months before implementing it. had you been working with us, you would have been aware of it and could have requested we stick with a 64-bit ID.

unfortunately, now, it is too late. the FlakeIDs change works fine with the clients we officially support and certify as Pleroma-compatible.

indeed, it is also unfortunate that users bought Toot!.app without reading our certified app list, but obviously that is something out of my control.

at any rate, if you would like to release your app under free license terms and support FlakeIDs, it is always possible to work together in the future. considering a user added features to Pleroma explicitly to support Toot! despite knowing it is a non-certified app, I would say there is interest, and in interest there are business opportunities.

something to think about anyway. more customers are always better than less, no?

Long post 

@kaniini @charlag Incidentally, though, if you wanted source code access to verify it I'm happy to give it to you, but that would obviously be if I can get it it working with Pleroma again.

@WAHa_06x36 @charlag

I would be open to looking at the code and adapting it to efficiently handle IDs of any sort. I've been doing systems-level programming for the majority of my life so I believe I can come up with something that will suit your needs that will meet or exceed your current hashtable performance, while also working well for us.

i do not have an iOS device though, so I won't be able to test effectively.

@kaniini @charlag If you have a Bitbucket account, I can add you to the project.

@WAHa_06x36 @charlag I do, I will have to reset the password because I haven't used it in years. will circle back.

@kaniini How are you planning to handle web push notifications in Pleroma? Notification payloads have a number-typed notfication id field.

@WAHa_06x36 Pleroma already supports web push notifications. Notification objects indeed have a numeric ID (notification_id) and that is presented accordingly.

@kaniini All right. Looking into what is needed to use string IDs now.

@WAHa_06x36 i suggest using a TST structure instead of a hashtable, it will be more memory compact (unless the hash function is very well balanced across bucketspace) and lookups will be more efficient (no need to hash the data, just walk the bytes in sequence).
@WAHa_06x36 (if you need help designing and implementing a TST, the door is open, I have just been very busy.)

@kaniini Robin Hood hashing lets you get very compact hash tables, so it is already about as compact as it can get, and really fast. Also, touching that is a bit too likely to introduce problems so I will just use hashes of strings there when needed. The main thing to fix is the timeline cache format, which has so far just been a long array of ints.

@kaniini The current cache format is basically a fixed-size hash table at the start of a file, followed by actual entry data. The hash table contains (key, offset, timestamp) tuples. This works great for a cache, as you can just mmap() the start of the file for really fast lookups of where the actual data is. Occasionally, the oldest entries will be discarded and the trailing data compacted.

@WAHa_06x36 yeah, makes sense. also makes sense why you got hit so hard by the IDs issue.

@Gargron bit dishonest as only one app was effected afaik and most app devs are on good terms without pleroma devs with no "rudeness" at all

@doublah At least two apps, and there has been plenty of rudeness going about.

@WAHa_06x36

no, it's really just you. Subway Tooter was effected but the author works with us and fixed his implementation quickly. Subway Tooter failing testing in fact delayed deployment of the change by a week.

we did not deploy the change until all clients on the list of officially supported clients were green.

@doublah

@kaniini @WAHa_06x36 @doublah what about you said? ST still use long id in mastdon-like servers.

@tateisu @doublah @WAHa_06x36

well Pleroma is mastodon-style but uses IDs like Misskey. I guess it wouldn't be that hard for you to adjust ST for that quirk.

@kaniini @WAHa_06x36 @doublah I do not feel like making a list of problems myself. I welcome issues and PR in ST's repo.

@Gargron I'm slightly irritated by "the API was never intended to be implemented by non-Mastodon software". I'm a strong believer of the open source principles that developers of different systems should support each other.

I think it would be best if API users (app developers) and API provider (server systems) could coordinate each other, since we all do share only a small part of the whole cake and we shouldn't fight against each other. Instead we should combine our forces to improve the whole fediverse.

@heluecht I think maybe you misunderstand. The Mastodon REST API (client-to-server) was designed specifically for Mastodon, in stark comparison to ActivityPub, which was designed by a working group as a standard through a bureaucratic process at the W3C. Pleroma or Misskey emulating it was not an intended use case that was planned for. It's an "on your own risk" thing that they do.

@Gargron @heluecht Will Mastodon will consider implementing the C2S #ActivityPub protocol, and not just the S2S part?

@Gargron That's why documentation and specification are seperate documents …

Sign in to participate in the conversation
Mastodon

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!