Follow

Any devs interested in porting the Mastodon streaming API from Node.js to Rust?

We would likely not be able to abandon the Node.js version completely for portability's sake, but there is no reason not to have a more performant drop-in for those who can install it.

The Node.js version consumes about 1GB of RAM for 21k connections.

Paid gig, of course. OpenCollective and all that.

Just to update on this, I have found the person to do it thanks to this thread, no further applications are necessary.

@Gargron Rustodon exists, though I'm not sure if that feature's implemented yet github.com/rustodon/rustodon

@Gargron ah well, I thought that maybe some data types and stuff might help someone trying to implement sth

@Gargron

I'm new to rust—only a couple of months and a couple of apps in—so I don't know if I'd be able to handle this solo or not. But I'd love to try/help in any way I can.

(I do know JS, which would help with porting, though I haven't done much with streams in either language.)

@Gargron Ok, I was interested before you said that, and am more interested now. Reading over the JS file now.

@Gargron

Ok, having looked over the JavaScript file and having gotten a sense of what it accomplishes, I'd definitely be interested in the #rust port and definitely feel like it's something I could manage.

@Gargron I'll throw my hat in. About a year of Rust experience

@ByAndBy Do you have a homepage or GitHub I could look at?

@Gargron why choose rustlang instead of some other languages(e.g. golang)? (this question is just interest)

@rinsuki I have looked over some benchmarks and Rust seems to come above Go and Crystal in terms of memory consumption. Considering that I am not proficient in any of those languages myself, I picked the most promising in terms of performance.

@Gargron @rinsuki what benchmarks have you looked at?
Usually Go performs great with "web stuff".

I can see the superiority of a completely safe language though :)

@Gargron I have experience in DLang if that has any interest, I do know how to do custom allocators and such as well in D.

@Gargron Did you write that request to the Rust IRC? This won't be a big thing for a trained Rust dev. I would like to try this myself. But I wouldn't want to get paid for it, since I'm usually doing Rust professionally

@Gargron So, I took a look at the file. I'm a little short on my nodeJs knowledge but I think I might be able to port this on a weekend or so. BUT, I might need some help understanding what's happening in the file. Would you able to spend some time pairing on that?

@sphinxc0re Yes, I'd be available. Also, I have the OpenCollective $70k grant with the explicit purpose of getting more developers involved with Mastodon, so don't do it for free.

@Gargron Okay, then I have to check with my employer first. Is it urgent?

@Gargron Okay, I would love to be able to help the project. Also, I like throwing away the nodeJS dependency in production 😁

@sphinxc0re The problem is that Node.js is still required for webpack, so it cannot be removed completely.

@Gargron Sure, but I mean Webpack is not really a production dependency, right?

@sphinxc0re

I'd be happy to collaborate if you'd like the help—I'm primarily a JavaScript developer and have gotten in to Rust recently-ish, so it sounds like we might make a good team. Here's my latest #rust project, in case it's of interest: github.com/codesections/mnemon

(Also, I coincidentally have next week off work due to my employer's schedule, so I'd have some upcoming availability).

@Gargron

@codesections @sphinxc0re I am okay with this if the work can be split between two people

@Gargron I don't have enough time to commit to an implementation, but I'm happy to do a code review and answer questions (for free). Ping me github.com/kornelski if any Rust issues come up.

@Gargron May I ask why Rust and not Go?

Currently deciding on which of them I should learn, so I'm just curious :)

@Trollwut @Gargron

Handling lots of connections efficiently sounds more like Erlang's domain (or these days maybe Elixir) rather than rust or go.

@h_d @Trollwut Yeah but distributing a binary is a bit more straightforward than making people install and configure an erlang environment

@Gargron would be interested to do this as a side-project to learn Rust

@Gargron once you’ve done it, just package it up as wasm and have it as a dependency of the original - they get a free speedup! Everyone’s happy!

@Gargron @wezm I could (most likely very quickly) write it up in ObjC using @ObjFW. ObjC basically combines the speed of C with the object orientation of Ruby, while ObjFW provides all the stuff including web related stuff that you would expect from node.

I realize though that this is not a very popular choice as it’s not very well known.

@Gargron could the Ruby part of Mastodon also be ported to Rust?

And for that matter, the web front end as well? (at least, I'm pretty sure it supports compilation to WebAssembly)

Sign in to participate in the conversation
Mastodon

Server run by the main developers of the project 🐘 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!