Yo yo yo! Remember I promised to write about scaling mastodon.social to 43k users? I did it! https://medium.com/@Gargron/scaling-mastodon-1becde463090
@Gargron What about querying database slaves? I notice few FOSS web projects that offer support for that.
@blacksam Didn't have to go there yet (also - I prefer read-only replicas terminology) but it's possible to implement
@Gargron I assume the db activity is somewhat heavier on reads?
@Gargron You're missing some images to agrement the text ;)
@Gargron 8 workers with 2 threads ? Wow, seems like a serious problem :(
@TouitTouit What do you mean?
@Gargron I mean Ruby is poor with multithreading (understand system threading). Puma was designed to combine multiple processes and multiple threads (Ruby threads). If you have more processes than (Ruby) threads, you're obviously do it wrong.
@TouitTouit MRI is not the only Ruby out there. There's Rubinius and JRuby. Puma's threads might work better there.
@Gargron sure. You and I know that.
@Gargron especially when system threads for Puma are shared with Sidekiq
@TouitTouit Threads are not shared between Puma and Sidekiq in any way, what do you mean?
@Gargron system threads, sure they are ! How many admins understand what's the difference between Puma and Sidekiq ?
@Gargron system threads. Puma needs CPU to answer requests, while Sidekiq wants some. Processes are racing
@Gargron I have to sleep, I'll be glad to chat and explain why (in my opinion) your paper is misleading
@gargron This is awesome, thanks!
@Gargron good stuff! haven't heard of pgbouncer before, so thanks for introducing me to it.
@Gargron Thanks for that, that is already quite an impressive setup. I was hoping to start out with a single VM. Maybe need to rethink that :)
@wonderfall je dois actualiser manuellement ma timeline dans amaroq. Je me demande si ce n'est pas en raison des paramètres Nginx en lisant l'article de @gargron
@Gringo_93 Non pas du tout, qu'est-ce qui te ferait penser ça ?
@Gringo_93 J'avais déjà appliqué les paramètres qu'il préconise avant.
@Gargron Really interesting to read, thanks for all your good work. :)
Out of curiosity, any plan to move in the direction of something more horizontally scalable like MongoDB in the coming months? I guess that's the main thing stopping the current version supporting *really* big instances.
@Gargron I'm also looking at seeing if I can deploy it on #kubernetes when I get some free time. I have a bit of experience from work and it's a good way to go for resilience and scaling.
@followthesam @Gargron I run my instance on kubernetes and some cheapo nodes, happy to answer any questions or share some code as necessary!
Highly recommend using helm if you aren't already!
@sharktopus What do you use for a DB? Postgres in a container? Not used helm, must read up on it.
@followthesam for now, yes I'm just running postgres in beefy container(s). My approach with this instance is basically to just do things the /wrong/ way but not shoot myself in the foot, and then improve whatever is stressing the most rather than trying to predict what is going to break. I'd like to try out the google cloud postgres offering but can't justify spending the money for it just yet.
@followthesam but re: helm, what are you using to do templating of your kube configs and track deployments and such? I haven't published my helm configs for my instance because they have sensitive info in my local repo but I'd be happy to scrub them manually and share the current version of the flatfiles if you or anyone else would like to take a peek.
@Gargron Cheers for the about. Interesting read :)
@Gargron Thanks very much ! :)
@Gargron why are you using medium? Honest question.
@Gargron This is so great! Thank you :D :D