Remember on Monday when I had caused an issue by changing some configuration, but then I fixed it? Well, I think I didn't quite fix it properly until today. Noticed it when my own toot didn't appear on the home timeline. I think that might have been... happening.

Lesson: Do not use read-replication on the Sidekiq processes. A job might execute before the data is available on the replica, causing it to just disappear.

Follow

Me: Why does Mastodon feel kinda quiet suddenly?

*Sidekiq jobs are literally disappearing into the void because of replication lag*

Me: :pika:

@gargron i have a great monitoring system i'm just watching the timeline and shitposting all day

@Gargron I've seen a drop in interactions until today when I've seen a huge burst. I think that's driven by serendipity than a technical thing though.

@hankg You might not get as many interactions if your posts don't appear in home feeds of your mastodon.social-based followers due to a technical glitch.

@hankg @Gargron
LOL Yes, not sure if it is from it being 'quiet' or if it was just due to me only having 6 followers..LOL I'm sure it's a bit of both

@Gargron That explains a few things.

Is there a way to monitor the activity of Sidekiq such that if there's too much of a lag, it triggers a run?

Sign in to participate in the conversation
Mastodon

Follow friends and discover new ones. Publish anything you want: links, pictures, text, video. This server is run by the main developers of the Mastodon project. Everyone is welcome as long as you follow our code of conduct!