Follow

I had to block some external instances on Masto.host shared hosting.

Here is why and how that can be reverted: masto.host/i-had-to-block-some

This was a tricky decision but couldn't think of a better solution.

If you have any suggestions, please do let me know.

@parkgaedae your instance is ok but can't get federation from the listed instances

@mastohost
mastodon.bitpage.de has an email contact on their about/more page:

cp@bitpage.de

@frankiesaxx Thanks :) I have and will continue to try to contact the admins one by one.

@frankiesaxx yep, combined with their profiles not even responding when trying to load them via Mastodon.social :|

@mastohost
Hopefully people will keep boosting and it will get sorted. I assume they aren't blocked from Mastodon.social so they should see the boost. I did @ the Japanese and French instances in case they don't pay much attention to English

@mastohost
haha hope they get it sorted. I guess friends.nico might have some issues scheduling an upgrade, they're massive

@mastohost I'm trying to read the about (just for pure curiosity) but it seems I can't connect to the host; Firefox complains about connection timing out.

Is the server overloaded or am I behind some blocked IP? (I don't run any instances, so I'm not sure what's going on...)

@juliobiason could you send me the link you are trying to access so I can have a look?

@mastohost "masto.host/i-had-to-block-some

it's being resolved to
[13:18:23] julio.biason@dts-bhs-ovh007s:~$ host masto.host
masto.host has address 51.255.95.5

DuckDuckGo says my IP is
Your IP address is 177.19.162.5 in Porto Alegre, Rio Grande do Sul, Brazil

@juliobiason somehow your IP was in CSF on the server where I host Mast.host, could you please check it is working now.

@juliobiason awesome, I had played with rules of CSF because these last days Claro.com.br was sending a lot of login attempts and somehow think I blocked the all requests coming from Brazil to that server :P

Sorry about that, it has nothing to do with the block being discussed, different servers, different situation.

@mastohost Hey, don't worry. Those things happen.

And our internet providers (Claro is a mobile one) aren't the greatest about security and "being a good internet citizen".

@mastohost shouldn't that also be fixed in a newer mastodon 2.4.x version?

@mastohost what i mean is: 2.4.x should be updated, that it doesn't cause this kind of behaviour when connecting to an older version

@hirojin not sure I understand.

The problem is their instances being on 2.3.x, they send requests to instances with 2.4.x and start going in loop sending more and more but this only happens in very specific cases.

@mastohost, that’s a tough position to be in, but it seems like the right decision was made. Have you heard of this affecting other servers? Does the bug affect you more because you’re running multiple instances or is that irrelevant?

@iiogama It's not because I am hosting multiple instances.
The same will happen for single server hosting.
The specifics of why and when it happens, I am not sure. I know it has to do with pinned toots.

@mastohost Is the bug tracked somewhere on the Mastodon project? If the maintainer(s) know about it, maybe they could help with a fix for 2.4.x boxes.

Even if the fix is to auto-reject requests from old instances that could cause this bug, at least then you wouldn't have to manually keep checking if they've upgraded yet.

@glcls I don't know if I should open an issue on GitHub for an issue that I don't know what causes it and how to reproduce it, I just know that it has something to do with accounts with pinned toots being followed from instances running older versions of Mastodon.

What do you think? Should there be an issue opened for this?

@glcls OK, it looks like this has already been reported and Nolan has even made a hotfix for it: toot.cafe/@nolan/1006345209326

@mastohost I'm pretty sure that this affects some of the people I follow. :( I can understand why this was done, and I would say it is indeed a wise decision. I'm not sure if there is a sense that there is a good way to handle this type of thing which though not fixing the initial problem, but would somehow mitigate it in a way that it doesn't affect things adversely like this. And also somehow be able to do a "this seems to be a problem" message to the admins.

@JigmeDatse The only way I can think to avoid this kind of situations is for Mastodon dev team to implement a system that will stop requests to an instance upon a certain signal for X amount of time.

But that would need to be implemented in future releases and both instances needed to be in a version higher than that release.

That's the only thing that comes to my mind right now.

@mastohost I was thinking that a "throttle" type feature which would drop the requests, or respond with a 500 level response to indicate that there has been throttling. It would *still* mean the requests get to Mastodon, but if the processing and data drop dramatically, it could be helpful. Then I'm not sure how easy it would be to send some sort of message to the admin saying "your instance has been throttled x times in the last 24 hours, please upgrade to version X or later."

@JigmeDatse yesterday I manually built a redirect for those request to return 404 and it was like that for hours and it didn't slow the requests down. It was killing my nginx, I had to use iptables to block them or I the servers would be overloaded with requests.
So, I think that it should be something that would stop the request at the source and/or respected a signal to slow down/stop

@mastohost Darn, so even with not processing them with Mastodon (it sounds like you were handling them straight in nginx) it was still far too much.

@JigmeDatse around 5 requests per second per instance. Now multiple that for about 20 instances that I had yesterday doing that :|

@mastohost Well, I'll probably will have to deal with these sorts of things once I start looking at my own instance.

@mastohost It looks like the issue you're running into is the same one I experienced on toot.cafe on v2.3.3 a couple months ago. github.com/tootsuite/mastodon/

I had a hotfix that seemed to work; maybe you can ask instance admins on v2.3.3 who can't upgrade to apply the hotfix? github.com/tootcafe/mastodon/c

@nolan This is awesome, thanks.

It really looks the same thing, pinned toots are my main point of reference in this bug. I will send it to the admins that already responded :) as always, great work, Nolan

For instances that cannot upgrade from v2.3.x, they could try and apply the hotfix that @nolan created: github.com/tootcafe/mastodon/c months ago and that was reported here: github.com/tootsuite/mastodon/

@mastohost I sent the admin of friends.nico a DM in Japanese explaining the issue (I missed the patch downthread, but I will send it to them if they follow up)

@mastohost you are wonderful!! thanks for being such a caring superadmin

Sign in to participate in the conversation
Mastodon

Follow friends and discover new ones. Publish anything you want: links, pictures, text, video. This server is run by the main developers of the Mastodon project. Everyone is welcome as long as you follow our code of conduct!