@tim Everything is working fine on my end. What version were you before the upgrade? Did you jump multiple versions?

@mastohost no, 3.2.0 to 3.2.1. Seems maybe related to certificates/auth keys, might be specific to the relay software (federating between mastodon instances works fine, or I'd not see your post :-)). As ever, when it works it works and when it doesn't you're on your own; hopefully I'll have time to dig in deeper at the weekend...

@tim hummm... what steps did you perform during the upgrade?

@mastohost I changed the image tav from v3.2.0 to v3.2.1 and redeployed the Docker container 😁. Nothing else indicated in the readmes/changelog :(.

@tim so, docker-compose down and docker-compose up -d? you are using tootsuite from hub.docker, right?

@mastohost Kubernetes, so the equivalent using helm. And yes, tootsuite/mastodon.

Rolling back doesn't fix the problem (it just changes it to http/202 errors instead of 401), so it persistently breaks something. Makes me suspect that it's borked wherever it stores the key/certificate it uses to validate the relay's key; whether that's redis or postgres I don't know, but that's where I'll start digging at the weekend...

I do appreciate your taking the time, by the way! Thanks.

@mastohost: After doing a little more digging, I strongly suspect this change is to blame - https://github.com/tootsuite/mastodon/pull/14556
- which somewhat breezily breaks compatibility with other suitably 'old' Fediverse software. Which doesn't seem a super smart thing to do in a point release, at least not the way I understand SemVer...

More interesting to me now is why downgrading back to 3.2.0 didn't seem to fix things 😐

@mastohost: OK, to answer my own question downgrading did fix the problem after deleting and adding the relays again.

So, short version - don't upgrade to 3.2.1 if you value federating with "old" Fediverse instances, 'cos the new changes to signature validation break it.

@tim I was suspecting this one would be the problem: https://github.com/tootsuite/mastodon/pull/14919

If you have some custom setting for "/.well-known/webfinger" that could also affect you. Not sure how you are handling that in Kubernetes.

But yep it can also be https://github.com/tootsuite/mastodon/pull/14556 although it says that is compatible with Mastodon.

If you can find the culprit that would be great information to share on GitHub with the dev team.

@calvin_thefreak Hello, that is really strange but your instance is the only reporting something like that, so I assume there is something on your end. I only see attempts to follow @nitro on my servers logs. Can you please let me know what is your real server IP and do an attempt to follow my personal account @hugo so I can try and help you debug this?