Upgrade finished. All servers hosted by Masto.host are now running v3.4.1

Any issues please let me know 馃悩 thanks

Show thread

After testing v3.4.1 for 48 hours the issue that I reported has not caused any problems.

So, I will be starting the upgrade to v3.4.1 for every server on Masto.host. The downtime during upgrade should be less than 30 seconds.

You can see the v3.4.1 release notes here: github.com/tootsuite/mastodon/

Any questions please let me know.

Show thread

There is new release of Mastodon ( v3.4.1 - github.com/tootsuite/mastodon/ ) but it introduces a strange warning github.com/tootsuite/mastodon/ when running the upgrade commands.

It's probably not a problem and wouldn't affect the upgrade but out of an abundance of caution, I will wait to see if someone reports any problems related to this issue or if a patch is released.

So, the upgrade to v3.4.1 will be delayed for the time being but I will announce as soon as I have more information.

Upgrade finish. All servers on Masto.host are now running Mastodon v3.4.0

Any problems please let me know. Thanks 馃悩

Show thread

I will be starting the upgrade to Mastodon v3.4.0 for all servers hosted in Masto.host

During the upgrade there will be around 1 minute of downtime.

You can find the v3.4.0 release notes here: github.com/tootsuite/mastodon/

Any issues or questions please let me know.

Data center incident not affecting production 

Finally the new media backups have been copied to the new data center and the daily rotation is starting now. In the next couple of days it should be running daily for every account.

The old Object Storage (PCS) is still "Under investigation" by OVH to see if the data is recoverable: ovhcloud.com/en/lp/status-serv - still, even if it's recovered I will end up deleting it as the new backups are up to date.

Any questions please let me know.

Show thread

Data center incident not affecting production 

All going as expected, the backups for database, configuration files, SSL certificates and other config files will be fully copied today and will start the usual daily copy to the new data center tomorrow.

The media files it will take some days (or a couple of weeks) to be fully copied and doing the daily rotation that I had in place.

Still, hopefully the old backups will be retrieved but if not the new backup is already going.

Show thread

Data center incident not affecting production 

Replies from OVH about restoring their Object Storage from the data center have been vague. So, I can't be sure of a recovery of the backup data.

To try and be proactive in case of non recovery, I have started the backup from zero to another OVH data center in Warsaw.

The database backups from the past 2 days are currently being copied there and will start backing up media files later today.

Show thread

Data center incident not affecting production 

The reason to locate backups in a different data center is exactly to allow for a situation like the mentioned fire to be recovered from.

Will see if I can get a confirmation if the backup data was lost or not. The backup data was hosted on their Object Storage so hopefully it was not but can't be sure right now.

I will keep you posted when I have more information.

Show thread

Data center incident not affecting production 

There was a fire on one OVH data centers (Strasbourg) travaux.ovh.net/?do=details&id and from what I could understand the building is lost.

In case you don't know. I use OVH to host all of Masto.host but thankfully everything is hosted on a different data center in France, with the exception of backups. The backups were in the building that burned.

So, everything is working in production, just not sure if the backups from the last days were lost.

Because I am getting too many strange signups in the last 24 hours and most people don't read the Terms of Service, I added a Temporary Notice to Masto.host Pricing page: masto.host/pricing/

"If you are looking for an alternative to Parler, this is not it. Please look elsewhere. Thanks."

I hope that's clear enough :)

Changes to the SMTP finished.

Please let me know if you notice any problems with email deliverability. Thanks 馃悩

Hopefully this will be the last maintenance service of 2020 :)

Show thread

I will be changing the SMTP service (what is used to send all emails on the platform, unless you specified to use your own SMTP) from MailGun US to MailGun EU and this will require a restart on all services.

This will cause ~15 seconds of downtime.

Any questions please let me know.

Upgrade finished. Every Mastodon server on Masto.host is now running v3.3.0 馃帀

Please let me know if you run into any issues. Thanks 馃悩

Show thread

I will be starting the installation of v3.3.0 for every Mastodon server on Masto.host.

This is a big one 馃

The downtime depends on the size of your database but I expect that for most instances it should around 1 minute.

You can read the change log here: github.com/tootsuite/mastodon/

"What am I doing on the 25th of December?" you might ask. I am fixing a bug.

There is a small configuration that I forgot to change during the migration to the new CDN and that is causing problems with Mastodon audio player.

I will be doing a restart to fix this. You should expect less than 30 seconds of downtime.

Thanks 馃悩

Upgrade finished! Every Mastodon server on Masto.host is now running v3.2.2

Any issues or questions please let me know.

Thanks 馃悩

Show thread

I will be starting the upgrade to v3.2.2 on all Mastodon servers on Masto.host

There should be less than 30 seconds of downtime.

You can see changelog here: github.com/tootsuite/mastodon/

Notice that the changes that I applied on v3.2.1 will also be enabled:
mastodon.social/@mastohost/105

Any questions please let me know.

Migration to the CDN finished. Everyone on Masto.host is now getting media files via the new CDN.

Any issues or questions please let me know.

Thanks 馃悩

Show thread

I have decided to change the media files CDN for Masto.host. I will start making that change now and this will involve a restart (~30 seconds of downtime).

You can read the details here: masto.host/new-cdn-for-media-f

Show older
Mastodon

Server run by the main developers of the project 馃悩 It is not focused on any particular niche interest - everyone is welcome as long as you follow our code of conduct!