--- slug: daily-matrix-and-mastodon-backups-with-zero-downtime title: "Daily Matrix and Mastodon backups with zero downtime" tags: [self-hosting, homelab, Matrix, Glitchy.Social, Synapse, Mastodon, PBS, Tailscale, Ansible] image: /img/homelab.png --- As I now have my own [Mastodon instance](https://mastodon.glitchy.social) where anyone can request registration, there are goals to achieve with it. For example, the [Mastodon Server Covenant](https://joinmastodon.org/covenant). There is "Daily backups" among other requirements, and today's story is about that. It involves Proxmox Backup Server, Ansible, and Tailscale as the three pillars of "Daily Matrix and Mastodon backups with zero downtime." The simplest way to back up my Mastodon and Matrix servers, which run in a separate LXCs, was to set up snapshot-type backups on the Proxmox datacenter level and be happy, but snapshotting the database files is not the right way. You can catch your Postgres in the middle of the transaction and leave it corrupted after the restore. The right way was to make a dump of the databases, for example, with [`pg_dump`](https://www.postgresql.org/docs/current/app-pgdump.html), because "It generates a consistent snapshot of the database at the time the command is executed, even if the database is being used concurrently, without blocking other users." But what about the other files? Configuration files, secrets, media storage? Sure, we can `rsync` them away from the server along with the database dump and be happy, but what about backup versioning, retention, and deduplication? As I already had a Proxmox Backup Server to handle those, my plan for backing up a server was simple: Make a database dump to the file system of the server Make a snapshot-type backup of the server to the PBS This way, I'd have consistent daily backups with a database dump and all needed files. And the restore procedure would consist of the same two steps, but in reverse: Restore the server from PBS Restore the database from the dump on the server's file system I'm not pretending to be an inventor of something new; I just think it is a good way. Missing or corrupting several media files during this process is not so critical as missing database records or a corrupted database.