Posted by afilipovski 13 hours ago
I never though of daylight savings times causing such an issue. Learned something today.
Obviously for production systems automation & monitoring is the way. I kind of like this setup on my own machine though.
https://www.jwz.org/doc/backups.html
(maybe copy/paste this into browser instead of following link, since referrer from hn apparently does something)
This leaves you needlessly exposed to this failure mode: Commit some "braino" that wipes out some recent work. Go to bed without noticing the loss. 5am rsync run wipes your backup as well.
The fix is to back up to not just one but to a rotation of images. Use --link-dest to reduce storage overhead to the size of the directory hierarchy only.
- do several backups (not using the same hardware brand or host if you do it online)
- check your backup (if something was wrong, you have another backup from the first rule)
- do not store your backups at the same place (if your place get flooded or burn, it will be pretty handy)
Also why you sync through LAN only?
Two weeks later they had a fire in one of their huge printing machines which caused quite a lot of direct damage due to heat and rendered a lot of equipment broken due to smoke/fumes. It was also, as it turns out, a week before the first COVID 19 lockdown in the UK.
Their backups landed on a XFS file system with reflinks enabled. I cloned their repo and fired up NFS with their VMs running on our gear. It took me another hour to spin up another OpenVPN server (CA etc) for them to use and sort out a few other details (coffee doesn't brew itself).
They ran via VPN out of our data centre for most of the pandemic.
That was an utter triumph but I also have some rather less triumphant stories about backups and lack thereof. Let's skip over those 8)
https://relax-and-recover.org/
This is two-fold useful, first as it enables boot from backup media for total recovery, second in that it creates a backup.tar.gz that holds everything that was not explitictly ignored in the /etc/rear/local.conf file.
I'm running Oracle databases on these systems (and the local.conf is configured to ignore the datafile directories). I have standby databases (not Dataguard in that I don't have online redolog replication) that allow me to recover them to the primary in a disaster, with some loss of committed transactions.
I finally have rsync configured for some scratch temporary files.
I have dallied with btrfs snapshot replication for home directories (in a loopback mount). My vendor support (via a CSI number) has been on and off, so I don't use btrfs in areas that we need it most.
My replacements do not like the complexity (and I am retiring).
The best part: it is just files, not some proprietary archive. I have high confidence in this. I can restore with the most rudimentary tools.
This setup has worked for me well on linux, windows and now mac. I migrated to new computer&os by restoring the backup mostly.