Backups #

Backing up your files is very important. Relying on any single cloud provider or piece of hardware is dangerous. Protect your valuable, irreplaceable digital assets as you would valuable physical assets: think carefully about how to keep them safe from disaster. Indeed, with digital assets, you can and should achieve the multiple levels of redundancy that you can’t with “real” assets (you can’t duplicate your cash and store it twice to keep it safe).

Principles #

You should use a password manager, with proper end-to-end encryption, and store a physical copy of whatever information is required to restore access (usually some sort of cryptographic information) in two distinct, safe and trusted physical locations. Even better is to split it cleverly across three locations in such a way that you need two of the three to have the full data (Shamir’s secret sharing).

You should have all your files on your computer, with your disk locked and encrypted by a secure password.

You may want to have all those files backed up to a cloud storage service with native integration with your computer and/or phone, like iCloud Drive or Google Drive, behind a secure password. Note that these services are not usually end-to-end encrypted, by design.

You should have the files from your computer backed up to an external hard drive on a regular basis. This backup should be encrypted by a secure password.

If possible, this external backup should be mirrored, at least once locally (e.g. RAID1), and ideally both locally and remotely. These mirrors should be encrypted by secure passwords.

My system #

Hardware, software and services #

My system, still not perfect but pretty good, is built from:

  • Hardware

    • Computer

      A 14" M1 Pro MacBook Pro (unfortunately with only 512GB storage).

      • Update

        I tried to use a Transcend JetDrive providing another 512GB of (much much slower) storage: a constant reminder that next time I should just pay more for extra storage. Unfortunately the drive failed and my backup photos library got corrupted, only a few days after I bought it. It was also extremely slow. Recommend steering clear of this product.

    • NAS

      A Synology DS220+ NAS with 2x8TB drives in RAID 1.

  • Services

    In order to get multiple levels of redunancy you either have to be a really serious and seriously confident Unix greybeard, and own your own servers in multiple physical locations, or you will need to pay someone for hosting and storage.

    • iCloud Drive (6TB plan: $30/month)

      I am very deep inside the walled garden and iCloud Drive is very nicely integrated with all my devices. I really don’t like the fact that it isn’t end-to-end encrypted: this seems like a fairly high price for tech-savvy people to pay on behalf of the non-tech-savvy masses, since if there really was end-to-end encryption there would be a very significant burden on the user to handle their password and keys responsibly that many people just do not understand.

    • rsync.net hosting (680GB plan with geo-redundancy: $17.50/month)

      rsync.net is a nice service that allows you to buy a simple server that is specifically designed to be used to back up files (in technical terms, you are renting some Linux machine with a ZFS file system). My server is located in Europe and I manually back my files up to it every now and then (this should be automated). I use borg (see below), which is smart enough to do incremental backups.

      They also have a great geo-redundancy feature where they will backup an additional copy of your files, without you having to do anything, for another 75% of the cost. So this gets me another remote copy in the US.

    • 1Password ($50/year)

      All my passwords are managed through 1Password. This service is end-to-end encrypted, and if you lose your master password and recovery private key, you cannot recover your passwords. I have my private key printed out and stored in two very distinct and safe physical places.

  • Software

    • Time Machine

      Time Machine is Apple’s incremental backup software. It has a nice interface where you can “go back in time” and look at your files in previous states that I don’t think I have actually ever needed to use. It’s slow, but after you pass the hurdle of the initial backup, which can take days, you don’t really notice it and it just works in the background. I use it only to backup my ~100GB of documents, since I haven’t had a lot of luck with it backing up photos.

    • borg

      Borg is a fantastic command line tool that performs incremental backups to remote repositories. It has a similar syntax to git and it works really fantastically and fast and can be connected up with scripting to do all sorts of fancy automation. I am constantly impressed by how fast this is compared to the lumbering slowness of Time Machine even though they are doing exactly the same thing.

    • icloudpd

      icloudpd (photo downloader) is a script for downloading all your photos and videos from iCloud Photos. I run it in a docker container on my NAS.

Process #

In order to replicate my setup, apart from buying the hardware and signing up to the above services, you will need to:

  • Set up the NAS so the two drives are in RAID1 (there are many guides on how to do this online).

  • Turn on iCloud Drive and make sure you turn “Optimize Mac storage” off so all your files are always locally available on your machine (without an internet connection).

  • Turn on Time Machine, setting the NAS (visible to macOS as a network drive) as the location.

  • Set up borg. This depends on exactly how you want to use it, and you should start by reading the guide here.

  • Set up icloudpd. This took a little longer for me to set up, but I got there in the end.

This setup provides you with six (!) copies of your data:

  • all documents stored on MacBook Pro
  • all documents backed up to iCloud Drive (and photos backed up to iCloud Photos)
  • all documents and photos backed up to NAS with Time Machine (2x redundant with RAID1)
  • all documents and photos backed up to an rsync.net server with borg (2x redundant with geo-redundancy)

I only do the full six copies for my Documents folder, and five for my iCloud Photos. My other stuff is mostly media (music, films) that I feel is disposable enough that RAID-1 should be enough. If something really catastrophic happens I will be much more worried about my 20 years of documents than a few episodes of Seinfeld.

Colophon #

Cover image generated by stable diffusion.