MacOS Backup Strategy

I simplified our backup strategy for our MacBooks. Here’s where I landed:

7 Backup Principles

1. Coverage. A backup should be comprehensive. I try to err on the side of backing up everything and create exclusion lists.
2. Frequency. A backup should be done often–this means it must be automated.
3. Separation. Physical separation of at least one backup to protect against local, regional catastrophes (house fire, hurricane, etc.). I use cloud backups for this. But make sure you have your cloud login and encryption keys stored separately as well! You don’t want a house fire to wipe out your only copy of your cloud login info.
4. History. The ability to pick a date and do a point-in-time restore. This is important to prevent your only backup being the latest version of a file that you corrupted!
5. Testing. On World Backup Day (March 31st), I always do a spot check by restoring a few files. And when getting a new computer, I restore from backup (a great way to test).
6. Security. Backups must be encrypted. But be careful you don’t forget the password. If relying on encryption keys, make sure those are distributed as broadly well as your backups!
7. Integrity. Cold or immutable versioned backups are a must have. It’s the only way to recover after long-unnoticed data corruption.

I’m using two backup solutions:

1. Time Machine (to TrueNAS) for local backups.
2. Arq Backup (to AWS S3 and StorJ) for offsite backups.

Time Machine Backup to TrueNAS (local)

The MacBooks primarily backup to Apple Time Machine on my NAS. I have a local TrueNAS server with a special Time Machine backup share via SMB. All our Macs backup to Time Machine hourly. Those backups get pruned to daily and weekly as they age out. I set a 2TB quota for each Mac’s backup so that it doesn’t grow in size infinitely. TrueNAS automatically creates a ZFS snap upon SMB disconnect (at the completion of a backup). This ensures we have clean immutable snapshots of the backups.

⚠️ ** WARNING ** If you enable “Optimize Mac Storage” on iDrive or “Optimize Mac Storage” for Apple Photos, your Time Machine backups will not include data and photos offloaded to iCloud. This is because the files aren’t on your computer. I set the following:

Settings -> Apple Account (Your name) -> Drive -> I make sure “Optimize Mac Storage” is NOT enabled.

Photos -> Settings -> iCloud -> I make sure “Download Originals to this Mac” is selected.

Arq Backup (cloud) to AWS S3 and StorJ

One problem I’ve had with cloud backups in the past is the restore speed. But I’ve not found that to be a problem recently. With gigabit internet, restoring over the internet is as fast as the LAN.

🧪 If one of Kris & Eli’s home school science experiments blew up the house, destroying the MacBooks and the TrueNAS server, and we somehow survived; maybe I’d want a copy of our insurance policy. I could run over to BestBuy, get a new Mac, go over to a friend’s house with gigabit internet, and download the few files I needed in minutes. Or I could do a complete restore in a few hours.

Arq Backup is designed to perform cloud backups. A family license covers 5 computers.

Arq is one of the few backup tools that can backup data offloaded to iCloud. It can be set to automatically materialize a file in iCloud and then let the system dematerialize it until the file changes again, which allows you to run with “Optimize Mac Storage” enabled (I don’t do this, but a nice option if you have limited space on your computer). Unfortunately this does not apply to the Photos Library. You still want to make sure “Download Originals to this Mac” is selected in the Photos app.

Arq supports object locks on S3, B2, and StorJ, which means it can make cloud backups immutable. It also chunks small files together which helps reduce cloud storage costs.

Cloud Backup Providers

I chose to backup to two cloud locations: AWS and StorJ.

AWS S3 Glacier Deep Archive storage class costs $0.00099/GB/month. The Arq backup data set for my MacBook is 370GB (this includes all my documents, Library, Photos, videos, etc.), so the cost to back it up is $0.37/month. It may be a little closer to $0.42 with the transaction costs.

Retrieval costs: A lot of people mention the retrieval delay and high AWS restore fees and egress fees. But it would only cost around $34.00 to do a bulk (48 hours) restore and download that out of AWS. That’s assuming I needed to restore everything. Chances are most if not all of the data would be available in iCloud. To restore all 3 of our laptops would be around $100. Most flat-fee cloud backup services cost more than that annually.

StorJ is distributed storage network with nodes all over the world. Anyone can run a node. When you upload an object to StorJ it is segmented and split up into 88 pieces–only 29 are needed to rebuild the file. StorJ runs $0.004/GB monthly plus a per-segment fee of $0.0000088. Download is $0.007/GB making it ideal if you need to restore frequently. Arq does a good job at chunking up small files to reduce the number of segments. I mostly use StorJ because I’ve been running a node, so I have a lot of StorJ coins. It’s like trading storage with others node operators.

iCloud (not a backup)

iCloud is a sync and file sharing service. I don’t consider it a backup because its limited version history abilities. It’s better thought of as a sync service. I’ve also noticed it excludes syncing some folders like Videos and some application library data. I’m not confident it would have everything. But, I think it can still be considered as an extra partial copy of your data for some DR (Disaster Recovery) purposes.

Appendix A: MacOS System Restore from Encrypted Time Machine Backups

I use Encrypted Time Machine backups. For restoring individual files you can just use it like normal. Go into Time Machine mode, pick a point in time, and restore the file. Doing a system restore was tricky but here are the steps that worked for me:

1. Create account. Boot computer, create a temporary account (you don’t need to setup iCloud since it’ll be wiped out).
2. Migration Assistant. I found if I used the Migration Assistant, it would try and fail to mount the backup with “mount failed” after entering credentials. In Finder, browse to the TrueNAS share and open up the Time Machine backup. Enter the encryption password when prompted and wait until it mounts. Then run the Migration Assistant. Ignore the TrueNAS server this time. You may need to wait a few minutes, but the mounted Time Machine backup will appear. Select that one.
3. Select the latest point time to restore from.
4. My backup (which was a nearly full 512GB drive with over a million files) took about 12 hours to fully restore over a gigabit wireless connection. The restore is slow. It doesn’t saturate a gigabit. But you can check the TrueNAS network graphs to see that data is transferring. On one of the MacBooks, it got stuck 8 hours in and just hung. I had to start over but it worked the second time.

Appendix B: My Arq Backup Exclusion List

I added exclusions to Arq’s default wildcard exclusion list in case anyone finds it useful… these are things I don’t need backed up. DevonThink3 is already synced to my TrueNAS server which is backed up so I don’t need each and every Mac to back it up, cache, tmp, and temp folders, Logos data, the trashcan, etc.

.DocumentRevisions-V100
.MobileBackups
.MobileBackups.trash
.Spotlight-V100
.TemporaryItems
.Trash
.Trashes
.dbfseventsd
.dropbox
.dropbox.cache
.fseventsd
.hotfiles.btree
.vol
Backups.backupdb
Cache
Caches
DerivedData
node_modules
*/iTunes/iTunes Media/Downloads
*/iTunes/iTunes Media/Podcasts
*/iTunes/Album Artwork
*/iTunes/Previous iTunes Libraries
*/Library/Application Support/CrashReporter
*/Library/Application Support/Dropbox
*/Library/Application Support/Google
*/Library/Application Support/MobileSync/Backup
*/Library/Application Support/com.apple.LaunchServicesTemplateApp.dv
*/Library/Biome
*/Library/Caches
*/Library/Containers/com.apple.mail/Data/Library/Mail Downloads
*/Library/Containers/com.apple.mail/Data/DataVaults
*/Library/Developer
*/Library/Google/GoogleSoftwareUpdate
*/Library/Metadata/CoreSpotlight
*/Library/Mirrors
*/Library/PubSub/Database
*/Library/PubSub/Downloads
*/Library/PubSub/Feeds
*/Library/Safari/Favicon Cache
*/Library/Safari/Icons.db
*/Library/Safari/Touch Icons Cache
*/Library/Safari/WebpageIcons.db
*/Library/Safari/HistoryIndex.sk
*/Library/VoiceTrigger/SAT
*/MailData/AvailableFeeds
*/MailData/BackingStoreUpdateJournal
*/MailData/Envelope Index
*/MailData/Envelope Index-journal
*/MailData/Envelope Index-shm
*/MailData/Envelope Index-wal
tmp
temp
*/Library/Weather
Cache.db
.DS_Store
Library/Application Support/Logos4
com.apple.milod/milo.db-wal
*/Library/Mail/V10/MailData/recentSearches.plist
*/Library/Application Support/Logos4
*/Data/com.apple.milod
*/Library/Assistant
*/Library/Group Containers/group.com.apple.siri.referenceResolution
*/Library/Group Containers/group.com.apple.AppleSpell
*/Library/Group Containers/group.com.apple.replicatord
*/Library/Group Containers/group.com.apple.tips
*/Library/Group Containers/group.com.apple.siri.remembers
*/Library/Group Containers/group.com.apple.spotlight
*/Library/Group Containers/group.com.apple.siri.sirisuggestions
*/Library/Group Containers/group.com.apple.chronod
*/Library/Group Containers/group.com.apple.feedbacklogger
*/Library/Group Containers/group.com.apple.tipsnext
*/Library/com.apple.icloud.searchpartyd
*/Library/Containers/com.apple.news.widget
*/Library/Containers/com.apple.lighthouse.*
*/Library/Containers/com.apple.Safari
*/Library/Containers/com.apple.stocks
*/Library/Containers/com.apple.stocks.widget
*/Library/Containers/com.apple.iCloudDriveCore.telemetry-disk-checker
*.dtBase2
*/Library/Suggestions
*/Library/DuetExpertCenter
*/Library/Saved Application State
*/Library/News
*/Library/Application Support/DEVONthink 3
*/Library/IntelligencePlatform

Ecclesiastes 11:2 ESV:

Give a portion to seven, or even to eight,
for you know not what disaster may happen on earth.

– Solomon