Friday 08 Mar 2024

Testing backups and DFG new feature

I have recently changed my secondary storage provider from Storj to IDrive e2. Once I upload data to a new storage provider, I like to restore a few folders and check the hashes. These tests are an ongoing process for me to have confidence in my backups so, if a disaster occurs, I know I can restore with confidence. In this example, I am testing my photograph collection.

I use my own tool, DEFIANT-FG, to hash the pictures and store in a database file. The issue with 1.2.3 (fixed in 1.2.4) is the fact that if I download one folder, say year 2010, within the main picture folder, during the scan I would get errors for all the missing pictures because I only downloaded one folder. So, I added a new flag (--skip-missing) to DFG to only check hashes.

After testing two folders, I was pleased to see all hashes match. I keep a spreadsheet to document which folders I have tested so I can, over a year, download all data to test.

dfg -d /path/to/file/ --skip-missing