I wanted the ability to check file integrity within a directory structure; this would be useful should I ever need to restore from the cloud or test that an external hard drive is telling the truth (help detect bitrot). As I have thousands of pictures doing this check one-by-one would be a pain.
MD5Deep is an excellent tool I can use to check every file within a directory structure. As my picture directory does not change, it was a great starting place to test this script. The application will check each time, generate a checksum, and store within a defined text file. You can choose different hashing methods. In my case I used SHA256.
Let's get started.
Install the hashing tools:
sudo apt install md5deep
Once installed, let's get hashing. This may take some time if you are hashing a large directory.
sha256deep -r /foo/bar > hash.sha256
The -r switch is recursive so will follow directories. You can store the hash file whenever you want, such as ~/Documents/hashes/pictures.sha256
Great, you now have a list of file hashes to play with. Back these up and you're good.
Once you are ready to check the directory against your hash file, use this command:
sha256deep -X hash.sha256 -r /foo/bar
If you want to watch progress, add -e to the end. However, because I was hashing pictures and short videos, the progress hashed so quickly this was not helpful, unless hashing large files.
Now that I have a list of hashes, I can be more confident my working hard drive is not failing and corrupting files. I can run this check monthly like my own "scrub" tool. Should I need to restore files I will know whether they are damaged.