Wednesday 11 Jan 2023



Rclone


There are many cloud storage providers out there: AWS S3, Backblaze, OneDrive, pCloud, Mega, etc. I like to use a few of these to create cloud redundancy for essential data. It would be a pain to log into each app, or download software for every cloud provider to upload data.

The solution I use is rclone - an open source command-line application to upload/download data. I am a heavy user of rclone: it's literally the only tool I use to upload to the cloud. My use case for cloud storage is back-up only, my primary data are created and edited locally.

After a little reading, it's easy to use. Because it's a command-line application you can write scripts and set-up scheduled tasks to run those scripts. Do not trust yourself to run back ups manually - automate the process to ensure they back-up tasks are completed. Read more about my back-up strategy.

I am not going to rehash how to install, set-up etc. This is clearly detailed in the rclone manual, a place I like to visit often.

Encryption


A small note, "the cloud" is somebody else's computer. You can not trust that they encrypt or protect your data; what if they are hacked. To mitigate this, I encrypt all my data prior to uploading. Rclone allows me to do this seamlessly.

To enable encryption, first create a remote host; by now, you should have created a remote host using this guide. I append "clear" so I know not to upload this, for example "cloud_clear". Then you need to add an encryption configuration on top of the remote.

rclone config

fWo&7$D75d56z9qNU@DLEa#h

rclone lsd cloud://


How I use rclone



Image copright

I use rclone to back-up my data (to the cloud). It's fast, secure and cross-platform. On Windows, I copy the rclone data to C:\Users\Me\ so I can easily run from command line as: rclone config.

This is the command I use to copy data. I do not sync data, I only copy to my cloud providers:

rclone copy C:\Users\Me\Documents\ cloud://documents/

The above command will copy all files and directories from my documents folder to the remote called cloud. This will be encrypted by my local machine and sent to the cloud.

I use rclone in the background via a script that is run on a schedule. I just use a BAT script and Windows Task Scheduler to run the script.

However, to help troubleshoot I like the script to create a log file. To do this, I add:

--log-file=C:\Users\Me\logs\cloud.log --log-level INFO

Log level "INFO" will only log added and deleted files, but if you are having problems with your back-up, or testing, change "INFO" to "DEBUG".

Conclusion


I love rclone - it's one of my favourite command-line applications. I use it often and is an integral part of my back-up strategy to "off-site" data. There is community available for support and rclone continues to be maintained.

Read more about crypt.
Read about rclone mount



Backlinks: Computing:Backup Journal:Index Journal:2023:01 Journal:2023:02:01 Journal:2023:02:06