Backing up with rclone

Why back up

Scratch files are automatically deleted after 45 days, and project spaces are not backed up. Anything critical needs a regular backup strategy — Stony Brook BOX is the preferred destination, offering unlimited storage that supports high-risk and sensitive data, accessible from any device.

Setup

1. Load the module.

   module load rclone/1.74.4

2. Configure a Box remote.

   rclone config

   Select Box from the storage list, leave the credential fields blank, and choose No for auto config (SeaWulf is headless).

3. Authorize from your own computer. Install rclone locally, run rclone authorize "box", sign in, and paste the generated token back into the SeaWulf config prompt.

Google Drive works the same way — pick Drive in rclone config and authorize with rclone authorize "drive".

Transfer commands

rclone copy ./myfile.txt my_box:seawulf_backup          # single file
rclone copy ./mydir/ my_box:seawulf_backup/mydir        # directory
rclone sync ./mydir/ my_box:seawulf_backup/mydir        # mirror (deletes extras!)

Performance

• Transfer speeds typically reach 350–450 Mbps for individual files
• Directories with many small files hit API limits — compress first: tar -zcvf mydir.tar.gz ./mydir
• Add rate-limit flags (e.g. --tpslimit) if the cloud provider throttles requests