Globus File Transfer

Globus File Transfer

Overview

Globus is a secure and reliable service for research data transfers. It handles interruptions, retries automatically, and enables high-performance movement of large datasets. SeaWulf does not operate as a permanent endpoint, so users create a temporary one as needed.

Setup Your Endpoint

module load globus/3.15.0
globus login

You will be prompted to open a URL in your browser, select “Stony Brook University,” sign in with your NetID, then copy the authorization code back to your terminal.

globus endpoint create --default-directory /gpfs/scratch/$USER/ --personal seawulf

Record the Endpoint ID and Setup Key returned by this command.

globusconnectpersonal -setup SETUP_KEY

Replace SETUP_KEY with the key you were given.

  1. Load the Globus tools on SeaWulf (works on nodes milan1 and milan2):
  2. Log in using your Globus account:
  3. Create your personal endpoint (default directory in scratch):
  4. Complete setup using the key:

Using Your Endpoint

globusconnectpersonal -start -restrict-paths rw/gpfs/scratch/$USER,rw/gpfs/home/$USER &

The ampersand lets the endpoint run in the background.

globus ls $ep1
globus transfer $ep1:/gpfs/scratch/$USER/myfolder $ep2:~/otherfolder --recursive --label "CLI single folder"
globusconnectpersonal -status
globusconnectpersonal -stop
  1. Start the endpoint (allow access to your scratch and home directories):
  2. Check if it is working by listing files on your scratch directory (replace $ep1):
  3. Transfer files between two endpoints (replace $ep1 and $ep2):
  4. Check endpoint status:
  5. When done, stop the endpoint:

Summary Table

Action Command
Load Globus tools module load globus/3.15.0
Log in to Globus globus login
Create endpoint globus endpoint create --default-directory /gpfs/scratch/$USER/ --personal seawulf
Complete setup globusconnectpersonal -setup SETUP_KEY
Start endpoint globusconnectpersonal -start -restrict-paths rw/gpfs/scratch/$USER,rw/gpfs/home/$USER &
Transfer files globus transfer source_ep:/path target_ep:/path --recursive --label "description"
Check status globusconnectpersonal -status
Stop endpoint globusconnectpersonal -stop