To move files between your computer and the cluster from a terminal, use scp for quick copies, rsync for folders and big transfers, or sftp to browse. Windows users can also drag and drop with MobaXterm or WinSCP. For very large datasets, Globus is faster and more reliable.
On this page: Which tool? · scp · rsync · sftp · Windows apps · Fewer Duo prompts · Troubleshooting
<netid> with your NetID. Every transfer needs Duo, just like logging in.Which tool?
| You want to… | Use |
|---|---|
| Copy a few files once | scp |
| Copy a folder, resume after a dropped connection, or send only what changed | rsync |
| Look around and pick files interactively | sftp |
| Drag and drop on Windows | MobaXterm or WinSCP |
| Upload or download in the browser | Open OnDemand file manager |
| Move hundreds of GB or more | Globus |
scp: quick copies
# Upload two files to your home folder scp file1.txt file2.txt <netid>@milan.seawulf.stonybrook.edu:~/ # Download a file to the current folder scp <netid>@milan.seawulf.stonybrook.edu:~/output.txt ./ # Upload a whole folder scp -r my_folder/ <netid>@milan.seawulf.stonybrook.edu:~/ # Several remote files at once (quotes make the braces expand on the cluster) scp <netid>@milan.seawulf.stonybrook.edu:'~/results/{a,b,c}.txt' ./
Group files into one command where you can: each scp is a new login and a new Duo prompt.
rsync: folders and big transfers
# Upload a folder (only copies what's new or changed) rsync -avz --progress my_folder/ <netid>@milan.seawulf.stonybrook.edu:~/my_folder/ # Download from a project space rsync -avz --progress <netid>@milan.seawulf.stonybrook.edu:/gpfs/projects/SmithGroup/data/ ./data/ # See what would be copied without copying anything rsync -avz --dry-run my_folder/ <netid>@milan.seawulf.stonybrook.edu:~/my_folder/
Flags: -a keeps permissions and dates, -v lists files, -z compresses, --progress shows progress. If a transfer stops, run the same command again and rsync picks up where it left off.
Watch the trailing slash: my_folder/ copies the folder's contents; my_folder copies the folder itself.
sftp: browse and pick
sftp <netid>@milan.seawulf.stonybrook.edu sftp> ls # list files on the cluster sftp> cd /gpfs/scratch/<netid> sftp> get results.csv # download sftp> put input.dat # upload sftp> bye
Windows apps
- MobaXterm (recommended): start a new SFTP session (Session, then SFTP) to
milan.seawulf.stonybrook.eduwith your NetID, then drag files between the panels. Avoid the file browser in the sidebar of a normal SSH session, which triggers repeated Duo prompts. - WinSCP: choose the SFTP protocol, host
milan.seawulf.stonybrook.edu, your NetID, and drag and drop. - Open OnDemand: upload and download to home, scratch, or project space in your browser.
Fewer Duo prompts
- Connect to the Stony Brook VPN first. You approve Duo once, and transfers through the VPN don't ask again.
- Set
DUO_PASSCODE=pushso tools that can't show the Duo menu still work. See Duo authentication. - Reuse one connection on macOS and Linux. See fewer Duo prompts.
Troubleshooting
The transfer hangs or never asks for Duo
File-transfer tools often can't show the Duo menu. Set DUO_PASSCODE, connect through the VPN, and check your default device in the Duo device settings.
"Permission denied" when copying into a project space
You may not be a member of that project's group yet. Ask your PI to request access. Also check the path is spelled exactly right.
"Disk quota exceeded"
Your home folder is only 20 GB. Copy large data to scratch or project space instead, and check with myquota.
Transfers are slow or keep dropping
Use rsync so you can resume, bundle many small files into one archive with tar first, or switch to Globus for large amounts of data.