Copying Files to/from the Grid

Copying files between computers is made easy via terminal and GUI.  But before you start the copying process please pause a moment and consider the following:

• What type of data do you have in terms of classification level? Level 3 data must not be stored on user devices or portable media unless the device or media is encrypted. Level 4+ data must not be stored on user computing devices (i.e. laptops, smartphones, tablets), but may be stored on external encrypted portable storage media. Data designated as level 3 and higher may only be used, stored or processed on servers or services that comply with Harvard data protection requirements. For details on data classification please visit the Harvard University Information Security Policy page.

• Remember that home folders are not sharable, and typically contain files that pertain to your work only. This also includes the default location for configuration or preference files, and work that is considered 'in development'.

• Project folders are shared locations, and ideally used for collaborative project spaces. Please see our recommendation below on organizing project spaces. NOTE: Only those persons who are listed on the data usage agreement or IRB proposal should have access to these directories. This is the responsibility of the primary HBS faculty member.

• Do not dump the contents of your laptop or desktop in your home or project spaces as a backup. Please consider solutions like Dropbox, CrashPlan, or BackBlaze; or the free Code42 backup service from HBS IT if the functionality of a backup is needed (HBS members only).

• HBS IT regularly scans the research storage areas for viruses as a part of the Harvard University security policy. Normally viruses are quarantined or removed via antivirus software on HBS-issued laptops and desktops, preventing them from reaching the research storage. But personal devices may not have similar protections. If you should receive a message from HBS IT or RCS about virus-containing files in your home or project spaces, please remove these as soon as possible to ensure the safety and integrity of research data across the entire computing grid and research storage.

Below is a brief guide on how to copy files.

Copying Files Using Terminal

SSH terminal protocols such as SFTP, SCP, and rsync allow you to copy files with simple unix commands.

scp

Simply issue the scp command and make sure that the order of the files is always source before destination.

scp [username@server:] [location of file] [destination of file]
or
scp [location of file] [username@server:][destination of file]

For details please refer to Accessing Project Spaces & Folder via Terminal: scp

rsync

Rsync is known for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination.

The basic syntax is:

rsync [username@server:]SOURCE [username@server:]DESTINATION

where SOURCE and DESTINATION are filesystem paths on local servers. Include the [username@server:] specification only if the source or destination are on a remote server.

For more details please refer to Accessing Project Spaces & Folder via Terminal: rsync

Copying Files Using a GUI

Another convenient way to copy files is by directly connecting to your home folder or project space. Common ways to achieve this include mapping a network drive (PC) or mounting a shared volume (Mac), and using a file transfer program such as SecureFX, Filezilla, or Cyberduck.

Mapped Drive (PC)

Map a network drive to your home folder or project spaces in Windows Explorer to easily access and copy over files.

Mapped Volume (Mac)

Connect to your home folder or project spaces in Mac OS with Finder "Connect to Server..." to conveniently access and copy over files.

SecureFX

Windows SecureFX GUI file transfer client implements the SFTP protocol for fast and simple file transfers.

Please see this document for using the Windows SecureFX GUI file transfer client.

FileZilla

FileZilla is a cross-platform GUI file transfer client.

Please see this document for using the cross-platform FileZilla GUI file transfer client.

Cyberduck

Cyberduck is a Mac GUI file transfer client.

Please see this document for using the Mac Cyberduck GUI file transfer client.