Archiving Data

Last update on Thursday, 03-May-2007 13:27:36 CDT.

Summary

Archive System StorageTeK L180
Current Capacity 10 TB
Host K2 (Origin 3800)
HOME Backups All files in all HOME directories, on all SC compute servers, are backed up nightly to the L180
User File Quota 50 GB for non-home resident files
Intended Use Long-term storage
User Access To use the archive for non-home files, explicitly request access by emailing the SC Helpdesk. You do this only once. Then, from any SC compute server, issue one of the following commands:

To copy a file to your archive directory, issue 

        scp filename  k2:/archive/home/your_username

To copy a file from the archive to your current directory, issue 

        scp k2:/archive/home/your_username/filename  .

When logged on to k2, you should use the regular cp command

The StorageTeK L180 is a robotic, tape-based archival system. It is directly attached to K2. It has four tape drives that are SCSI-connected to K2's I/O subsystem. Data transfers to/from the archive are first buffered on 2 TB (raw) of disk that is part of a TP9400 raid array. The TP9400 is also directly attached to K2. All transfers between the TP9400 and the L180 are handled by SGI's DMF program. At this time (circa April 2004), the L180's raw capacity is 16 TB, but the current DMF license restricts use to 10 TB. The maximum configurable capacity of the L180 is 28 TB (uncompressed). As the demand for archiving dictates, and as funding permits, the SC will progressively increase both the archive's raw capacity and DMF's coverage.

Please note that all files in a user's HOME directory are backed up every day. So there is normally no need to explicitly save on the archive any home-resident file. Valuable files, not resident in your home directory, which you must backup on a long-term basis, can be stored on the L180. You transfer such files by using the, 

  k2:/archive/home/your_username

directory. Access to this directory is possible only from any of the SC compute hosts. You also first need to notify the SC staff of your intention to use the archive in this manner and wait to receive acknowledgement to that effect.

File Transfers

To send a file from the current working directory on an SC host to your archive directory, issue: 

 scp filename k2:/archive/home/your_username

To recover a file from the archive to your current working directory (".") on an SC host, issue: 

 scp k2:/archive/home/your_username/filename .

When logged on to k2, normal copying commands should be used.

Other File Handling

To delete files, make subdirectories in your archive area, and so on, you need to log on to K2 first and proceed from there in normal UNIX fashion.

Batch Use Not Allowed

File transfers involving the archive and batch jobs is not allowed. Direct user access of the archive is predicated on the understanding that the files involved will be infrequently used. When such files are needed in batch jobs, transfer them (interactively!) first from the archive to an area on an SC host

Beyond Your Quota

Exceeding your 50 GB quota requires prior permission from the SC staff. All high quota requests will be referred to the Steering Committee for approval.