Filesystems/Directories and Quotas
Last modified: Monday September 01, 2014 12:31 PM
Allocation Areas & Quotas
There are two main directories or filesystems of importance that exist on Eos. These are global: each and every node and process have access to them. Physically their storage space is provided by the disks of the DDN S2A9900 RAID Array. Your disk usage on the RAID Array is limited to a finite amount on each filesystem. See the Filesystem Quotas section for more information.
|Directory||Quota||Backed Up?||Environment Variable||Description|
|/g/home/$USER||1 GB||Yes||$HOME||Upon login, you will be situated in /g/home/$USER, where $USER is your logon name. Store smaller files such as your source code, executables, and input files. It is also the only area that will be backed up nightly.|
|/scratch/$USER||50 GB||No||$SCRATCH||This is a high performance filesystem, intended to temporarily
hold rather large files for your current processing needs. It is not
intended as long-term storage of any files.
The $TMPDIR environment variable is also defined as /scratch/$USER/tmp.
|/tmp (on local disk or RAM)||1-5 GB||No||N/A||On each node, login or compute, the /tmp storage provided is local to the node. Different /tmp's on different nodes are completely different storage spaces. The size of the space available is very limited. Although available to you, for the most part, you should not store data there. Some application programs as well as the Operating System, however, use /tmp as a scratch area. Such application programs, typically, also provide options to redirect your data elsewhere, say, /scratch/$USER/tmp.|
The /g/home and /scratch areas are configured as parallel (GPFS) file systems and are globally accessible from each node. All data transfers from any node employ the 4x QDR Infiniband fabric for the Eos cluster.
REMINDER: The /scratch area is NOT for long-term storage of valuable but largely inactive files. Valuable files which undergo infrequent use should be moved elsewhere.
Of the two filesystems mentioned above only your $HOME directory is backed up. Files can be restored upon request.
Extensions to your $HOME space will be strongly resisted beyond the standard allocation. The staff will consider increases to your $SCRATCH space only for justifiably legitimate needs.
Needed input files that are unavailable on Eos should first be copied locally, say, to your $SCRATCH directory. These files can then be used by your batch job.
Unlike hydra, we do NOT provide temporary directories for each batch job in a separate filesystem. If necessary, use the $PBS_JOBID environment variable to create a unique directory in your $SCRATCH directory for your batch job.
The saving of data files on Eos that are unrelated to processing on Eos is NOT allowed. Users should also delete all files that are no longer needed. Please cooperate with the staff in this important task.
Below is a sample output of the quota command, mmlsquota, on Eos. Your quota is composed of two parts: block and file limits. The block limits affect your total disk space usage while the file limits affects how many files you can store in each filesystem.
eos# mmlsquota Block Limits | File Limits Filesystem type KB quota limit in_doubt grace | files quota limit in_doubt grace Remarks hul USR 144 1048576 1048576 0 none | 8 20000 30000 0 none scratch USR 256 52428800 52428800 40960 none | 2 50000 50000 20 none
The block limits (all values in kilobytes) indicate your current usage (KB column), soft limit (quota column), and hard limit (limit column) values. The file limits section (number of files) indicates your current number of files (files column), soft limit (quota column), and the hard limit (limit column).