Filesystems/Directories and Quotas

Last modified: Monday September 01, 2014 12:31 PM

Allocation Areas & Quotas

There are two main directories or filesystems of importance that exist on Eos. These are global: each and every node and process have access to them. Physically their storage space is provided by the disks of the DDN S2A9900 RAID Array. Your disk usage on the RAID Array is limited to a finite amount on each filesystem. See the Filesystem Quotas section for more information.

Directory Quota Backed Up? Environment Variable Description
/g/home/$USER 1 GB Yes $HOME Upon login, you will be situated in /g/home/$USER, where $USER is your logon name. Store smaller files such as your source code, executables, and input files. It is also the only area that will be backed up nightly.
/scratch/$USER 50 GB No $SCRATCH This is a high performance filesystem, intended to temporarily hold rather large files for your current processing needs. It is not intended as long-term storage of any files.

The $TMPDIR environment variable is also defined as /scratch/$USER/tmp.

/tmp (on local disk or RAM) 1-5 GB No N/A On each node, login or compute, the /tmp storage provided is local to the node. Different /tmp's on different nodes are completely different storage spaces. The size of the space available is very limited. Although available to you, for the most part, you should not store data there. Some application programs as well as the Operating System, however, use /tmp as a scratch area. Such application programs, typically, also provide options to redirect your data elsewhere, say, /scratch/$USER/tmp.

The /g/home and /scratch areas are configured as parallel (GPFS) file systems and are globally accessible from each node. All data transfers from any node employ the 4x QDR Infiniband fabric for the Eos cluster.

REMINDER: The /scratch area is NOT for long-term storage of valuable but largely inactive files. Valuable files which undergo infrequent use should be moved elsewhere.


Of the two filesystems mentioned above only your $HOME directory is backed up. Files can be restored upon request.

Extensions to your $HOME space will be strongly resisted beyond the standard allocation. The staff will consider increases to your $SCRATCH space only for justifiably legitimate needs.

Needed input files that are unavailable on Eos should first be copied locally, say, to your $SCRATCH directory. These files can then be used by your batch job.

Unlike hydra, we do NOT provide temporary directories for each batch job in a separate filesystem. If necessary, use the $PBS_JOBID environment variable to create a unique directory in your $SCRATCH directory for your batch job.

The saving of data files on Eos that are unrelated to processing on Eos is NOT allowed. Users should also delete all files that are no longer needed. Please cooperate with the staff in this important task.

Filesystem Quotas

Below is a sample output of the quota command, mmlsquota, on Eos. Your quota is composed of two parts: block and file limits. The block limits affect your total disk space usage while the file limits affects how many files you can store in each filesystem.

eos# mmlsquota
                 Block Limits                               | File Limits
Filesystem type     KB     quota     limit  in_doubt  grace | files  quota  limit in_doubt  grace  Remarks
hul        USR     144   1048576   1048576         0   none |     8  20000  30000        0   none
scratch    USR     256  52428800  52428800     40960   none |     2  50000  50000       20   none

The block limits (all values in kilobytes) indicate your current usage (KB column), soft limit (quota column), and hard limit (limit column) values. The file limits section (number of files) indicates your current number of files (files column), soft limit (quota column), and the hard limit (limit column).