Filesystems and Quotas
- Allocation Areas & Quotas
- Policies
- Common File Commands
- Protection & File Security
- Additional Information
Allocation Areas & Quotas
There are three main directories or filesystems of importance that exist on the Hydra. You are allocated a finite amount of diskspace on each.
| Directory | Quota | Backed Up? | Description |
|---|---|---|---|
| /home/userid | 300MB | Yes | Upon login, you will be situated in /home/userid, where userid is your logon name. This area is automatically replicated for safety. So in effect the maximum disk space you can occupy here is 600MB. The disk system supplying this space is directly attached to four (external) Power 520 systems and made available to all the cluster nodes via ethernet. For that reason it is the lowest performing of the three disk areas that users have access. Please be mindful of this fact when you compile and run codes in your home directory. The response will not be nearly as good, especially when large files are involved, compared to working in the /scratch or /work areas. It is also the only area that will be backed up nightly. This directory can be referenced with the $HOME environment variable. |
| /scratch/userid | 10GB | No | This is a high I/O performance area, intended to temporarily hold rather large files that you will need for current processing needs. The disk supplying this storage is hosted on a fiber-channel DataDirect Networks (DDN) raid array. The DDN array is directly attached to four p575 nodes, the so called I/O nodes of the cluster by fiber channel links. Because this is a high performance area, its intended role is to hold files that undergo frequent processing. It is not intended as a long-term store for needed but largely inactive files. Both /scratch/username and /scratch/username/tmp are automatically available upon login. Upon login to the cluster, the environment variable TMPDIR has the value of /scratch/username/tmp. This is only true for interactive processing. We will adjust your quota for justifiable processing needs. You are automatically granted an allocation of diskspace in the scratch directory. Please notify the helpdesk when one is not already in place. |
| /work/batch_jobid | n/a | No | No user quota is in effect here. This directory is available only to executing batch jobs and only for the duration of the job. A LoadLeveler job gets access to this area by using the command, cd $TMPDIR. The value of the environment variable TMPDIR is /work/batch_jobid during job execution. Like /scratch above, /work also resides on the DDN. It has the highest I/O performance on the cluster. This is especially the case with sequential I/O involving large transfer sizes. |
All three areas, /home, /scratch, and /work are configured as parallel (GPFS) file systems and are globally accessible from each node. All data transfers from any node that are associated with /scratch or /work employ the HPS switch and the DDN raid array. Data movement that involves only /home uses Gigabit Ethernet and Ultra-320 SCSI drives that are attached to four Power 520 IBM workstations.
REMINDER: The /scratch area is NOT for long-term storage of valuable but largely inactive files. Valuable files which undergo infrequent use should be moved else- where.
Policies
Of the three filesystems mentioned above only your /home directory is backed up. Files can be restored upon request.
Extensions to your /home space will be strongly resisted beyond the standard allocation. The staff will consider increases to your /scratch space only for justifiably legitimate needs.
Needed input files that are unavailable on hydra should first be copied locally, say, to /scratch. These files can then be copied to a job's $TMPDIR using the standard 'cp' command. Large output files resulting from batch job execution should be handled similarly.
The saving of data files on hydra that are unrelated to processing on it is NOT allowed. Users should also delete all files that are no longer needed. Please cooperate with the staff in this important task.
Common File Commands
Display disk space allotments: myquota
hydra# myquota aggie
Filesystem blocks quota limit grace files quota limit grace
/var 320 10240 10240 3 1000 1000
/tmp 0 10240 10240 0 1000 1000
Block Limits | File Limits
Filesystem type KB quota limit in_doubt grace | files quota
limit in_doubt grace Remarks
gpfs_home USR 240 614400 614400 0 none | 15 20000
30000 0 none gpfs520.gpfs1-priv
gpfs_scratch USR 32 10485760 10485760 0 none | 1 30000
30000 0 none
The four leftmost columns indicate the filesystem, blocks used, quota, and upper limit. The system will prevent you from writing files beyond your upper limit. Numerical amounts of disk space are given in kilobytes.
Display files and directories: ls
% ls -l total 464 drwx------ 2 aggie user 18 May 26 11:03 dira drwx------ 2 aggie user 30 May 26 11:03 dirb drwx------ 2 aggie user 18 May 26 11:20 dirc drwx------ 2 aggie user 18 May 26 11:26 dird -rw------- 1 aggie user 43008 May 26 10:56 filea -rw------- 1 aggie user 57344 May 26 10:56 fileb -rw------- 1 aggie user 353280 May 26 10:56 filec
Sizes are given in bytes. The -a option lists files preceded by a dot. Consult the ls man page for more information about the ls command..
Display (recursively) disk usage: du
% du -H 32k ./.ssh 48k ./dira 416k ./dirb 23M ./dirc 45M ./dird 69M . 69M total
This indicates the amount of disk space used for each directory including the current working directory. Consult the du man page for more information about the du command.
Locate files in a directory tree: find
Files can be searched according to a variety of conditions. Options that are most commonly used are -name and -size. The full list is given in the find man page.
% find . -size +5000k ./dira/core ./dirc/filed ./dird/filee ./core
In this example, we are searching for files that are greater than 5 megabytes. Consult the find man page for more information about the find command.
View Protection & File Security: ls -al
File security in unix involves three different catagories of users. Files and directories may be owned by user, group and other. Each of class of user can possess read, write, and execute permissions. Permissions for a particular file are listed with the 'ls -al' command.
% ls -al drwx------ 7 aggie user 16384 May 26 13:34 . drwxr-xr-x 29 root root 16384 May 21 12:22 .. drwx------ 2 aggie user 29 May 26 12:21 dira -rw-rw---- 1 aggie user 43008 May 26 10:56 filea -rw----rw- 1 aggie user 57344 May 26 10:56 fileb -rw------- 1 aggie user 353280 May 26 10:56 filec -rwx------ 1 aggie user 23552 May 26 13:34 script -rw------- 1 aggie user 3072 May 26 14:24 scriptb
The left column indicates what class of user and what kind of permission belongs to a file. The first field indicates whether or not it is a directory. The next three indicate permissions for the user class. Three more indicate permissions for the group class and the last three highlight permissions for all others.
As an illustrated example, we see that 'dira' is a directory that has read, write, and execute permissions available to user 'aggie'. Execute permissions for directories permit traversal for that class of user.
'filea' has read and write permissions for the user 'aggie' and the group 'user'.
'fileb' has read and write permissions for the user 'aggie' and all other users not in the group user. This is undesirable if 'fileb' is intended to be a private file.
Notice that the current directory, as indicated by the single dot, has full permissions for aggie and no other. If this is aggie's home directory then no other user would be allowed to view the directory contents. Under normal circumstances, modifications to the default home directory permissions are not needed.
Change permissions to files and directories: chmod
The chmod command adds or removes specified permissions for any of the three classes of users.
% ls -l fileb -rw----rw- 1 aggie user 57344 May 26 10:56 fileb % chmod o-rw fileb % ls -l fileb -rw------- 1 aggie user 57344 May 26 10:56 fileb
Here we have revoked read and write permissions for all other users not in the group 'user'.
% ls -l scriptb -rw------- 1 aggie user 3072 May 26 14:24 scriptb % chmod u+x scriptb % ls -l scriptb -rwx------ 1 aggie user 3072 May 26 14:24 scriptb
'aggie' has now applied user execute permissions to the file 'scriptb'. Consult the chmod man page for more information about the chmod command.
Additional Information
All of the commands outlined above are described in detail in on-line manual pages. To call up the manual page for the 'find' command simply type at the command prompt:
% man find
An introduction to UNIX commands can be found here.