Example Job Scripts

• Example of Serial Job
• Example of OpenMP Job
• Example of MPI Job
• Example of MIMD Job

Below are various example job scripts. For more information about batch processing, see the Introduction to Batch Processing section of this user guide.

Example of Serial Job

This example runs a serial program with 12 GB of memory for up to two hours.

 #PBS -l nodes=1:ppn=1,walltime=02:00:00,mem=12gb
 #PBS -N thisjobname
 #PBS -S /bin/bash
 #PBS -j oe

 # Initialize the Intel compilers to use for this job
 module load intel/compilers

 ## Some job PRE-processing may go here
 mkdir $SCRATCH/some/dir
 cd $SCRATCH/some/dir

 # $PBS_O_WORKDIR is the directory from which the job was submitted
 cp $PBS_O_WORKDIR/files.* .

 ## Issue the command
 ./myprog.exe arg1 arg2

 ## Some job POST-processing may go here
 cp ./files.*  $PBS_O_WORKDIR

Example of OpenMP Job

This example runs a OpenMP program with 6 OpenMP threads and 3 GB of memory per thread for up to four hours. The value for the PBS ppn parameter should match the value for the OMP_NUM_THREADS environment variable.

Do not use this example job script unless your code has been modified to use OpenMP threads.

 #PBS -l nodes=1:ppn=6,walltime=04:00:00,mem=18gb
 #PBS -N thisjobname
 #PBS -S /bin/bash
 #PBS -j oe

 # Initialize the Intel compilers and Open MPI to use for this job
 module load intel/compilers

 ## Some job PRE-processing may go here
 mkdir $SCRATCH/some/dir
 cd $SCRATCH/some/dir

 # $PBS_O_WORKDIR is the directory from which the job was submitted
 cp $PBS_O_WORKDIR/files.* .

 ## set number of OpenMP threads to match requested processors
 export OMP_NUM_THREADS=6

 ## Issue the MPI command
 ./myomp arg1 arg2

 ## Some job POST-processing may go here
 cp ./files.*  $PBS_O_WORKDIR

Example of MPI Job

This example runs an Open MPI program with 5 nodes, 8 tasks per node, and 1 GB of memory per task (40 GB total). Note, that Open MPI picks up the node information from the $PBS_NODEFILE file implicitly.

Do not use this example job script unless your code has been modified to use MPI.

 #PBS -l nodes=5:ppn=8,walltime=04:00:00,mem=40gb
 #PBS -N thisjobname
 #PBS -S /bin/bash
 #PBS -j oe
 #
 ## Some job PRE-processing may go here
 mkdir $SCRATCH/thisjobname
 cd $SCRATCH/thisjobname
 #
 # Initialize the Intel compilers and Open MPI to use for this job
 module load intel/compilers
 module load openmpi

 # $PBS_O_WORKDIR is the directory from which the job was submitted
 # and mympi, the executable, is stored in
 cp $PBS_O_WORKDIR/mympi .

 ## Issue the MPI command

 mpirun ./mympi arg1 arg2

 ##  NOTE  the $PBS_NODEFILE is picked up implicitly 
 echo "*** ACTIVATED NODES: $PBS_NODEFILE ***\n"

 ## Some job POST-processing may go here
 cp ./datafiles.*  $PBS_O_WORKDIR

Example of MIMD Job

This example demonstrates one way of running in batch mode programs in MIMD (Multiple-Instruction-Multiple-Data) fashion. The programs involved can be serial and/or parallel (MPI & OpenMP). Redirection of standard input--especially--and standard output many times does not work and alternative means need to be found.

 #PBS -l nodes=2:ppn=8,mem=2gb,walltime=00:40:00
 #PBS -N mimd_sample
 #PBS -j oe
 #PBS -V
 # Shows how run MIMD jobs. May include multiple serial runs also.
 # Assumes all needed files are in PBS_O_WORKDIR
 #
 cd $PBS_O_WORKDIR
 #
 module load intel/compilers
 module load openmpi
 #
 mpirun -np 1 serial1.exe : -np 1 serial2.exe : -np 1 serial1.exe : \
         -np 1 serial1.exe : -np 1 serial2.exe : -np 1 serial1.exe : \
         -np 1 serial1.exe : -np 1 serial2.exe : -np 1 serial1.exe : \
         -np 1 serial1.exe : -np 1 serial2.exe : -np 1 serial1.exe : \
         -np 1 serial1.exe : -np 1 serial2.exe : -np 1 serial1.exe : \
         -np 1 serial2.exe
 #
 mpirun -np 4 ./hello_world_ompi_c.exe : \
         -np 4 ./hello_world_ompi_c.exe : \
         -np 4 ./hello_world_ompi_c.exe : \
         -np 4 ./hello_world_ompi_c.exe