Last modified: Tuesday February 12, 2013 2:13 PM
- Initial License Setup for CFX 12
- Graphical Interface Support
- License Limitations
- Parallel Job Scripts
- Advanced CFX Problems
Initial License Setup for CFX 12
Note, this procedure only applies to versions 12.x (12.0 or 12.1) of CFX. This procedure is not necessary for CFX 13.0.
For ANSYS 12.x software (ANSYS, CFX, ICEMCFD, Fluent), you MUST set your preferences to use the academic licenses. This process needs to be done for each version on each system and applies ONLY to all ANSYS 12.x software.
NOTE: The following procedure is also slightly different for CFX than the other ANSYS software. The Academic Teaching Advanced license must be disabled as a workaround for an incompatibility between the academic and research licenses.
- Make sure you are running a Xwindows server on your local computer and can display graphical programs remotely. For more information, see the Accessing TAMU Supercomputing Machines page.
- SSH to the system where you want to run ANSYS products.
- Load the cfx module with the 'module load cfx' command.
- Run the anslic_admin command to start the ANSLIC_ADMIN utility. A window with the title ANSLIC_ADMIN Utility should appear if your local computer is configured as in step 1.
- Select Set License Preferences for User XXXX button. A popup window will appear.
- Select Use Academic Licenses button in the Global Settings section. THIS STEP IS ONLY NEEDED FOR CFX VERSIONS 12.x!
- Select the ANSYS Academic Teaching Advanced license and set it to 0 for Don't Use. This will ensure that CFX uses the research license for parallel runs.
- Select the OK button.
- Select File from the pull down menu of the ANSLIC_ADMIN Utility and then select Exit to close the ANSLIC_ADMIN utility.
Before running CFX in batch jobs, please review Chapter 11: Starting the CFX-Solver from the Command Line in the CFX-Solver Manager User's Guide.
For CFX and Fluent, a base license token will allow CFX or Fluent to use up to 4 cpus without any additional tokens. However, if you want to use more than 4 cpus, you will need an additional "HPC" token per cpu. So a parallel CFX run with 8 cpus will need 1 base token and 4 HPC tokens. These HPC tokens are shared between CFX and Fluent. Due to licensing costs and limited usage, our current license is limited to 10 HPC tokens.
Graphical Interface Support
CFX is configured to use HP MPI and SSH for remote task launching. For Eos, please use the init-ssh command to setup your passwordless public/private SSH keys:
Due to license limitations and performance considerations, multi-node jobs are not recommended at this time on Hydra. Also, the CFX documentation recommends a minimum number of nodes per partition for good performance when running in parallel:
- Tetrahedral meshes: 30000 nodes per partition
- Hexahedral meshes: 75000 nodes per partition
Please also review Chapter 14: Using the Solver in Parallel in the CFX-Solver Modeling Guide before running parallel CFX jobs.
For Eos, there are different start methods, "HP MPI Local Parallel" and "HP MPI Distributed Parallel", for using single or multiple nodes respectively with CFX.
Single Node Parallel Job Script for Eos
HP MPI is recommended for running parallel jobs with CFX on Eos. The example below is for running CFX in parallel on a single node with HP MPI.
#PBS -l walltime=24:00:00,nodes=1:ppn=4,mem=8gb #PBS -N cfx #PBS -j oe #PBS -S /bin/bash module load cfx cd $PBS_O_WORKDIR mkdir $PBS_JOBID cd $PBS_JOBID cp $PBS_O_WORKDIR/solverinput.def . cfx5solve -batch -def solverinput.def -start-method "HP MPI Local Parallel" -part 4
Multi-Node Parallel Job Script for Eos
HP MPI is recommended for running parallel jobs with CFX on Eos. The example below is for running CFX in parallel on two nodes with HP MPI. Please note the different start method when using multiple nodes with CFX.
#PBS -l walltime=24:00:00,nodes=2:ppn=8,mem=40gb #PBS -N cfx #PBS -j oe #PBS -S /bin/bash module load cfx cd $PBS_O_WORKDIR mkdir $PBS_JOBID cd $PBS_JOBID cp $PBS_O_WORKDIR/solverinput.def . # convert PBS_NODEFILE information into format for CFX host list nodes=`/g/software/bin/cfx_machines.pl` # run CFX in parallel cfx5solve -batch -def solverinput.def -par-host-list $nodes -part 16 \ -start-method "HP MPI Distributed Parallel"
Single-Node Parallel Job Script for Hydra
CFX does NOT support the native MPI library, POE, on AIX. Thus, the start method for CFX should use MPICH. The example below is for running CFX in parallel on a single node with MPICH.
#@ shell = /bin/ksh #@ comment = CFX single node job #@ initialdir = /scratch/user/cfx #@ job_name = cfx #@ error = $(job_name).o$(schedd_host).$(jobid).$(stepid) #@ output = $(job_name).o$(schedd_host).$(jobid).$(stepid) #@ environment = MP_SINGLE_THREAD=yes #@ resources = ConsumableCpus(1) ConsumableMemory(1000mb) #@ wall_clock_limit = 24:00:00 #@ job_type = parallel #@ network.MPI_LAPI = sn_single, shared, US #@ node = 1 #@ tasks_per_node = 4 #@ notification = error #@ queue module load cfx mkdir $LOADL_STEP_ID cd $LOADL_STEP_ID cp $LOADL_STEP_INITDIR/solverinput.def . cfx5solve -batch -def solverinput.def -part 4 -start-method "MPICH Local Parallel"
Advanced CFX Problems
The SC staff has limited experience with CFX and ICEM. Due to certain restrictions, students may not be able to use the official support methods. For help about using CFX or ICEM, please visit CFX User Forum at CFD Online.