OpenFOAM is a free, open source CFD software package; this does not require a license to use.
OpenFOAM is an open-source set of C++ based command tools which can be used to
perform Computational Fluid Dynamics simulations, it is entirely based inside
the terminal and has no direct user interface. Results from the simulations can
be viewed by running the command
paraFoam to launch an external viewing
Versions older than v1906 may behave slightly differently, such as not
source $FOAM_SRC_FILE but use of these versions is not covered by
this documentation. Usage described in this document assumes you’re using
v1906 on ARC4, and may require some adjustment if using other versions, or
It also assumes you’re using the default module environment, and haven’t altered compilers being used. If you have, please adjust accordingly.
You are best looking at the official Tutorials
Setting up the environment#
The openfoam installed on our systems is compiled with a gnu compiler rather than the intel one which is the default in your environment. Before you can load the openfoam module you will need to switch compilers.
$ module swap intel gnu/6.3.0 $ module add openfoam/9
The environment is not yet fully loaded. To check what you need to do you can run the command:
$ module help openfoam/9
Next run the following command to access the openFoam command line variables:
$ source $FOAM_SRC_FILE
Launching on the front end#
Visualisation of OpenFOAM results can be done using paraFoam at the command prompt. You need to have an openfoam case file to do this fully:
If you’re visualising large data sets this should not be done on the login nodes, since this can use a considerable amount of RAM and CPU. Instead, this should be done using an interactive job with SGE.
Running through an interactive shell#
The following will launch paraFoam interactively, displaying the full GUI:
$ qrsh -cwd -V -l h_rt=<hh:mm:ss> paraFoam
In the above command,
<hh:mm:ss> is the length of real-time the shell will
-cwd indicated the current working directory and
-V exports the
the current environment. For example, to run paraFoam for 1 hour:
$ qrsh -cwd -V -l h_rt=1:00:00 paraFoam
This will run paraFoam within the terminal from which it was launched.
To run OpenFOAM in batch-mode you first need to setup your case.
A script must then be created that will request resources from the queuing
system and launch the desired OpenFOAM executables; script
#!/bin/bash # Run in current working directory and import the current environment #$ -cwd -V # Set a 6 hour limit #$ -l h_rt=6:00:00 # Load OpenFOAM module module swap intel gnu/6.3.0 module add openfoam/9 source $FOAM_SRC_FILE # Run actual OpenFoam commands blockMesh icoFoam
This can be submitted to the queuing system using:
$ qsub runfoam.sh
If you’ve configured your OpenFOAM job to be solved in parallel, you need to
prepare and submit it differently. You will want to decompose your case first
decomposePar after setting up the case for the desired number of
processors. This can normally be done on a login node before submitting your
main job. Here is an example of a suitable submission script:
#$ -l h_rt=01:00:00 #$ -l h_vmem=4.8G #$ -pe smp 4 #$ -cwd -V module swap intel gnu/6.3.0 module add openfoam/9 source $FOAM_SRC_FILE mpirun interFoam -parallel
This will request 4 cores, and 19.2Gbytes of RAM. If you needed less memory, reduce the requested RAM per core. If you needed more memory, you’re best asking for more cores rather than more RAM per core, if possible.
Jobs requiring a whole node or more#
#$ -l h_rt=01:00:00 #$ -l np=40 #$ -cwd -V module swap intel gnu/6.3.0 module add openfoam/9 source $FOAM_SRC_FILE mpirun interFoam -parallel
This will request 40 cores and exclusive access to the nodes. On ARC4, this results in a single node being allocated, with a full node’s memory split between those 40 cores.
If more memory per core is required, then reduce the number of processes per node to split the available memory netween those active processes. For example:
#$ -l np=40,ppn=10
This will allocate 40 cores as before, but with 10 processes per node and so 4 nodes overall. The available memory per node is therefore allocated across four nodes rather than one and so four times the memory would be available per core compared to the previous example.