MonARCH on Rocky Linux 2024

What is the plan?

As part of the security uplift, we are migrating MonARCH login, data transfer and compute nodes into Rocky Linux. This migration is done progressively by first deploying a pair of new front-facing nodes for you to access; and then the compute nodes are scheduled to be rebuilt to Rocky Linux.

The most noticeable aspect of this change is that the existing software modules on the previous OS (CentOS) are potentially unusable on Rocky Linux. You may need to update your job scripts so that a compatible module (if applicable) is loaded instead of the older one. Please see below for the table of which software modules are compatible on Rocky Linux, and if they are not compatible, we have installed a newer Rocky Linux version, or alternatively, have installed one on the OS itself.

Important

Existing CentOS nodes will be converted into Rocky Linux soon. Please take the necessary steps to future-proof your job scripts.

What has not changed

• Your username and password is as on the existing MonARCH

• Your current $HOME folder and existing project / scratch folders are accessible

• The SLURM job scheduler is shared on both CentOS and Rocky Linux nodes

How to access the Rocky Linux front-facing nodes?

A pair of new front-facing nodes are ready for you to use the Rocky Linux:

monarch-login4.erc.monash.edu
monarch-dtn2.erc.monash.edu

We are still applying some finishing touches on the ``dtn2`` so some functions are possibly missing.

How do I submit jobs to Rocky nodes?

The easiest way is to submit Rocky Linux jobs is to sbatch them from within the monarch-login4 login node. Jobs submitted from this login node will automatically be directed to run on the Rocky Linux nodes.

If you are submitting jobs to the existing CentOS nodes, there is no need to change your job scripts, simply submit them from the either monarch-login1 or monarch-login2. Rocky Linux capacity will increase as we convert existing nodes.

From these existing login nodes, you can also submit jobs to the Rocky Linux nodes by adding a --constraints line into your job scripts as follows:

#SBATCH --constraints=r9

Alternatively, during the sbatch command, you can include a flag -C as follows:

sbatch -C r9 jobscript.sh

What modules are available?

When running the module avail command, you will see three sections of installed software modules. The first section:

------------------------ /apps/modulefiles -------------------------
anaconda/2024.02-1           lammps/20230802
ansys/24r1                   leveldb/1.23
canu/v2.2                    libjpeg-turbo/3.0.2
ctffind/4.1.8                molden/6.9
fftw/3.3.10                  motioncor2/1.6.4

The second section shows additional modules installed using Spack.

--------------------- /apps/spack/modulefiles ----------------------
hpl/2.3-gcc-11.3.1-npardb4  hpl/2.3-npardb4

Lastly, the existing CentOS software modules are shown. These modules are considered legacy and they are possibly incompatible on Rocky Linux. The table below shows what works (Compatible). Those software that are not compatible are either built fresh specifically for the Rocky Linux or they have been installed on at the operating system.

Software Module Compatibility Table

Software	Current module on CentOS	Compatibility / New version
adf	adf/2019.104 adf/2023.103	Compatible – load /usr/local module
anaconda	anaconda/2019.03-Python3.7-gcc5	anaconda/2024.02-1
ansys	ansys/22r1	ansys/24r1
bison	bison/2.7.1	OS built-in – no module load needed
blas	blas/3.8.0-gcc5-pic	OS built-in – no module load needed
boost	boost/1.67.0-gcc5	OS built-in – no module load needed
canu	canu/1.7.1	canu/2.2
cblas	cblas/20032302-gcc5	OS built-in – no module load needed
cmake	cmake/3.26.3-linux-centos7-skylake_avx512-gcc11.2.0	OS built-in – no module load needed
comsol	comsol/6.0	Compatible – load /usr/local module
cplex	cplex/12.10.0	Compatible – load /usr/local module
cuda	cuda/8.0	Compatible – load /usr/local module
eigen	eigen/3.3.7	OS built-in – no module load needed
ffmpeg	ffmpeg/4.3.1	OS built-in – no module load needed
fftw	fftw/3.3.5-gcc	OS built-in – no module load needed
gaussian	gaussian/g16a03	Compatible – load /usr/local module
gcc	gcc/5.4.0	OS built-in – no module load needed
gdal	gdal/3.8.4	gdal/3.8.4
geos	geos/3.6	geos/3.12.1
gflags	gflags/master (https://github.com/gflags/gflags)	gflags/2.2.2
glog	glog/master (https://github.com/google/glog)	gflags/2.2.2
gmp	gmp/6.1.2	gmp/6.3.0
gromacs	gromacs/2018-openmpi-cuda8.0-NVML	gromacs/2024.1-openmpi-cuda gromacs/2024.1-openmpi-mpi
gsl	gsl/2.2-system	gsl/2.7
gurobi	gurobi/10.0.1	Compatible – load /usr/local module
hdf5	hdf5/1.10.0-patch1	OS built-in – no module load needed
intel	intel/2018u3	intel-compiler/2024.02 intel-mkl/2024.0
java	java/openjdk-1.8.0_242	OS built-in – no module load needed
julia	julia/1.9.4	``julia/1.10.2 ``
lammps	lammps/20180510	lammps/20230802
lapack	lapack/3.6.1-gcc5	OS built-in – no module load needed
leveldb	leveldb/master	leveldb/1.23
libjpeg-turbo	libjpeg-turbo/1.4.2	OS built-in – no module load needed
lmdb	lmdb/latest	OS built-in – no module load needed
make	make/4.3	OS built-in – no module load needed
mathematica	mathematica/12.1.1	Compatible – load /usr/local module
matlab	matlab/r2019b	Compatible – load /usr/local module
mesa	mesa/default	OS built-in – no module load needed
mkl	mkl/2018u3	intel-mkl/2024.0
molden	molden/5.7	molden/6.9
mpfr	mpfr/3.1.5	mpfr/4.2.1
ncurses	ncurses/6.2-linux-centos7-skylake_avx512-gcc11.2.0	OS built-in – no module load needed
netcdf	netcdf/4.4.1.1-openmpi-1.10.7-mlx	OS built-in – no module load needed
openblas	openblas/0.3.10-linux-centos7-skylake_avx512-gcc8.1.0	OS built-in – no module load needed
opencv	opencv/3.4.11	OS built-in – no module load needed
openfoam	openfoam/v2206	openfoam/2312
openmpi	openmpi/1.10.7-mlx	hpcx/latest
openssl	openssl/1.1.1t-linux-centos7-skylake_avx512-gcc11.2.0	OS built-in – no module load needed
proj	proj/8.1.1	proj/9.4.0
protobuf	protobuf/master	protobuf/26.1
python	python/3.7.3-system	OS built-in – no module load needed
qt	qt/5.7.1-gcc5	OS built-in – no module load needed
raven	raven/1.1.10	raven/1.8.3
samtools	samtools/1.9-gcc5	samtools/1.19.3
scalapack	scalapack/2.0.2	OS built-in – no module load needed
snappy	snappy/master	snappy/1.2.0
sqlite3	sqlite3/3.30.1	OS built-in – no module load needed
virtualgl	virtualgl/2.5.0	Compatible – load /usr/local module
zlib	zlib/1.2.13-linux-centos7-skylake_avx512-gcc11.2.0	OS built-in – no module load needed

How do I request software to be installed on Rocky Linux?

Please submit a request for a software installation via the google form below: https://docs.google.com/forms/d/e/1FAIpQLScSuz80Bof2QyGUNo-6aWsUgjTcQ5ZN6H91g5R1QEkyxULhZA/viewform