A guide to SeaWulf's AMD nodes

SeaWulf recently introduced new AMD EPYC 96 core nodes. This article will explain how to access these nodes and provide some recommendations for compiler and math library recommendations.

As with the other types of nodes available on SeaWulf, there are multiple queues (Slurm partitions) available for the 96 core AMD nodes.  Depending on the desired number of nodes and required runtime, users can choose from the following queues to access the 96 core nodes:

• short-96core
• long-96core
• extended-96core
• medium-96core

Here is an example job script for running a simple "Hello World" job on one of the AMD nodes:

#!/usr/bin/env bash

#SBATCH --job-name=AMD_job
#SBATCH --output=AMD_job.log
#SBATCH --nodes=1
#SBATCH --time=05:00
#SBATCH -p short-96core

# load AMD AOCC compiler
module load aocc/4.0.0

# compile the code
clang /gpfs/projects/samples/helloworld/hello.c -o amd_hello

# execute the code
./amd_hello

In this example, we loaded the aocc/4.0.0 module to get access to the latest version of the AMD Optimizing C/C++ and Fortran Compilers, which are well suited for the AMD CPU architecture.  This compiler suite is based on LLVM, so the compiler commands to use are:
 

In addition, we offer the aocl/4.0.0 module to provide access to AMD's Optimizing CPU Libraries, a suite of numerical libraries that are optimized for AMD's architecture.  The math libraries present in this software suite include implementations of FFTW, SCALAPACK, BLIS (BLAS-like linear algrebra routines), among many others.

While the AOCC and AOCL compilers and libraries are expected to work well on the AMD nodes, SeaWulf users are not limited to only use AMD compilers. GCC and Intel compilers are available on the AMD nodes, just like they are on all other nodes.

Here are a few flags for different compilers that may help with performance on the AMD nodes:
 

Please see the AMD documentation for a more detailed list of recommended compiler flags.