Some software may not be available as a module or through package managers like Conda. In these cases, you can compile and install software from source. This approach allows you to customize the installation, optimize for specific hardware, control versions precisely, and access features not yet available in pre-built packages.

When to Install from Source

Install from source when:

• The software is not available as a module on SeaWulf
• You need a newer version than what's provided
• You need specific compile-time options or optimizations
• You're developing or modifying the software yourself
• The software requires architecture-specific optimizations

Before installing from source, check if:

• A module already exists: module avail software-name
• It's available through Conda: conda search software-name
• Multiple users will need it (consider requesting a system-wide install)

Choosing an Installation Location

Home directory (/gpfs/home/username):

• Pros: Backed up, persistent, private to you
• Cons: 20 GB quota limit
• Best for: Personal tools, small programs, utilities
• Example: --prefix=$HOME/software/myapp-1.0

Project space (/gpfs/projects/GroupName):

• Pros: Shared with research group, large capacity, persistent
• Cons: Not backed up, accessible to all group members
• Best for: Collaborative projects, large software, shared tools
• Example: --prefix=/gpfs/projects/GroupName/software/myapp-1.0

Understanding SeaWulf Node Architectures

SeaWulf has four different CPU architectures. Compiling with the correct optimization flags can significantly improve performance. For the full list of queues and their architecture extensions, see our SeaWulf Queues Table.

Tip: If you plan to run on multiple architectures, compile separate versions or use generic flags like -march=x86-64 for broader compatibility at the cost of some performance.

Loading Compilers and Dependencies

Before compiling, load required compiler and library modules:

# Check available compilers
module avail gcc
module avail intel

# Load a compiler
module load gcc/11.2.0
# or
module load intel/oneAPI/2023.1 compiler

# Load common dependencies as needed
module load cmake/3.24.2
module load openmpi/4.1.4

Many software packages require specific compilers or libraries. Check the software's documentation for dependencies.

Common Compiler Environment Variables

# Use these to specify compilers explicitly
export CC=gcc
export CXX=g++
export FC=gfortran

# For Intel oneAPI
export CC=icx
export CXX=icpx
export FC=ifx

# For Older Intel Compilers
export CC=icc
export CXX=icpc
export FC=ifort

Basic Installation Process

1. Download and Extract Source Code

# Download (example with wget or curl)
wget https://example.com/software-1.0.tar.gz
# or
curl -O https://example.com/software-1.0.tar.gz

# Extract
tar -xzf software-1.0.tar.gz
cd software-1.0

2. Configure the Build

Most software uses one of these build systems:

Autotools (./configure):

./configure --prefix=$HOME/software/myapp-1.0 \
            CFLAGS="-O3 -march=skylake-avx512" \
            CXXFLAGS="-O3 -march=skylake-avx512"

CMake:

mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/software/myapp-1.0 \
         -DCMAKE_BUILD_TYPE=Release \
         -DCMAKE_C_FLAGS="-O3 -march=skylake-avx512" \
         -DCMAKE_CXX_FLAGS="-O3 -march=skylake-avx512"

Make only (with Makefile):

Edit the Makefile directly to set PREFIX and compiler flags, or use:

make PREFIX=$HOME/software/myapp-1.0 CFLAGS="-O3 -march=skylake-avx512"

3. Compile

# Use parallel compilation to speed up the build
make -j $(nproc)

4. Test (Optional but Recommended)

# Many packages include test suites
make check
# or
make test

5. Install

make install

This copies executables, libraries, and documentation to your specified prefix directory.

Complete Example: Installing a Tool

Here's a complete example of installing a hypothetical analysis tool. This would be run on an interactive job requesting 4 cores on the short-40core-shared partition:

#!/bin/bash

# Load required modules
module load gcc/11.2.0
module load cmake/3.24.2

# Set installation location
INSTALL_DIR=$HOME/software/analysis-tool-2.1

# Download and extract
wget https://github.com/example/analysis-tool/archive/v2.1.tar.gz
tar -xzf v2.1.tar.gz
cd analysis-tool-2.1

# Configure for Skylake nodes
mkdir build && cd build
cmake .. \
  -DCMAKE_INSTALL_PREFIX=$INSTALL_DIR \
  -DCMAKE_BUILD_TYPE=Release \
  -DCMAKE_C_FLAGS="-O3 -march=skylake-avx512" \
  -DCMAKE_CXX_FLAGS="-O3 -march=skylake-avx512" \
  -DENABLE_OPENMP=ON

# Compile (using 4 cores)
make -j 4

# Test the build
make test

# Install
make install

# Optional cleanup
cd ~
rm -rf ~/analysis-tool-2.1 ~/v2.1.tar.gz

echo "Installation complete!"
echo "Add to your PATH: export PATH=$INSTALL_DIR/bin:\$PATH"

After Installation: Using Your Software

Setting Up Your Environment

After installation, you need to tell your system where to find the software:

# Add to PATH (for executables)
export PATH=$HOME/software/myapp-1.0/bin:$PATH

# Add to LD_LIBRARY_PATH (for shared libraries)
export LD_LIBRARY_PATH=$HOME/software/myapp-1.0/lib:$LD_LIBRARY_PATH

# Add to MANPATH (for documentation)
export MANPATH=$HOME/software/myapp-1.0/share/man:$MANPATH

Making Changes Persistent

Add these exports to your ~/.bashrc for interactive sessions:

# Edit ~/.bashrc
nano ~/.bashrc

# Add these lines at the end:
export PATH=$HOME/software/myapp-1.0/bin:$PATH
export LD_LIBRARY_PATH=$HOME/software/myapp-1.0/lib:$LD_LIBRARY_PATH

Using in Job Scripts

Always include environment setup in your SBATCH scripts:

#!/bin/bash
#SBATCH --job-name=myanalysis
#SBATCH -p short-40core
#SBATCH --nodes=1
#SBATCH --ntasks=40
#SBATCH --time=01:00:00

# Load required modules (same as during compilation)
module load gcc/11.2.0

# Set up paths to your software
export PATH=$HOME/software/myapp-1.0/bin:$PATH
export LD_LIBRARY_PATH=$HOME/software/myapp-1.0/lib:$LD_LIBRARY_PATH

# Run your analysis
myapp input.dat output.dat

Creating Personal Modulefiles (Advanced)

For frequently used software, you can create your own modulefile:

# Create directory for personal modules
mkdir -p $HOME/privatemodules/myapp

# Create modulefile
nano $HOME/privatemodules/myapp/1.0

Example modulefile content:

#%Module1.0 -*- tcl -*-
##
## modulefile
##
proc ModulesHelp { } {
    puts stderr "My Analysis App version 1.0"
}

module-whatis "My Analysis App 1.0"

source /gpfs/shared/modulefiles/internal/common-vars

set               root              $env(HOME)/software/myapp-1.0

prepend-path      PATH              $root/bin
prepend-include-path                $root/include
prepend-lib-path                    $root/lib
prepend-path      MANPATH           $root/share/man

Enable your personal modules:

# Add to ~/.bashrc
module use $HOME/privatemodules

# Now you can load your module
module load myapp/1.0

Troubleshooting Common Issues

Configure fails with "compiler not found":
Load a compiler module first: module load gcc/11.2.0

Missing dependencies during configure:
Check if the dependency is available as a module: module avail dependency-name. Load it before configuring.

Compilation fails with "command not found":
You may need to install build tools. Try module load cmake or check documentation for required tools.

"Permission denied" during installation:
Check that you have write access to your installation prefix. Make sure you're not trying to install to system directories like /usr/local.

Program runs but crashes immediately:
Likely a library path issue. Check with ldd $HOME/software/myapp-1.0/bin/myapp to see if all libraries are found. Add missing library directories to LD_LIBRARY_PATH.

"Symbol not found" or "undefined reference" errors:
This usually means linking issues. Make sure you load the same modules when running the software as you did when compiling it.

Performance is poor:
You may not have used optimization flags. Recompile with -O3 and appropriate -march flags for your target architecture.

Software compiled on one node type won't run on another:
You used architecture-specific optimizations. Either compile with generic flags (-march=x86-64) or create separate builds for each architecture.

Best Practices

• Always read the software's README and INSTALL documentation first
• Load the same modules at runtime that you used during compilation
• Keep a log of your compilation commands and configuration options
• Test software interactively before using in batch jobs
• Document your installation in a text file within the installation directory
• For large compilations, use an interactive job on a compute node
• Match optimization flags to the node architecture you'll use most
• Back up important custom software installations periodically
• If compiling GPU-enabled software, compile on a GPU node

When to Request System-Wide Installation

Consider requesting a module installation through the support system if:

• Multiple users or research groups will need the software
• The software is widely used in your field
• Installation is complex or requires system-level dependencies
• You need the software compiled optimally for multiple architectures
• The software requires licensed commercial compilers

Submit requests at iacs.supportsystem.com.