Using Freesurfer

FreeSurfer is a software package for the analysis and visualization of structural and functional neuroimaging data from cross-sectional or longitudinal studies. It is developed by the Laboratory for Computational Neuroimaging at the Athinoula A. Martinos Center for Biomedical Imaging. FreeSurfer is the structural MRI analysis software of choice for the Human Connectome Project. See https://surfer.nmr.mgh.harvard.edu/fswiki for more info.

 

This KB Article References: High Performance Computing
This Information is Intended for: Guests, Instructors, Researchers, Staff
Created: 07/31/2017 Last Updated: 05/15/2024

Current Version

While only parts of Freesurfer 6.0.0 are parallelized, we have found optimal performance within a single node using 28 cores. A basic overview of a Freesurfer workload can be found here: https://surfer.nmr.mgh.harvard.edu/fswiki/DownloadAndInstall

These steps have been modified for use on SeaWulf with a sample batch submission script available in /gpfs/projects/samples/freesurfer.

FreeSurfer Examples

First make a new directory and copy the example Slurm script and data file from the samples folder:

module load freesurfer/6.0.0

cd /gpfs/projects/samples/freesurfer_samples
mkdir -p $HOME/freesurfer_example
cp * $HOME/freesurfer_example && cd $_

Then load the Slurm module and submit the job script to the scheduler:

module load slurm
sbatch freesurfer.slurm

The submission script, freesurfer.slurm, converts the sample file sample-001.mgz to nifti format and performs a full recon-all on the file.

#!/bin/bash

#SBATCH --nodes=1
#SBATCH --ntasks-per-node=28
#SBATCH --time=48:00:00
#SBATCH --job-name freesurfer_example2
#SBATCH -p long-28core

module load freesurfer/6.0.0

cd $HOME/freesurfer_example

export SUBJECTS_DIR=$PWD

mri_convert sample-001.mgz sample-001.nii.gz

recon-all -openmp 28 -i sample-001.nii.gz -s bert -all > out.txt

After completion a directory called bert/mri and bert/surf should have been created containing the results. Along with these directories a file, out.txt, should contain a record of the job.

Visualizing the Results

After completion the results can be visualized through an interactive job and the freeview  command:

module load slurm

salloc -N 1 --ntasks-per-node=28 --time=08:00:00 -p long-28core

ssh -X $SLURM_NODELIST

module load freesurfer/6.0.0

cd $HOME/freesurfer_example

freeview -v \
    bert/mri/T1.mgz \
    bert/mri/wm.mgz \
    bert/mri/brainmask.mgz \
    bert/mri/aseg.mgz:colormap=lut:opacity=0.2 \
    -f \
    bert/surf/lh.white:edgecolor=blue \
    bert/surf/lh.pial:edgecolor=red \
    bert/surf/rh.white:edgecolor=blue \
    bert/surf/rh.pial:edgecolor=red

 

 

Article Topic
Modules