PBS to Slurm

Below is some information which will be useful in transitioning from the PBS style of batch jobs used on Fionn to the Slurm jobs used on Kay. While the concepts are quite similar in most common use cases, different commands and options are used.

Translation table for commands

Function PBS Slurm
Submit job script qsub sbatch
Request interactive job qsub -I srun --pty bash
Query queue state qstat (or showq) squeue
Cancel job qdel (or canceljob) scancel

 

Translation table for batch file options

Job specifications can be given on the commandline (sbatch and srun) or as part of a batch submission script. These options must be preceded by #PBS or #SBATCH for PBS and Slurm respectively. 

To specify: qsub option sbatch option Comment
queue to run in -q queuename -p queuename Slurm refers to queues as partitions
account to charge -A accountname -A accountname  
number of nodes -l nodes=X:ppn=Y -N X On Kay, you just need to request the number of Nodes, X as all nodes have the same core count
walltime -l walltime=HH:MM:SS -t HH:MM:SS Slurm also allows other time formats, see man page
stdout file -o outfilename -o outfilename This will combine stdout/stderr on slurm if -e not given also
stderr file -e errorfilename -e errorfilename This will combine stdout/stderr on slurm if -o not given also
Combining stdout/stderr -j oe   see above
email address -M me@x.y --mail-user=me@x.y  
email options -mb --mail-type=BEGIN Send email when job starts
-me --mail-type=END Send email when job ends
Job name -N NAME --job-name=NAME  
       
environment variables $PBS_O_WORKDIR $SLURM_SUBMIT_DIR submit directory
$PBS_NODEFILE $SLURM_JOB_NODELIST list of nodes assigned to job
$PBS_JOBID $SLURM_JOBID job ID
$PBS_NUM_NODES $SLURM_JOB_NUM_NODES number of nodes allocated to job
$PBS_NUM_PPN $SLURM_CPUS_ON_NODE number of cores per node

 

Equivalent PBS and Slurm example submit scripts
#!/bin/sh
#PBS -q ProdQ
#PBS -l nodes=2:ppn=24
#PBS -l walltime=120:00:00
# Charge job to myaccount
#PBS -A myaccount
# Write stdout+stderr to file
#PBS -o output.txt
#PBS -j oe
# Mail me on job start & end
#PBS -M myemail@universityname.ie
#PBS -m be
cd $PBS_O_WORKDIR
module load intel
mpiexec -n 48 ./my_mpi_app
#!/bin/sh
#SBATCH -p ProdQ
#SBATCH -N 2
#SBATCH -t 120:00:00
# Charge job to myaccount
#SBATCH -A myaccount
# Write stdout+stderr to file
#SBATCH -o output.txt
# Mail me on job start & end
#SBATCH --mail-user=myemail@universityname.ie
#SBATCH --mail-type=BEGIN,END
cd $SLURM_SUBMIT_DIR
module load intel
srun ./my_mpi_app