ICC Home Privacy and Legal Notice LC User Documents Banner


SLURM Reference Manual

SRUN Resource-Allocation Options

These SRUN options (used alone or in combination) assign compute resources to your parallel SLURM-managed job. Most options have a one-character (UNIX) and a longer (Linux) alternative syntax. See also SRUN's other options that can affect node management for your job, especially the control options and constraint options, in separate subsections below. Speciality allocators follow the primary ones (-n, -N) in alphabetical order.

-n procs (lowercase, --ntasks=procs)
requests that SRUN execute procs processes or tasks. To control how these processes are distributed among nodes and CPUs, combine -n with -c or -N as explained below (default is one process per node). On LC machines where Moab has replaced LCRM as the batch-job scheduler, you can use SRUN's -n to replace the (former) PSUB -np option that MSUB does not offer.
-N n (uppercase, --nodes=n)
allocates at least n nodes to this job, where n may be either
(1) a specific node count (such as -N16), or
(2) a hyphen-separated range from a minimum to a maximum node count (such as -N2-4).
If the nodes are partitioned, each partition's node limits supersede those specified by -N (jobs that request more nodes than the partition allows never leave SLURM's PENDING or Moab's IDLE state). To change partitions, use SRUN's -p option (below). See also -B (--extra-node-info) below in this section to specify your socket, core, or thread allocation as well as nodes. Combinations of -n and -N control how job processes are distributed among nodes according to the SRUN policies listed here:

SRUN infers your intended number of processes per node if you specify both the number of processes and the number of nodes for your job. Thus -n16-N8 normally results in running 2processes/node (but see the next policy for exceptions).

SRUN interprets all node requests as minimum node requests (so -N16 means "at least 16 nodes"). If some nodes lack enough CPUs to cover the process count specified by -n, SRUN will automatically allocate more nodes (than mentioned with -N) to meet the need (for example, if not all nodes have 2 working CPUs, then -n32-N16 together will allocate more than 16 nodes so that all processes are supported). The actual number of nodes assigned (not the number reqested) is stored in environment variable SLURM_NNODES.

By default, SRUN never allocates more than one process per CPU. If you intend to assign multiple processes per CPU, you must invoke SRUN's -O (uppercase oh) option along with -n and -N (thus -n16-N4-O together allow 2processes/CPU on the 4 allocated 2-CPU nodes).

SRUN rejects as errors inconsistent -n/-N combinations. For example, -n15-N16 requests the impossible assignment of 15 processes to 16 nodes.

-B sockets[:cores[:threads]] (uppercase, --extra-node-info=sockets[:cores[:threads]])
requests a resource allocation with specific details that supplement the general node allocation requested with -N (--nodes) above. Use -B to specify your job's needed sockets (subnodes) per addressable node (and optionally cores (CPUs) per socket and threads per core). Instead of this one combined option you can use these three separate options if you wish, with the same results:

When the task/affinity plug-in is enabled, invoking -B or its three equivalent options also makes SLURM use a CPU affinity mask to fill your allocation request as specified.

As with -N (--nodes), you can specify values here with
(1) a single count number (-B 4), or
(2) a hyphen-separated range from minimum to maximum (-B 2-4), or
(3) an asterisk (*) to request all available resources of the specified type (-B 4:*).

All of these four related options depend on SLURM's configuration:
(1) The task/affinity plug-in must be configured.
(2) The select/linear or select/con-res plug-in must be configured.
(3) If select/con-res, then it must have one of these parameters: CR_Core, CR_Core_Memory, CR_Socket, or CR_Socket_Memory.

-c cpt (lowercase, --cpus-per-task=cpt)
assigns cpt CPUs per process for this job (default is one CPU/process). This option supports multithreaded programs that require more than a single CPU/process for best performance. For a different approach to the same need, see the --ntasks-per-node option in the "Affinity Constraints" section below.

For multithreaded programs where the density of CPUs is more important than a specific node count, use both -n and -c on the same SRUN execute line (rather than -N). Thus -n16-c2 results in whatever node allocation is needed to yield the requested 2CPUs/process. This is the reverse of CPU overcommitment (see -N and -O, above).

assigns different executable programs with (perhaps) different arguments to each task. When you use this option, SRUN's own argument is not a parallel program but instead a 3-column configuration file that specifies your matrix of tasks, programs, and their arguments. See the Multiple Program Usage section below for details and an example of using this option.
-p part (lowercase, --partition=part)
requests nodes only from the part partition (the default partition is assigned by the system administrator on each separate LC machine).
-t min (lowercase, --time=min)
allocates a total of min minutes for this job to run (default is the current partition's time limit). If min exceeds the partition's time limit, then the job never leaves the PENDING state. When the time limit has been reached, SLURM sends each job process SIGTERM followed (after a pause specified by SLURM's KillWait configuration parameter) by SIGKILL.
-T nthreads (uppercase, --threads=nthreads)
requests that SRUN allocate nthreads threads to initiate and control the parallel tasks in this job (default is the smaller of 10 or the number of nodes actually allocated, SLURM_NNODES).

Navigation Links: [ Document List ] [ HPC Home ] [ Next ]