SLURM Reference Manual
SRUN Resource-Allocation Options
These SRUN options (used alone or in combination) assign compute
resources to your parallel SLURM-managed job.
Most options have a one-character (UNIX) and a longer (Linux)
See also SRUN's other options that can affect node management
for your job, especially the
control options and
in separate subsections below.
Speciality allocators follow the primary ones (-n, -N) in
- -n procs (lowercase, --ntasks=procs)
requests that SRUN execute procs processes
To control how these processes are distributed among nodes and CPUs,
combine -n with -c or -N as explained below
(default is one process per node).
On LC machines where Moab has replaced LCRM as the batch-job
scheduler, you can use SRUN's -n to replace the (former) PSUB
-np option that MSUB does not offer.
- -N n (uppercase, --nodes=n)
allocates at least n nodes to this job, where
n may be either
(1) a specific node count (such as -N16), or
(2) a hyphen-separated range from a minimum to a maximum node count
(such as -N2-4).
If the nodes are partitioned, each partition's node limits supersede
those specified by -N (jobs that request more nodes than the partition
allows never leave SLURM's PENDING or Moab's IDLE state).
To change partitions, use SRUN's -p option (below).
See also -B (--extra-node-info) below in this section to specify
your socket, core, or thread allocation as well as nodes.
Combinations of -n and -N control how job processes are distributed
among nodes according to the SRUN policies listed here:
SRUN infers your intended number of processes per node if you specify
both the number of processes and the number of nodes for your job.
Thus -n16-N8 normally results in running 2processes/node
(but see the next policy for exceptions).
SRUN interprets all node requests as minimum node requests (so -N16
means "at least 16 nodes").
If some nodes lack enough CPUs to cover the process count specified by
-n, SRUN will automatically allocate more nodes (than mentioned with -N)
to meet the need (for example, if not all nodes have 2 working CPUs,
then -n32-N16 together will allocate more than
16 nodes so that all processes are supported).
The actual number of nodes assigned (not the number reqested) is
stored in environment variable SLURM_NNODES.
By default, SRUN never allocates more than one process per CPU.
If you intend to assign multiple processes per CPU, you must invoke
SRUN's -O (uppercase oh) option along with -n and -N (thus
-n16-N4-O together allow 2processes/CPU on
the 4 allocated 2-CPU nodes).
SRUN rejects as errors inconsistent -n/-N combinations.
For example, -n15-N16 requests the impossible assignment
of 15 processes to 16 nodes.
- -B sockets[:cores[:threads]] (uppercase, --extra-node-info=sockets[:cores[:threads]])
requests a resource allocation with specific details that supplement
the general node allocation requested with -N (--nodes) above.
Use -B to specify your job's needed sockets (subnodes) per
addressable node (and optionally cores (CPUs) per socket and threads
Instead of this one combined option you can use these three
separate options if you wish, with the same results:
When the task/affinity plug-in is enabled, invoking -B or its
three equivalent options also makes SLURM use a CPU affinity mask
to fill your allocation request as specified.
As with -N (--nodes), you can specify values here with
(1) a single count number (-B 4), or
(2) a hyphen-separated range from minimum to maximum (-B 2-4), or
(3) an asterisk (*) to request all available resources of the
specified type (-B 4:*).
All of these four related options depend on SLURM's configuration:
(1) The task/affinity plug-in must be configured.
(2) The select/linear or select/con-res plug-in must be configured.
(3) If select/con-res, then it must have one of these parameters:
CR_Core, CR_Core_Memory, CR_Socket, or CR_Socket_Memory.
- -c cpt (lowercase, --cpus-per-task=cpt)
assigns cpt CPUs per process for this job
(default is one CPU/process).
This option supports multithreaded programs that require more than
a single CPU/process for best performance.
For a different approach to the same need, see the --ntasks-per-node
option in the "Affinity Constraints"
For multithreaded programs where the density of CPUs is more important
than a specific node count, use both -n and -c on the same SRUN
execute line (rather than -N).
Thus -n16-c2 results in whatever node allocation is needed
to yield the requested 2CPUs/process.
This is the reverse of CPU overcommitment (see -N and -O, above).
- assigns different executable programs with (perhaps)
different arguments to each task.
When you use this option, SRUN's own argument is not a parallel
program but instead a 3-column configuration file that specifies
your matrix of tasks, programs, and their arguments.
See the Multiple Program Usage
section below for details and an example of using this option.
- -p part (lowercase, --partition=part)
requests nodes only from the part partition
(the default partition
is assigned by the system administrator on each separate LC
- -t min (lowercase, --time=min)
allocates a total of min minutes for this job to run
(default is the current partition's time limit).
If min exceeds the partition's time limit, then the job never
leaves the PENDING state.
When the time limit has been reached, SLURM sends each job process
SIGTERM followed (after a pause specified by SLURM's KillWait
configuration parameter) by SIGKILL.
- -T nthreads (uppercase, --threads=nthreads)
requests that SRUN allocate nthreads threads to initiate and
control the parallel tasks in this job (default is the smaller of 10
or the number of nodes actually allocated, SLURM_NNODES).
Navigation Links: [
Document List ] [
HPC Home ]