ICC Home Privacy and Legal Notice LC User Documents Banner

UCRL-WEB-201386

SLURM Reference Manual


Environment Variables

To see how the SLURM environment variables discussed here fit into the larger context of all environment variables used at LC to manage jobs (both interactively and by LCRM in particular), consult the comparative sections of LC's Environment Variables user manual.

Option Variables.
Many SRUN options have corresponding environment variables (analogous to the approach used with POE). The SRUN option, if invoked during execution, always overrides (resets) the corresponding environment variable (which contains each job feature's default value, if there is a default).


Environment Variable Corresponding SRUN Option(s)

SLURM_ACCOUNT -U, --account
SLURM_CPU_BIND --cpu_bind
SLURM_CPUS_PER_TASK -c, --ncpus-per-task
SLURM_CONF [configuration file location]
SLURM_CONN_TYPE --conn-type
SLURM_CORE_FORMAT --core-format
SLURM_DEBUG -v, --verbose
SLURMD_DEBUG -d, --slurmd-debug
SLURM_DEPENDENCY -P, --dependency
SLURM_DISABLE_STATUS -X, --disable-status
SLURM_DIST_PLANESIZE -m plane
SLURM_DISTRIBUTION -m, --distribution
SLURM_EPILOG --epilog
SLURM_EXCLUSIVE --exclusive
SLURM_GEOMETRY -g, --geometry
SLURM_JOB_NAME -J, --job-name
SLURM_LABELIO -l, --label
SLURM_MEM_BIND --mem_bind
SLURM_NETWORK(*) --network
SLURM_NNODES -N, --nodes
SLURM_NO_REQUEUE --no-requeue
SLURM_NO_ROTATE --no-rotate
SLURM_NPROCS -n, --ntasks
SLURM_NTASKS_PER_CORE --ntasks-per-core
SLURM_NTASKS_PER_NODE --ntasks-per-node
SLURM_NTASKS_PER_SOCKET --ntasks-per-socket
SLURM_OVERCOMMIT -o, --overcommit
SLURM_PARTITION -p, --partition
SLURM_PROLOG --prolog
SLURM_REMOTE_CWD -D, --chdir
SLURM_SRUN_COMM_IFHN --ctrl-comm-ifhn
SLURM_STDERRMODE -e, --error
SLURM_STDINMODE -i, --input
SLURM_STDOUTMODE -o, --output
SLURM_TASK_EPILOG --task-epilog
SLURM_TASK_PROLOG --task-prolog
SLURM_TIMELIMIT -t, --time
SLURM_UNBUFFEREDIO -u, --unbuffered
SLURM_WAIT -W, --wait
SLURM_WORKING_DIR -D, --chdir

(*)See explanatory details at the end of this section.

Task-Environment Variables.
In addition, SRUN sets these environment variables (a few are the same as option variables listed above) for each executing task on each remote compute node (any operating system). On LC machines where Moab has replaced LCRM as the scheduler for batch jobs, Moab uses (most of) these variables instead of PSUB variables to manage the attributes of the jobs that it runs. A few (as noted) were added exclusively to support Moab.

SLURM_CPU_BIND_VERBOSE
affects the reporting of CPU/task binding, as explained in the "Affinity or NUMA Constraints" section under --cpu_bind.
SLURM_CPU_BIND_TYPE
affects the binding of CPUs to tasks, as explained in the "Affinity or NUMA Constraints" section under --cpu_bind.
SLURM_CPU_BIND_LIST
affects the binding of CPUs to tasks, as explained in the "Affinity or NUMA Constraints" section under --cpu_bind.
SLURM_CPUS_ON_NODE
specifies the number of processors available to the job on this node.
SLURM_JOBID
specifies the job ID of the executing job (see also SRUN's --jobid option).
SLURM_LAUNCH_NODE_IPADDR
specifies the IP address of the node from which the task launch initiated (the node where SRUN executed).
SLURM_LOCALID
specifies the node-local task ID for the process within a job.
SLURM_MEM_BIND_VERBOSE
affects the reporting of memory/task binding, as explained in the "Affinity or NUMA Constraints" section under --mem_bind.
SLURM_MEM_BIND_TYPE
affects the binding of memory to tasks, as explained in the "Affinity or NUMA Constraints" section under --mem_bind.
SLURM_MEM_BIND_LIST
affects the binding of memory to tasks, as explained in the "Affinity or NUMA Constraints" section under --mem_bind.
SLURM_NNODES
is the actual number of nodes assigned to run your job (which may exceed the number of nodes that you explicitly requested with SRUN's -N option).
SLURM_NODEID
specifies the relative node ID of the current node.
SLURM_NODELIST
specifies the list of nodes on which the job is actually running.
SLURM_NPROCS
specifies the total number of processes in the job.
SLURM_PRIO_PROCESS
(for Moab jobs) is the NICE value of the job when submitted. SLURM propagates this to all spawned processes.
SLURM_PROCID
specifies the MPI rank (or relative process ID) for the current process.
SLURM_STEPID
specifies the step ID of the current job.
SLURM_TASK_PID
(for Moab jobs) specifies the process ID of the task being started on a compute node.
SLURM_TASKS_PER_NODE
specifies the number of tasks to initiate on each node. Values are a comma-delimited list in the same order as SLRUM_NODELIST. To specify two or more nodes with the same task count, follow the count by (x#), where # is the repetition count. For example,
SLURM_TASKS_PER_NODE=2(x3),1
indicates two tasks per node on the first three nodes, then one task on the fourth node.
SLURM_UMASK
(for Moab jobs) captures the UMASK (user file-create mask) setting for this job's user at the time of submittal. SLURM propagates this to all spawned processes.
MPIRUN_PARTITION
(BlueGene/L only) specifies the block name.
MPIRUN_NOALLOCATE
(BlueGene/L only) prevents allocating a block.
MPIRUN_NOFREE
(BlueGene/L only) prevents freeing a block.

Other SLURM-Relevant Variables.
Other environment variables important for SRUN-managed jobs include:

MAX_TASKS_PER_NODE
provides an upper bound on the number of tasks that SRUN assigns to each job node, even if you allow more than one process per CPU by invoking SRUN's -O (uppercase oh) option.
SLURM_HOSTFILE
names the file that specifies how to assign tasks to nodes, rather than using the block or cyclic approaches toggled by SRUN's -m (--distribution) option.

On AIX (IBM) machines only, the SLURM_NETWORK environment variable (described below) is automatically set by LCRM when SLURM is used instead of LoadLeveler (alternatively, set with SRUN's --network=type option even though POE launches tasks instead of SRUN under AIX). On (non-AIX) machines that use Moab instead of LCRM for batch-job scheduling, SLURM_NETWORK and --network can also take the place of some specialized former PSUB options.

SLURM_NETWORK
specifies four network features for each SLURM job step (which under AIX means for each POE invocation), using an argument with this sequential format:
network.[protocol],[device],[adapteruse],[mode]
where:
protocol
specifies the network protocol (such as MPI).
device
specifies the kind of switch used for communication (ethernet, FDDI, etc.), where the choices are the same abbreviation strings as the possible values of environment variable MP_EUIDEVICE (see the POE User Guide, "Task Communications" section).
adapteruse
specifies whether (SHARED) or not (DEDICATED) your job is willing to share a node's switch adapter with other jobs (see the POE User Guide, "Other POE Environment Variables" section on corresponding environment variable MP_ADAPTER_USE).
mode
specifies which of two protocols or modes should be used for task communications, where the choices are the same as the possible values of environment variable MP_EUILIB (see the POE User Guide, "Task Communications" section).



Navigation Links: [ Document List ] [ HPC Home ] [ Next ]