ICC Home Privacy and Legal Notice LC User Documents Banner

UCRL-WEB-200040

CHAOS: Linux from Livermore


NUMACTL (Control Memory Policy)

NUMACTL has both a reporting role and a role in setting memory policy on LC Linux/CHAOS machines that have NUMA hardware. The parent of this subsection explains the meaning of NUMA hardware and memory policy.

Some NUMACTL options report on current NUMA node resources or their memory policies. Other NUMACTL options let you specify the memory policy that applies to a command that NUMACTL executes. (Still other options, omitted here, set memory policy for "shared memory segments" on machines where those exist.)

REPORTING NUMA NODES:
To discover details about the NUMA hardware on the addressable node (such as atlas32) where you execute NUMACTL, type

numactl --hardware|--show

where
--hardware
lists the "NUMA nodes" (multiple-cpu sockets) on the current addressable node and reports the total and free memory for each one. A typical report has the form:
     available: 4 nodes (0-3)
     node 0 size: 3311 MB
     node 0 free: 1072 MB
     ...

--show
lists the memory-policy attributes that you can set by using the NUMACTL options listed below and reports the current value of each attribute. A typical report has the form:
     policy: default
     preferred node: current
     physcpubind: 0 1 2 3 4 5 6 7
     cpubind: 0 1 2 3
     node bind: 0 1 2 3
     mem bind: 0 1 2 3

SETTING MEMORY POLICY:
To specify your desired memory-allocation policy for the NUMA node(s) where a specified command executes, type

numactl policyopt command [arguments]

where policyopt is one of the following mutually exclusive alternatives (in order from most restrictive to most flexible, plus two others for CPUs):

--localalloc
(-l) always allocates memory only on the node where the process runs (but no others).
--membind=nodes
(-m nodes) allocates memory only on the specified nodes (a comma-delmited list of NUMA-node numbers, a hyphen-linked node range A-B, or all). If the specified nodes lack enough free memory, the allocation fails.
--preferred=node
(no short form) allocates memory "preferably" on the specified node (always a single node number), but uses memory from other NUMA nodes if the free memory on the chosen node is insufficient.
--interleave=nodes
(-i nodes) allocates memory round-robin from the specified nodes (same syntax as for --membind). When free memory is not available on any interleave target node, memory from other NUMA nodes in the round-robin set is used.
--cpunodebind=nodes
(-N nodes) executes command only on CPUs of the specified node(s) (same syntax as for --membind).
--physcpubind=cpus
(-C cpus) executes command only on CPUs specified by their physical CPU numbers as shown in the /proc/cpuinfo file.



Navigation Links: [ Document List ] [ HPC Home ] [ Next ]