AIX+Colony, AIX+Federation
Comprehensive List of Environment Variables

Privacy & Legal Notice

The following tables summarize environment variables and command-line flags. You can set these variables and flags to influence the execution of parallel programs and the operation of certain tools. The command-line flags temporarily override their associated environment variable. Note that LC's IBM ASC systems preset several of these environment variables.

The tables divide the environment variables and flags by function. The primary source of information on this topic is the IBM publication "Parallel Environment, Operation and Use, Vol. 1." The IBM documentation is available online.

AIX_MINKTHREADS
AIX_MNRATIO
AIX_PTHREAD_SET_STACKSIZE
AIXTHREAD_COND_DEBUG
AIXTHREAD_MUTEX_DEBUG
AIXTHREAD_RWLOCK_DEBUG
AIXTHREAD_SCOPE
AIXTHREAD_SLPRATIO
MALLOCDEBUG
MALLOCTYPE
MALLOCMULTIHEAP
MP_ADAPTER_USE
MP_BUFFER_MEM
MP_CHECKDIR
MP_CHECKFILE
MP_CLOCK_SOURCE
MP_CMDFILE
MP_COREDIR
MP_COREFILE_FORMAT
MP_COREFILE_SIGTERM
MP_CPU_USE
MP_CSS_INTERRUPT
MP_DBXPROMPTMOD
MP_DEBUG_INITIAL_STOP
MP_DEBUG_LOG
MP_EAGER_LIMIT
MP_EUIDEVELOP
MP_EUIDEVICE
MP_EUILIB
MP_EUILIBPATH
MP_FENCE
MP_HINTS_FILTERED
MP_HOLD_STDIN
MP_HOSTFILE
MP_INFOLEVEL
MP_INTRDELAY
MP_IONODEFILE
MP_LABELIO
MP_LLFILE
MP_MAX_TYPEDEPTH
MP_MSG_API
MP_NEWJOB
MP_NOARGLIST
MP_NODES
MP_PGMMODEL
MP_PMD_VERSION
MP_PMDLOG
MP_PMDSUFFIX
MP_PMLIGHTS
MP_POLLING_INTERVAL
MP_PRIORITY
MP_PROCS
MP_PULSE
MP_RESD
MP_RETRY
MP_RETRYCOUNT
MP_RMPOOL
MP_SAMPLEFREQ
MP_SAVE_LLFILE
MP_SAVEHOSTFILE
MP_SHARED_MEMORY
MP_SINGLE_THREAD
MP_STDINMODE
MP_STDOUTMODE
MP_SYNC_ON_CONNECT
MP_TASKS_PER_NODE
MP_TBUFFSIZE
MP_TBUFFWRAP
MP_THREAD_STACKSIZE
MP_TIMEOUT
MP_TMPDIR
MP_TRACEDIR
MP_TRACELEVEL
MP_TTEMPSIZE
MP_USE_FLOW_CONTROL
MP_USRPORT
MP_WAIT_MODE
PSALLOC
RT_GRQ
SPINLOOPTIME
YIELDLOOPTIME
XLSMPOPTS

You can use the POE command-line flags on the pdbx and poe commands. You can also use some of these flags on program names when individually loading nodes from STDIN or a POE commands file. The flags you can use are mainly those having to do with parallel trace collection. They are:

Note: In the tables that follow, text in italic denotes those flags you can use when individually loading nodes.

Table 1.
POE Environment Variables/Command-Line Flags for Partition Manager Control
The Environment Variable Command-Line Flag(s): Set: Possible Values: Default:
MP_ADAPTER_USE

-adapter_use
How the node's adapter should be used.

Note:

    If using LoadLeveler, the US communication subsystem library does not require dedicated use of the SP switch on the node. Adapter use will be defaulted, but shared usage may be specified. If using the Resource Manager, this value is only used when a host list file is not being used, that is, only when POE is requesting non-specific nodes via the MP_RMPOOL or -rmpool setting.The value of this environment variable can be overridden using the -adapter_use flag.

One of the following strings:

dedicated

    only a single program task can use the adapter.

shared

    a number of tasks on the node can use the adapter.

dedicated (US jobs)
shared (IP jobs)
MP_CPU_USE

-cpu_use
How the node's CPU should be used. The value of this environment variable can be overridden using the -cpu_use flag.

Note:

    If using LoadLeveler, the US communication subsystem library does not require unique CPU use on the node. CPU use will be defaulted, but multiple usage may be specified. If using the Resource Manager, this value is only used when a host list file is not being used, that is, only when POE is requesting non-specific nodes via the MP_RMPOOL or -rmpool setting. 

One of the following strings:

unique

    only your program task can use the CPU.

multiple

    your program may share the node with other users.

unique (US jobs)
multiple (IP jobs)
MP_EUIDEVICE

-euidevice
The adapter set to use for message passing: either Ethernet, FDDI, token-ring, SP Switch or SP Switch2. The value of this environment variable can be overridden using the -euidevice flag. One of the following strings:

en0

    Ethernet

fi0

    FDDI

tr0

    token-ring

css0

    SP switch

csss

    SP switch2

The adapter set used as the external network address.
MP_EUILIB

-euilib
The communication subsystem library implementation to use for communication: either the IP communication subsystem or the US communication subsystem. Programs that use LAPI must set MP_EUILAB (or -ueilib) to us. To use the US communication subsystem, you must have an SP system configured with its high performance switch feature. The value of this environment variable can be overridden using the -euilib flag.. One of the following strings:

ip

    the IP communication subsystem

us

    the US communication subsystem

Note:

    This specification is case-sensitive.

ip
MP_EUILIBPATH

-euilibpath
The path to the message passing and communication subsystem libraries. This only needs to be set if the libraries are moved. The value of this environment variable can be overridden using the -euilibpath flag. Any path specifier /usr/lpp/ppe.poe/lib
MP_HOSTFILE

-hostfile
-hfile
The name of a host list file for node allocation. The value of this environment variable can be overridden by using the -hostfile or -hfile flag. Any file specifier or the word NULL. host.list in the current directory
MP_PMD_VERSION

-pmd_version
The version of the Partition Manager daemon to be used. It is intended for situations where you need to submit a parallel job from a POE version 3 node to a set of nodes that have not yet been upgraded to version 3. The value of this environment variable can be overridden by using the -pmd_version flag. 2

    Version 2 Partition Manager daemon - pmv2

3

    Version 3 Partition Manager daemon - pmv3

3
MP_PROCS

-procs
The number of program tasks. The value of this environment variable can be overridden by using the -procs flag. Any number from 1 to the maximum supported configuration. 1
MP_PULSE

-pulse
The interval (in seconds) at which POE checks the remote nodes to ensure that they are actively communicating with the home node. To disable the pulse function, specify an interval of 0 seconds. The value of this environment variable can be overridden using the -pulse flag.

Note:

    Pulse is ignored for pdbx and pedb.

An integer greater than or equal to 0. 600
MP_RESD

-resd
Whether or not the Partition Manager should connect to LoadLeveler to allocate nodes. This environment variable can be overridden by using the -resd flag.

Note:

    Setting MP_RESD=no completely avoids the LoadLeveler Resource Manager. It will not insure exclusive access to nodes--you can start a job on a node already occupied or have another job start using the node that you have already started working on. Programs initiated in this mode will be unable to utilize User Space communications. This might be an acceptable mode of operation in special cases. This mechanism might be used to execute a single job across multiple computers. It could also be used to initiate POE commands on specific nodes after having a block of nodes previously allocated via LoadLeveler.

yes

no

None
MP_RETRY

-retry
The period of time (in seconds) between processor node allocation retries if there are not enough processor nodes immediately available to run a program. This is only valid if you are using LoadLeveler. The value of this environment variable can be overridden by using the -retry flag. An integer greater than or equal to 0. 0 (no retry)
MP_RETRYCOUNT

-retrycount
The number of times (at the interval set by MP_RETRY) that the Partition Manager should attempt to allocate processor nodes. The value of this environment variable can be overridden by using the -retrycount flag. An integer greater than or equal to 0. 0
MP_MSG_API

-msg_api
To indicate to POE which message passing API is being used by the parallel tasks. You need to set this environment variable if a parallel task is using LAPI alone or in conjunction with MPI. You do not need to set it if a parallel task is using MPI only. The value of this environment variable can be overridden by using the -msg_api flag. MPI LAPI        or
MPI, LAPI
MPI
MP_RMPOOL

-rmpool
Determines the name or number of the pool that should be used for non-specific node allocation. This environment variable/command-line flag only applies to LoadLeveler. The value of this environment variable can be overridden by using the -rmpool flag. An identifying pool number. None
MP_NODES

-nodes
To specify the number of physical nodes on which to run the parallel tasks. It may be used alone or in conjunction with MP_TASKS_PER_NODE and/or MP_PROCS. The value of this environment variable can be overridden by using the -nodes flag. Any number from 1 to the supported configuration. None
MP_TASKS_PER_NODE

-tasks_per_node
To specify the number of tasks to run on each of the physical nodes. It may be used in conjunction with MP_NODES and/or MP_PROCS, but may not be used alone. The value of this environment variable can be overridden by using the -tasks_per_node flag. Any number from 1 to the supported configuration. None
MP_SAVEHOSTFILE

-savehostfile
The name of an output host list file to be generated by the Partition Manager. The value of this environment variable can be overridden by using the -savehostfile flag. Any relative or full path name. None
MP_TIMEOUT

(no associated command line flag)
The length of time that POE waits before abandoning an attempt to connect to the remote nodes. MP_TIMEOUT also changes the length of time the communication subsystem will wait for a connection to be established during message passing initialization.

Note:

    If the SP security method is "dce and compatibility", you may need to increase the MP_TIMEOUT value to allow POE to wait for the DCE servers to respond (or timeout if the servers are down).

Any number greater than 0. 150 seconds.
MP_CHECKFILE

(no associated command line flag)
The base name of the checkpoint file when checkpointing or restarting a file. Any file specifier. None
MP_CHECKDIR

(no associated command line flag)
The directory where the checkpoint file will reside when checkpointing or restarting a program. Any path specifier. None

Return to top of page

Table 2.
POE Environment Variables/Command-Line Flags for Job Specification
The Environment Variable Command-Line Flag(s): Set: Possible Values: Default:
MP_CMDFILE

-cmdfile
The name of a POE commands file used to load the nodes of your partition. If set, POE will read the commands file rather than STDIN. The value of this environment variable can be overridden by using the -cmdfile flag. Any file specifier. None
MP_LLFILE

-llfile
The name of a LoadLeveler job command file for node allocation. If you are performing specific node allocation, you can use a LoadLeveler job command file in conjunction with a host list file. If you do, the specific nodes listed in the host list file will be requested from LoadLeveler. The value of this environment variable can be overridden by using the -llfile flag. Any relative or full path name. None
MP_NEWJOB

-newjob
Whether or not the Partition Manager maintains your partition for multiple job steps. The value of this environment variable can be overridden by using the -newjob flag. yes

no

no
MP_PGMMODEL

-pgmmodel
The programming model you are using. spmd

mpmd

spmd
MP_SAVE_LLFILE

-save_llfile
When using LoadLeveler for node allocation, the name of the output LoadLeveler job command file to be generated by the Partition Manager. The output LoadLeveler job command file will show the LoadLeveler settings that result from the POE environment variables and/or command-line options for the current invocation of POE. If you use the MP_SAVE_LLFILE environment variable for a batch job, or when the MP_LLFILE environment variable is set (indicating that a LoadLeveler job command file should participate in node allocation), POE will show a warning and will not save the output job command file. The value of this environment variable can be overridden by using the -save_llfile flag. Any relative or full path name.  None

Return to top of page

Table 3.
POE Environment Variables/Command-Line Flags for I/O Control
The Environment Variable Command-Line Flag(s): Set: Possible Values: Default:
MP_LABELIO

-labelio
Whether or not output from the parallel tasks is labeled by task ID. The value of this environment variable can be overridden by using the -labelio flag. yes

no

no (yes for pdbx)
MP_STDINMODE

-stdinmode
The input mode. This determines how input is managed for the parallel tasks. The value of this environment variable can be overridden by using the -stdinmode flag. all

    all tasks receive the same input data from STDIN.

none

    no tasks receive input data from STDIN; STDIN will be used by the home node only.

a task id

    STDIN is only sent to the task identified.

all
MP_HOLD_STDIN

(no associated command line flag)
Whether or not sending of STDIN from the home node to the remote nodes is deferred until the message passing partition has been established. tasks. yes

no

no
MP_STDOUTMODE

-stdoutmode
The output mode. This determines how STDOUT is handled by the parallel tasks. The value of this environment variable can be overridden by using the -stdoutmode flag. One of the following:

unordered

    all tasks write output data to STDOUT asynchronously.

ordered

    output data from each parallel task is written to its own buffer. Later, all buffers are flushed, in task order, to STDOUT.

a task id

    only the task indicated writes output data to STDOUT.

unordered

Return to top of page

Note: Text in italics denotes flags you can use when individually loading nodes.

Table 4.
POE Environment Variables/Command-Line Flags for Parallel Trace Collection
The Environment Variable Command-Line Flag(s): Set: Possible Values: Default:
MP_SAMPLEFREQ

-samplefreq
-sfreq
The interval (in milliseconds) at which AIX kernel statistics are sampled when executing a program with tracing on. The value of this environment variable can be overridden by using the -samplefreq or -sfreq flags. An integer greater than or equal to 0.  10
MP_TBUFFSIZE

-tbuffsize
-tbsize
The size (in bytes) of the buffer used when generating trace files. The value of this environment variable can be overridden by using the -tbuffsize or -tbsize flags. nnnK

nnnM

5M
MP_TBUFFWRAP

-tbuffwrap
-tbwrap
A wraparound storage approach for trace records instead of the default three-tiered approach. With this approach, the system keeps overwriting the buffer instead of flushing it to a temp file. The value of this environment variable can be overridden by using the -tbuffwrap or -tbwrap flags. yes

no

no
MP_TMPDIR

-tmpdir
The temporary directory to which output trace files are written. The value of this environment variable can be overridden by using the -tempdir flag.

Note:

    This environment variable/flag is also used by the pdbx command to determine the directory to which individual startup files are written for each dbx task. This is where the remote debug logfile is written.

Any path specifier. /tmp/username

/tmp (for pedb and pdbx)

MP_TRACEDIR

-tracedir
-tdir
The directory to which the final integrated trace file is built. The value of this environment variable can be overridden by using the -tracedir or -tdir flags. Any path specifier. The current directory.
MP_TRACELEVEL

-tracelevel
-tlevel
The level of parallel tracing that should be generated during the execution of a program. The value of this environment variable can be overridden by using the -tracelevel or -tlevel flags.

Note:

    pedb automatically sets the trace level to 0.

One of the following integers:

0

    no trace records

1

    Application Markers

2

    AIX Kernel Statistic and Application Markers

3

    Message Passing, Collective Communication, and Application Markers

9

    all trace records

0
MP_TTEMPSIZE

-ttempsize
-ttsize
The size (in bytes) of the temp file used when generating trace files. The value of this environment variable can be overridden by using the -ttempsize or -ttsize flags. nnnM

nnnG

10M

Return to top of page

Note: Text in italics denotes flags you can use when individually loading nodes.
Table 5.
POE Environment Variables/Command-Line Flags for Diagnostic Information
The Environment Variable Command-Line Flag(s): Set: Possible Values: Default:
MP_INFOLEVEL

-infolevel
-ilevel
The level of message reporting. The value of this environment variable can be overridden by using the -infolevel or -ilevel flags.

Note:

    Value of output is proportional to number of processes and can become large. Users may find it useful to set this value to 2 or 3 during development debug, especially if there are concerns about which nodes are being used or why startup is slow or failing.

One of the following integers:

0

    error

1

    warning and error

2

    informational, warning, and error

3

    informational, warning, and error. Also reports high-level diagnostic messages for use by the IBM Support Center.

4,5,6

    informational, warning, and error. Also reports high-level and low-level diagnostic messages for use by the IBM Support Center.

1
MP_PMDLOG

-pmdlog
Whether or not diagnostic messages should be logged to a file in /tmp on each of the remote nodes. Typically, this environment variable/command-line flag is only used under the direction of the IBM Support Center in resolving a PE-related problem. The value of this environment variable can be overridden by using the -pmdlog flag. yes

no

no
MP_DEBUG_LOG

(no associated command-line flag)
The level of diagnostic messages written to $MP_tmp/dbelog.pid.taskid. Typically, this environment variable/command-line flag is only used under the direction of the IBM Support Center in resolving a PE-related problem.

Note:

    MP_DEBUG_LOG is only valid for pedb.

0 - 4 0
MP_DEBUG_INITIAL_STOP

(no associated command-line flag)
The initial breakpoint in the application where pdbx or pedb will get control. 

Note:

MP_DEGUB_INITIAL_STOP should be specified as file_name:line_number. The line_number is the number of the line within the source file file_name, where file_name has been compiles with -g. The line number has to be one that defines executable code. In general, this is a line of code for which the compiler generates machine level code. Another way to view this is that the line number is one for which debuggers will accept a breakpoint. Another valid string for MP_DEBUG_INITIAL_STOP would be the function_name of the desired initial stopping point in the debugger.

See Note The first executable source line in the main routine.
MP_PMDSUFFIX

(no associated command-line flag)
When using LoadLeveler, a string to be appended to the Partition Manager daemon executable. The PMD service specified in /etc/services is named pmv3. By setting MP_PMDSUFFIX, you can append a string to pmv3. If MP_PMDSUFFIX is set to abc, for example, then the service requested in /etc/services is pmv3abc.

When uing LoadLeveler, the string is appended to the Partition Manager daemon executable name, /etc/pmdv3. By setting MP_PMDSUFFIX, you can append a string to pmdv3. If MP_PMDSUFFIX is set to abc, for example, the Partition Manager daemon that gets run on each node is pmdv3abc.

If the MP_PMD_VERSION enviornment variable (or -pmd_version flag) is set to a value of 2, the appropriate Partition Manager daemon service name and executable name is appended to pmv2 and pmvd2, respectively.

This permits testing of alternate versions of the Partition Manager daemon. Typically, this environment variable is only used under the direction of the IBM Support Center in resolving a PE-related problem.

Any string. None

Return to top of page

Table 6.
POE Environment Variables/Command-Line Flags for Message Passing Interface (MPI)
The Environment Variable Command-Line Flag(s): Set: Possible Values: Default:
MP_BUFFER_MEM

-buffer_mem
To change the maximum size of memory used by the communication subsystem to buffer early arrivals. If you are using this environment variable to change the maximum size of memory used by the communication subsystem while checkpointing a program, be aware that the amount of space needed for the checkpointing files will be increased by the value of MP_BUFFER_MEM. The value of this environment variable can be overridden by using the -buffer_mem flag. nnnnn
nnnK (where:
K = 1024 bytes)
nnM (where:
M = 1024*1024 bytes
2.8MB (IP)
64MB (US)
or
2.8MB (US) if checkpointing
MP_CLOCK_SOURCE

-clock_source
Whether or not to use the SP switch clock as a time source. This may provide better performance for certain applications. The value of this environment variable can be overridden by using the -clock_source flag. AIX

SWITCH

None
MP_CSS_INTERRUPT

-css_interrupt
Whether or not arriving message packets cause interrupts. This may provide better performance for certain applications. The value of this environment variable can be overridden by using the -css_interrupt flag.

Note:

    See also MP_INTRDELAYand MP_POLLING_INTERVAL

yes

no

no
MP_EAGER_LIMIT

-eager_limit
To change the threshold value for message size, above which rendezvous protocol is used. The value of this environment variable can be overridden by using the -eager_limit flag.

To ensure that at least 32 messages can be outstanding between any two tasks, MP_EAGER_LIMIT will be adjusted based on the number of tasks according to the following table (when MP_EAGER_LIMIT and MP_BUFFER_MEM have not been set by the user):

# of Tasks   MP_EAGER_LIMIT
==========   ==============
  1 - 16        4096
 17 - 32        2048
 33 - 64        1024
 65 - 128        512
129 - 256        256
    >256         128
nnnnn
nnnK (where:
K = 1024 bytes)
nnM (where:
M = 1024*1024 bytes)
4096
MP_HINTS_FILTERED

-hints_filtered
Determines whether MPI info objects reject hints (key/value pairs) that are not meaningful to the MPI implementation. In filtered mode, an MPI_INFO_SET call that provides a key/value pair the implementation does not understand will behave as a no-op. A subsequent MPI_INFO_GET call will find that the hint does not exist in the info object.

In unfiltered mode, any key/value pair is stored and may be retrieved. Applications which wish to use MPI info objects to cache and retrieve key/value pairs other than those actaully understood by the MPI implementation must use unfiltered mode. The option has no effect on the way MPI uses the hints it does understand. In unfiltered mode, there is no way for a program to discover which hints are valid to MPI and which are simply being carried as uninterpreted key/value pairs.

Providing an unrecognized hint is not an error in either mode. The value of this environment variable can be overridden by using the -hints_filtered flag.

yes

    unrecgonized hints are filtered.

no

    unrecgonized hints are not filtered.

yes
MP_INTRDELAY

-intrdelay
To tune the delay parameter for how long the MPI pthread stays active after it has processed arriving packets without having to recompile existing applications. The value of this environment variable can be overridden by using the -intrdelay flag. Any integer greater than 0. 1 microsecond
MP_IONODEFILE

-ionodefile
The name of a parallel I/O node file--a text file that lists the nodes that should be handling parallel I/O. This enables you to limit the number of nodes that participate in parallel I/O, guarantee that all I/O operations are performed on the same node, and so on. The value of this environment variable can be overridden by using the -ionodefile flag. Any relative or full path name. All nodes
MP_MAX_TYPEDEPTH

-max_typedepth
To change the maximum depth of user-defined message data types. The value of this environment variable can be overridden by using the -max_typedepth flag. An integer greater than or equal to 1. 5
MP_POLLING_INTERVAL

-polling_interval
To change the polling interval, in microseconds. This environmental variable controls how often the MPI pthread wakes up to check for message arrival. The value of this environment variable can be overridden by using the -polling_interval flag.

Note:

    This may bump a user thread out of a CPU. With MP_CSS_INTERRUPT off by default, this will control how responsive a node is to incoming messages.

An integer between 1 and 2 billion. 400,000 (US)
180,000 (IP)

(the lower the setting, the more intrusive the check)

MP_SHARED_MEMORY

-shared_memory

Whether or not tasks running on the same node should use shared memory (instead of the SP switch) for message passing. This should improve performance for some codes. The value of this environment variable can be overridden using the -shared_memory flag.

Note:

 

There may be some overhead to enabling MP_SHARED_MEMORY. If degraded performance is observed, users may override the default by setting it to no.

yes

no

yes
MP_SINGLE_THREAD

-single_thread

Avoids mutex lock overheads in a single-threaded program. This is an optimization flag.

Note:

    If possible, this environment variable setting should be added to a user's default environment because locks are expensive. It is recommended that MPI use in a threaded application (OpenMP) be arranged so that the calls are always exclusive (done from the "master"). MPI-IO cannot be used if this is set to yes.

yes

    no concurrent MPI calls in the application.

no

    multiple MPI threads are assumed.

no
MP_SYNC_ON_CONNECT Whether or not the internal synchronization of MPI initialization is disabled, thereby reducing the amount of network traffic. yes

no

no (LLNL default)
MP_THREAD_STACKSIZE

-thread_stacksize
The additional stacksize allocated for user programs executing on an MPI service thread. If you allocate insufficient space, the program may encounter a SIGSEGV exception. The value of this environment variable can be overridden by using the -thread_stacksize flag. nnnnn
nnnK (where:
K = 1024 bytes)
nnM (where:
M = 1024*1024 bytes)
 0
MP_TIMEOUT

(no associated command-line flag)
The length of time POE waits when starting a job before deciding that there is some error and the job cannot start.   150 seconds (large job run setting should be greater)
MP_USE_FLOW_CONTROL

-use_flow_control
Applies flow control to MPI programs to limit the maximum number of outstanding messages posted by a sender. The value of this environment variable can be overridden by using the -use_flow_control flag. yes

no

Check the LLNL default settings. If this variable is not listed, then the default is yes on White and Snow, no on Blue and SKY.
MP_WAIT_MODE

-wait_mode
Determines how a thread or task behaves when it is blocked waiting for an incoming message. The value of this environment variable can be overridden by using the -wait_mode flag.

Note:

    Default mode for the signal-handling library is poll for US and sleep for IP (except on SMP nodes). The default for IP on an SMP node is poll if the number of tasks (for this job) on the node does not exceed the number of processors on the node. If the number of tasks exceeds the number of processors, the default is sleep.

poll

yield

sleep

See Note

Return to top of page

Note: Text in italics denotes flags you can use when individually loading nodes.

Table 7.
Other POE Environment Variables/Command-Line Flags
The Environment Variable Command-Line Flag(s): Set: Possible Values: Default:
MP_COREDIR

-coredir
To create a separate directory for each task's core file. The value of this environment variable can be overridden by using the -coredir flag. Any valid directory name. coredir.taskid
MP_COREFILE_FORMAT

-corefile_format
Determines the format of corefiles generated when processes terminate abnormally. The value of this environment variable can be overridden by using the -corefile_format flag. (MOHONK ONLY)

Note:

If not set, POE will generate standard AIX corefiles. If set to the string "STDERR," output will go to standard error. If set to any other string, POE will generate a lightweight corefile (conforming to the Parallel Tool consortium's Standardized Lightweight Corefile Format) for each process in your partition. The string you specify is the name you want to assign to each lightweight corefile. By default, these lightweight corefiles will be saved to subdirectories prefixed by the string coredir and suffixed by the task ID. You can specify a prefix other than the default coredir by setting the MP_COREDIR environment variable.

The string "STDERR" (to specify that the lightweight corefile information should be written to standard error) or any other string (to specify the lightweight corefile name). core.light (White default)

See Note

MP_COREFILE_ SIGTERM

-corefile_sigterm
Determines if POE should generate a core file when a SIGTERM signal is received. Valid values are yes and no. If not set, the default is no.

yes

no

no

MP_DBXPROMPTMOD

(no associated command-line flag)
A modified dbx prompt. The dbx prompt \n(dbx) is used by the pdbx command as an indicator denoting that a dbx subcommand has completed. This environment variable modifies that prompt. Any value assigned to it will have a "." prepended and will then be inserted in the \n(dbx) prompt between the "x" and the ")". This environment variable is useful when the string \n(dbx) is present in the output of the program being debugged. Any string. None
MP_EUIDEVELOP

-euidevelop
Whether or not the message passing interface performs more detailed checking during execution. This additional checking is intended for developing applications and can significantly slow performance. The value of this environment variable can be overridden by using the -euidevelop flag. yes

no

deb (debug)

nor (normal)

min (minimum)

no
MP_FENCE

(no associated command-line flag)
A "fence" character string for separating arguments you want parsed by POE from those you do not. Once set, you can then use the fence_string followed by additional_options on the POE command line. The additional_options will not be parsed by POE. Any string. None
MP_NOARGLIST

(no associated command-line flag)
Whether or not POE ignores the argument list. If set to yes, POE will not attempt to remove POE command-line flags before passing the argument list to the user's program. yes

no

no
MP_PMLIGHTS

-pmlights
The number of lights displayed (per row) on the Program Marker Array. An integer greater than or equal to 0.  0
MP_PRIORITY

(no associated command-line flag)
A dispatch priority class for execution. See IBM Parallel Environment for AIX: Installation for more information on dispatch priority classes. Any of the dispatch priority classes set up by the system administrator in the file /etc/poe.priority None
MP_USRPORT

-usrport
The port ID used by the Partition Manager to connect to the Program Marker Array. Any positive integer less than 32767. Standard TCP/IP practice suggests using ports greater than 5000 and less than 10000. 9999

Return to top of page

Table 8.
AIX Environment Variables
The Environment Variable Command-Line Flag(s): Set: Possible Values: Default:
AIX_PTHREAD_SET_STACKSIZE      
AIXTHREAD_COND_DEBUG
AIXTHREAD_MUTEX_DEBUG
AIXTHREAD_RWLOCK_DEBUG
For debugging deadlocked processes, this variable uses locking structures that enable debugging, making it possible to identify who owns a structure and what the queue is. Locks are viewable with TotalView. on

off

off (LLNL default)
AIXTHREAD_MINKTHREADS=n
AIXTHREAD_MNRATIO=p:k
AIXTHREAD_SLPRATIO=k:p
Control variables for lightweight thread scheduling (when AIXTHREADSCOPE=P)    
AIXTHREAD_SCOPE Control mapping of the threads used in the application to entries in the OS scheduling queue.

Note:

The AIX thread-scheduling options, when in the "lightweight" (=p) user-thread mode (known as m:n threads), don't seem to be designed for scientific use. They batch the threads in a blocked fashion across the kernel threads instead of cyclic mapping. On an SMP node, this results in all the threads going to one CPU.

s (system)

p (process scope)

p (IBM default)

s (recommended setting)

MALLOCDEBUG
MALLOCTYPE
Provides special facilities for debugging memory. Some current limitations for use with parallel jobs.

Note:

Experience with these variables has not been very good. For even a small code, it appears that the memory thrashing/paging caused by enabling these variables causes the code to hang the node. Best experience is with postfree_checking.

With MALLOC_TYPE=debug, set MALLOC_DEBUG to:

align:n
allow_overreading
override_signal_handling
postfree_checking
record_allocations
report_allocations
validate_ptrs 

See Note
MALLOCMULTIHEAP Supports multiple heaps so that locking is rarely needed for MALLOC called in threaded codes (a MALLOC goes to the next available heap). The considersize option compares the size of MALLOC request with what is available on different heaps and tries to select one that won't require sbrk call. heaps:10

considersize

 
PSALLOC To control whether full memory size is allocated before a job begins (and when allocated) or waits until pages are actually used.

Note:

This variable should be used with extreme caution because it may cause a program to be slow in starting (be very large) and may cause system problems if page space gets too low. However, in some cases it may be useful to determine whether program growth is due to untouched, but allocated, pages or to gradual memory leaks.

early

pages allocated when first defined

undefined (or other string)

allocation waits until a page is touched

See Note
RT_GRQ

Makes the thread run queues globally rather than pre-assigned to a specific processor.

Note:

Users may need to evaluate code performance under the default setting. If slowdowns are observed, setting the variable to on may be advisable.

on

off

off
SPINLOOPTIME
YIELDLOOPTIME
Number of times to retry a busy lock before yielding to another processor. A positive integer  
XLSMPOPTS   The following compiler flags:

-schedule

default scheduling algorithm to use for par loops

-parthds

default number of threads to use for par region

-stack

amount of stack space in bytes to reserve for each thread (default is 4194303)

-delays, spins, yields

 

Return to top of page

For more information, please contact Terry Jones.

UCRL-MI-128208 Rev. 1
Last modified September 7, 2006

High Performance Computing at LLNL    Lawrence Livermore National Laboratory