This section details the input file format for Sphinx and the tests included in this distribution. Sphinx determines parameters for a run by reading and parsing an input file. The parameters for the run determine which tests to perform and variables that control the harness operation, such as the number of iterations per repetition. The input file format consists of several different "modes" that determine what run parameters are being specified. The tests to perform are specified in one or more MEASUREMENTS modes. MEASUREMENTS modes consist of a list of tests to perform; test entries include test-specific parameters. The input file format may seem complex due to the flexibility provided by the test harness. However, experienced users can quickly create the desired input file, due in part to the flexibility of the input file format.
In general, Sphinx input file processing is very forgiving - mode or parameter identifiers must contain a string that uniquely determines them but can contain random other characters, allowing input file processing to complete even with most typos. Further, the determining string is not case specific. Each test in Sphinx has several different possible parameters. A sensible default is used if the input file doesn't specify anything for a given mode or parameter. The default values can be overridden for the entire input file or for a specific test.
The Sphinx input file format is fairly free form; an "@" as the first character of a line changes the input mode and modes can occur in any order. Most of the modes are optional; the only mode that is required is at least one MEASUREMENTS mode. The last occurrence of a mode determines the value for that mode except for the COMMENT and MEASUREMENTS modes; for these two modes, multiple occurrences are concatenated to form a single value. The following table describes the Sphinx input file modes (names listed in ALL CAPS for historic reasons).
Sphinx Input File Modes
| MODE | DESCRIPTION | DEFAULT |
|---|---|---|
| COMMENT | any comments that you'd like to include in the input file; this mode can be used to omit test entries without deleting them from the file; not included in output file | NULL |
| USER | a text field with no semantic implications; can be used to provide a short description of the user running the tests; included in output file | NULL |
| MACHINE | a text field with no semantic implications; can be used to provide a short description of the machine on which the tests are run; included in output file | NULL |
| NETWORK | a text field with no semantic implications; can be used to provide a short description of the network on which the tests are run; included in output file | NULL |
| NODE | a text field with no semantic implications; can be used to provide a short description of the nodes of the machine on which the tests are run; included in output file | NULL |
| MPILIB_NAME | a text field with no semantic implications; can be used to provide a short description of the MPI library with which the tests are run; included in output file | NULL |
| OUTFILE | a text field that specifies the output filename | input_filename.out |
| LOGFILE | a text field that specifies the log filename | input_filename.log |
| CORRECT_FOR_OVERHEAD | A yes or no text field that specifies whether test results should be corrected for any test harness overhead incurred in the measurement; overhead is generally a function call but depends on the test | no |
| MEMORY | an integer field that specifies the size in kilobytes of the buffer to allocate in each task for message passing tests; maximum message lengths are a function of this parameter and the test being run; generally maximum message lengths are equal to this parameter or half of it or this parameter divided by the number of tasks | 4096 (i.e. 4MB) |
| MAXREPDEFAULT | an integer field that specifies default limit on the number of timings until a test is declared "UNSETTLED" | 20 |
| MINREPDEFAULT | an integer field that specifies default minimum number of timings to average for a test result | 4 |
| ITERSPERREPDEFAULT | an integer field that specifies default number of iterations per timing of the code being measured | 1 |
| STANDARDDEVIATIONDEFAULT | A double field that specifies the default fraction of mean of timings that standard deviation must be less than for test to be declared settled. Sphinx uses standard deviation which may never achieve this mean, unlike SKaMPI which uses standard error which is guaranteed to achieve the percentage for a sufficiently large number of timings; thus MAXREPDEFAULT is more significant for Sphinx | 0.05 |
| DIMENSIONS_DEFAULT | an integer field that specifies default number of independent variables for a test | 1 |
| VARIATION | a text field that specifies the default independent variable; see below for valid independent variables | NO_VARIATION |
| VARIATION_LIST | a space-delimited text field that specifies the default independent variables | NO_VARIATION for all positions |
| SCALE | a text field that specifies the default scale to use for independent variable; see below for valid scale values | FIXED_LINEAR |
| SCALE_LIST | a space-delimited text field that specifies the default scales | FIXED_LINEAR for all positions |
| MAXSTEPSDEFAULT | an integer field that specifies default limit on the number of values for independent variables | 16 |
| MAXSTEPSDEFAULT_LIST | a space-delimited integers field that specifies default limits on the numbers of values for independent variables | 16 for all positions |
| START | an integer field that specifies default minimum value to use for independent variables; MIN_ARGUMENT has Sphinx use the minimum value semantically allowed for the independent variable (e.g. 1 for number of tasks) | MIN_ARGUMENT |
| START_LIST | a space-delimited integers field that specifies default minimum values to use for independent variables | MIN_ARGUMENT for all positions |
| END | an integer field that specifies default maximum value to use for independent variables; MAX_ARGUMENT has Sphinx use the maximum value semantic ally allowed for the independent variable (e.g. size of MPI_COMM_WORLD for number of tasks) | MAX_ARGUMENT |
| END_LIST | a space-delimited integers field that specifies default maximum values to use for independent variables | MAX_ARGUMENT for all positions |
| STEPWIDTH | a double field that specifies default distance between independent variable values | 1.00 |
| STEPWIDTH_LIST | a space-delimited doubles field that specifies default distances between the values of independent variables | 1.00 for all positions |
| MINDIST | SKaMPI artifact; an integer field that apparently was intended to specify a minimum distance between independent variable values; currently has no effect but may be supported in the future | 1 |
| MINDIST_LIST | a space-delimited integers field that specifies MIN_DIST values | 1 for all positions |
| MAXDIST | SKaMPI artifact; an integer field that apparently was intended to specify a maximum distance between independent variable values; currently has no effect but may be supported in the future (less likely than MINDIST) | 10 for all positions |
| MAXDIST_LIST | a space-delimited integers field that specifies MAX_DIST values | 10 |
| MESSAGELEN | an integer field that specifies default message length in bytes | 256 |
| MAXOVERLAP | an integer field that specifies default maximum iterations of the overlap for loop | 0 |
| THREADS | an integer field that specifies default number of threads | value returned by omp_get_max_threads |
| WORK_FUNCTION_DEFAULT | a text field that specifies the default function used inside OpenMP loops; see below for a list of valid work function values | SIMPLE_WORK |
| WORK_AMOUNT_DEFAULT | an integer field that specifies default duration of function used inside OpenMP loops | 10 |
| SCHEDULE_DEFAULT | a text field that specifies default OpenMP schedule option; see below for a list of valid schedule options | STATIC_SCHED |
| SCHEDULE_CAP_DEFAULT | an integer field that specifies default schedule cap for OpenMP tests | 10 |
| SCHEDULE_CHUNK_DEFAULT | an integer field that specifies default OpenMP schedule chunk size | 1 |
| OVERLAP_FUNCTION | a text field that specifies the default overlap function used in mixed non-blocking MPI/OpenMP tests; see below for valid overlap function values | SEQUENTIAL |
| CHUNKS | an integer field that specifies default number of chunks | 6 |
| MEASUREMENTS | a text field that determines the actual tests run including any default parameter overrides; see below for a description of the format of this field | NULL |
Many of the modes have list variants, as indicated. These allow the specification of different defaults for independent variables X0, X1, X2,... These variants are needed since Sphinx supports multiple independent variables per test, such as varying both the message size and the number of tasks for a MPI collective test. If a test uses more independent variables than specified in a corresponding list, then the non-list default is used for the additional defaults.
The MEASUREMENTS MODE is a structured text field. It describes the tests that will be run for the input file. The format is a series of test descriptions; blank lines between test descriptions are discarded. A test description consists of a name followed by a left curly brace ({) (optionally on a new line), followed by parameters specific to the test; a right curly brace (}) marks the end of the description. Each parameter field of a test description must be on a separate line; the general format of parameter field line is "parameter_name = value". The following table describes the parameter fields of the test description.
Sphinx Test Description Fields
| PARAMETER NAME | DESCRIPTION |
|---|---|
| Type | Type of test; this field determines the actual test run; see below for a description of the different test types available in Sphinx |
| Correct_for_overhead | See CORRECT_FOR_OVERHEAD mode |
| Max_Repetition | See MAXREPDEFAULT mode |
| Min_Repetition | See MINREPDEFAULT mode |
| Standard_Deviation | See STANDARDDEVIATIONDEFAULT mode |
| Dimensions | See DIMENSIONS_DEFAULT mode |
| Variation | See VARIATION_LIST mode |
| Scale | See SCALE_LIST mode |
| Max_Steps | See MAXSTEPSDEFAULT_LIST mode |
| Start | See START_LIST mode |
| End | See END_LIST mode |
| Stepwidth | See STEPWIDTH_LIST mode |
| Min_Distance | See MINDISTANCE_LIST mode |
| Max_Distance | See MAXDISTANCE_LIST mode |
| Default_Message_length | See MESSAGELEN mode |
| Default_Chunks | See CHUNKS mode |
All fields of a test description other than the type field are optional. Test descriptions often consist only of a name, a {, a Type = X line and a }. Properly specified defaults enable this simplicity.
A measurement name is text string without spaces and need not have any relation to the type. This is unfortunate; future versions may augment names in the output file with a type specific string.
If the same name is used for several test descriptions, Sphinx will automatically extend the second and later occurrences with a unique integer. This mechanism ensures that all test descriptions result in a test run.
Sphinx Independent Variable Types
| NAME | SEMANTIC MEANING |
|---|---|
| NO_VARIATION | No independent variable |
| ITERS | Number of iterations per timing |
| NODES | Number of MPI tasks |
| LENGTH | Message length (output format is in bytes) |
| ROOT | Root task (relevant to asynchronous MPI collective tests) |
| ACKER | Task that sends acknowledgement message (relevant to fan-out MPI collective tests) |
| OVERLAP | Computational overlap time |
| SECOND_OVERLAP | Second computational overlap time |
| MASTER_BINDING | CPU to which master (i.e. first/timing) thread is bound |
| SLAVE_BINDING | CPU to which slave (i.e. second) thread is bound |
| THREADS | Number of threads |
| SCHEDULE | OpenMP scheduling option |
| SCHEDULE_CAP | Iterations per OpenMP THREAD(?) |
| SCHEDULE_CHUNK | OpenMP schedule chunk size option |
| WORK_FUNCTION | Function used inside OpenMP loops |
| WORK_AMOUNT | Parameter that determines duration of function used inside OpenMP loops |
| OVERLAP_FUNCTION | Overlap function for mixed non-blocking MPI/OpenMP tests |
| CHUNKS | Number of chunks for master/worker tests (SKaMPI artifact; use at your own risk) |
Some independent variables do not alter anything for some test types. In general, an effort has been made to allow variation of these variables although some may lead to internally detected errors. In any event, care should be taken with independent variable selections to ensure that test descriptions test interesting variations and to ensure that overall run-time is not excessive.
Sphinx Test Types
| NUMBER | DESCRIPTION | TIMING RESULT |
|---|---|---|
| 1 | MPI Ping-pong using MPI_Send and MPI_Recv | Round trip latency |
| 2 | MPI Ping-pong using MPI_Send and MPI_Recv with MPI_ANY_TAG | Round trip latency |
| 3 | MPI Ping-pong using MPI_Send and MPI_Irecv | Round trip latency |
| 4 | MPI Ping-pong using MPI_Send and MPI_Iprobe/MPI_Recv combination | Round trip latency |
| 5 | MPI Ping-pong using MPI_Ssend and MPI_Recv | Round trip latency |
| 6 | MPI Ping-pong using MPI_Isend and MPI_Recv | Round trip latency |
| 7 | MPI Ping-pong using MPI_Bsend and MPI_Recv | Round trip latency |
| 8 | MPI bidirectional communication using MPI_Sendrecv in both tasks | Operation latency |
| 9 | MPI bidirectional communication using MPI_Sendrecv_replace in both tasks | Operation latency |
| 10 | SKaMPI artifact: master/worker with MPI_Waitsome | Not clear (use at own risk) |
| 11 | SKaMPI artifact: master/worker with MPI_Waitany | Not clear (use at own risk) |
| 12 | SKaMPI artifact: master/worker with MPI_Recv with MPI_ANY_SOURCE | Not clear (use at own risk) |
| 13 | SKaMPI artifact: master/worker with MPI_Send | Not clear (use at own risk) |
| 14 | SKaMPI artifact: master/worker with MPI_Ssend | Not clear (use at own risk) |
| 15 | SKaMPI artifact: master/worker with MPI_Isend | Not clear (use at own risk) |
| 16 | SKaMPI artifact: master/worker with MPI_Bsend | Not clear (use at own risk) |
| 17 | Round of MPI_Bcast over all tasks | Lower bound of operation latency |
| 18 | Repeated MPI_Barrier calls (provides a reasonable lower bound of operation latency) | Per task overhead at task zero |
| 19 | Round of MPI_Reduce over all tasks | Lower bound of operation latency |
| 20 | Repeated MPI_Alltoall calls (provides a reasonable lower bound of operation latency) | Per task overhead at task zero |
| 21 | Repeated MPI_Scan calls | Per task overhead at task zero |
| 22 | Repeated MPI_Comm_split (provides a reasonable lower bound of operation latency) (note: "leaks" MPI_Comm results; future changes will eliminate this problem) | Per task overhead at task zero |
| 23 | Repeated memcpy calls | Time per memcpy call |
| 24 | Repeated MPI_Wtime calls | Clock overhead |
| 25 | Repeated MPI_Comm_rank calls | Time per MPI_Comm_rank call |
| 26 | Repeated MPI_Comm_size calls | Time per MPI_Comm_size call |
| 27 | Repeated MPI_Iprobe calls with no message expected | MPI_Iprobe call overhead |
| 28 | Repeated MPI_Buffer_attach and MPI_BUffer_detach | MPI_Buffer_attach/detach call overhead |
| 29 | Empty function call with point to point pattern | Function call overhead |
| 30 | Empty function call with master/worker pattern | Function call overhead |
| 31 | Empty function call with collective pattern | Function call overhead |
| 32 | Empty function call with simple pattern | Function call overhead |
| 33 | Round of MPI_Gather over all tasks | Lower bound of operation latency |
| 34 | Round of MPI_Scatter over all tasks | Lower bound of operation latency |
| 35 | Repeated MPI_Allgather calls (provides a reasonable lower bound of operation latency) | Per task overhead at task zero |
| 36 | Repeated MPI_Allreduce calls (provides a reasonable lower bound of operation latency) | Per task overhead at task zero |
| 37 | Round of MPI_Gatherv over all tasks | Lower bound of operation latency |
| 38 | Round of MPI_Scatterv over all tasks | Lower bound of operation latency |
| 39 | Repeated MPI_Allgatherv calls (provides a reasonable lower bound of operation latency) | Per task overhead at task zero |
| 40 | Repeated MPI_Alltoallv calls (provides a reasonable lower bound of operation latency) | Per task overhead at task zero |
| 41 | Repeated MPI_Reduce_scatter calls | Per task overhead at task zero |
| 42 | Repeated calls to MPI_Bcast, each followed by an MPI_Barrier call | Upper bound of operation latency |
| 43 | Repeated calls to MPI_Bcast | Per task overhead at root task |
| 44 | Round of MPI_Bcast over all tasks (identical to type 17) | Lower bound of operation latency |
| 45 | Repeated calls to MPI_Bcast, each followed by an acknowledgement from every other task to root task | Upper bound of operation latency |
| 46 | Repeated calls to MPI_Bcast, each followed by an acknowledgement from one task to root task; tested over all acknowledgers provides accurate measure of operation latency | Operation latency to acknowledging task |
| 47 | Repeated calls to MPI_Alltoall, each call followed by a barrier implemented with MPI_Send and MPI_Recv operations (provides a reasonable upper bound of operation latency) | Upper bound of operation latency |
| 48 | Repeated calls to MPI_Gather, each call followed by a broadcast implemented with MPI_Send and MPI_Recv operations (provides a reasonable upper bound of operation latency) | Upper bound of operation latency |
| 49 | Repeated calls to MPI_Scatter, each followed by an acknowledgement from one task to root task; tested over all acknowledgers provides accurate measure of operation latency | Operation latency to acknowledging task |
| 50 | Repeated calls to MPI_Allgather, each call followed by a barrier implemented with MPI_Send and MPI_Recv operations (provides a reasonable upper bound of operation latency) | Upper bound of operation latency |
| 51 | Repeated calls to MPI_Allreduce, each call followed by a barrier implemented with MPI_Send and MPI_Recv operations (provides a reasonable upper bound of operation latency) | Upper bound of operation latency |
| 52 | Repeated calls to MPI_Gatherv, each call followed by a broadcast implemented with MPI_Send and MPI_Recv operations (provides a reasonable upper bound of operation latency) | Upper bound of operation latency |
| 53 | Repeated calls to MPI_Scatterv, each followed by an acknowledgement from one task to root task; tested over all acknowledgers provides accurate measure of operation latency | Operation latency to acknowledging task |
| 54 | Repeated calls to MPI_Allgatherv, each call followed by a barrier implemented with MPI_Send and MPI_Recv operations (provides a reasonable upper bound of operation latency) | Upper bound of operation latency |
| 56 | Repeated calls to MPI_Reduce_scatter, each call followed by a barrier implemented with MPI_Send and MPI_Recv operations | Upper bound of operation latency |
| 57 | Repeated calls to MPI_Alltoallv, each call followed by a barrier implemented with MPI_Send and MPI_Recv operations (provides a reasonable upper bound of operation latency) | Upper bound of operation latency |
| 58 | Repeated calls to MPI_Reduce, each call followed by a broadcast implemented with MPI_Send and MPI_Recv operations (provides a reasonable upper bound of operation latency) | Upper bound of operation latency |
| 59 | Function call with for loop of number of tasks iterations in collective pattern | Overhead of function call with for loop |
| 60 | Computation used for non-blocking MPI tests | Time of overlap computation |
| 61 | Overlap of computation with MPI_Isend (not fully tested; use at own risk) | Overlap potential of MPI_Isend |
| 62 | Overlap of computation with MPI_Isend plus overlap of acknowledgement message (not fully tested; use at own risk) | Overlap potential of MPI_Isend |
| 63 | Overlap of computation with MPI_Irecv (not fully tested; use at own risk) | Overlap potential of MPI_Irecv |
| 64 | Repeated MPI_Reduce calls | Per task overhead at root task |
| 65 | Repeated MPI_Gather calls | Per task overhead at root task |
| 66 | Repeated MPI_Gatherv calls | Per task overhead at root task |
| 67 | Repeated MPI_Comm_dup calls (internal difference improves scalability compared to test 69) (note: "leaks" MPI_Comm results; future changes will eliminate this problem) | Per task overhead at task zero |
| 68 | Repeated MPI_Comm_split calls (internal difference improves scalability compared to test 22) (note: "leaks" MPI_Comm results; future changes will eliminate this problem) | Per task overhead at task zero |
| 69 | Repeated MPI_Comm_dup calls (note: "leaks" MPI_Comm results; future changes will eliminate this problem) | Per task overhead at task zero |
| 70 | Repeated calls to MPI_Scan, each followed by an acknowledgement from one task to task zero; tested over all acknowledgers provides accurate measure of operation latency | Operation latency to acknowledging task |
| 71 | Repeated MPI_Scan calls (internal difference improves scalability compared to test 21) | Per task overhead at task zero |
| 101 | Ping-pong using pthread_cond_signal and pthread_cond_wait | "Round trip" latency |
| 102 | Repeated calls to pthread_cond_signal; as many as the slave thread can wait for are caught | Overhead of pthread_cond_signal |
| 103 | Repeated uncaught calls to pthread_cond_signal | Overhead of pthread_cond_signal |
| 104 | Repeated calls to pthread_cond_wait; matching calls to pthread_cond_signal are made "as quickly as possible" | Overhead of pthread_cond_wait |
| 105 | Ping-pong using pthread_mutex_lock and pthread_mutex_unlock (four separate locks) | "Round trip" latency |
| 106 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (four separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 107 | Repeated pthread_mutex_lock and pthread_mutex_unlock calls (one lock) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 108 | Repeated spin on shared variable; measures per thread time slice when bound to the same CPU | Per thread time slice |
| 109 | Chain of pthread_create calls for detached process scope threads | Overhead of pthread_create |
| 110 | Repeated calls to sched_yield (thr_yield for Suns); measures thread context switch time when bound to the same CPU (use with care, depends on OS thread scheduling) | Thread context switch time |
| 111 | Repeated pthread_mutex_lock calls (large array of locks) then repeated calls of pthread_mutex_unlock calls (large array of locks); each set of calls is timed separately | Overhead of pthread_mutex_lock and overhead of pthread_mutex_unlock (separate measurements) |
| 112 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (two separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 113 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (three separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 114 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (five separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 115 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (six separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 116 | Ping-pong using pthread_mutex_lock and pthread_mutex_unlock (array of four locks) | "Round trip" latency |
| 117 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (array of four locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 118 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (large array of locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 119 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (seven separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 120 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (eight separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 121 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (nine separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 122 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (ten separate locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 123 | Repeated interleaved pthread_mutex_lock and pthread_mutex_unlock calls (large array of locks, round robin access order) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 124 | Repeated pthread_mutex_lock and pthread_mutex_unlock calls (one lock, two tight pairs of calls) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 125 | Repeated pthread_mutex_lock and pthread_mutex_unlock calls (one lock, three tight pairs of calls) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 126 | Repeated pthread_mutex_lock and pthread_mutex_unlock calls (one lock, four tight pairs of calls) | Overhead of pthread_mutex_lock and pthread_mutex_unlock |
| 127 | Repeated calls to sched_yield (thr_yield for Suns); measures thread context switch time when bound to the same CPU (uses two "new" threads; can overcome some scheduling quirks) (use with care, depends on OS thread scheduling) | Thread context switch time |
| 128 | Chain of pthread_create calls for detached system scope threads | Overhead of pthread_create |
| 129 | Chain of pthread_create calls for undetached process scope threads | Overhead of pthread_create |
| 130 | Chain of pthread_create calls for undetached system scope threads | Overhead of pthread_create |
| 131 | Function call with for loop of number of tasks iterations in simple pattern | Overhead of function call with for loop |
| 201 | Repeated calls to work function (not fully tested, use at own risk) | Reference measurement for OpenMP parallel construct |
| 202 | Repeated calls to an OpenMP parallel region of work function | Overhead of OpenMP parallel construct |
| 203 | Repeated calls to for loop over work function (not fully tested, use at own risk) | Reference measurement for OpenMP parallel for construct |
| 204 | Repeated calls to an OpenMP parallel for over work function | Overhead of OpenMP parallel for construct |
| 205 | Repeated calls to an OpenMP parallel for with variable chunk sizes over work function | Overhead of OpenMP parallel for with variable chunk sizes construct |
| 206 | Repeated calls to an OpenMP parallel for loop over work function (not fully tested, use at own risk) | Reference measurement for OpenMP ordered construct |
| 207 | Repeated calls to an OpenMP parallel for with ordered clause over work function | Overhead of OpenMP parallel for with ordered clause |
| 208 | Repeated calls to an OpenMP parallel for with ordered work function calls | Overhead of OpenMP ordered construct |
| 209 | Repeated calls to for loop over work function (not fully tested, use at own risk) | Reference measurement for OpenMP single and barrier constructs |
| 210 | Repeated calls to for loop over work function inside OpenMP single construct (not fully tested, use at own risk) | Overhead of OpenMP single construct |
| 211 | Repeated calls to for loop over work function following an OpenMP barrier construct (not fully tested, use at own risk) | Overhead of OpenMP barrier construct |
| 212 | Repeated calls to for loop over work function, results summed (not fully tested, use at own risk) | Reference measurement for OpenMP reduction construct |
| 213 | Repeated calls to an OpenMP parallel for loop with reduction clause over work function (not fully tested, use at own risk) | Overhead of OpenMP reduction construct |
| 214 | Repeated calls to integer increment and work function (not fully tested, use at own risk) | Overhead of OpenMP single construct |
| 215 | Repeated calls to for loop over integer increment inside an OpenMP atomic construct and work function (not fully tested, use at own risk) | Overhead of OpenMP barrier construct |
| 301 | Repeated calls to a mixed OpenMP/MPI barrier followed by work function call (provides a reasonable lower bound of operation latency) | OpenMP-test-style overhead of mixed OpenMP/MPI barrier |
| 302 | Repeated mixed OpenMP/MPI barrier calls (provides a reasonable lower bound of operation latency) | Per task overhead at task zero |
| 303 | Repeated calls to a mixed OpenMP/MPI reduce across all threads in all tasks followed by work function call (provides a reasonable lower bound of operation latency) | OpenMP-test-style overhead of mixed OpenMP/MPI all reduce |
| 304 | Repeated calls to mixed OpenMP/MPI reduce across all threads in all tasks (provides a reasonable lower bound of operation latency) | Per task overhead at task zero |
| 305 | Repeated calls to mixed OpenMP/MPI reduce across all threads in all tasks (essentially) followed by a mixed OpenMP/generic MPI barrier | Upper bound of operation latency |
| 306 | Computation used for non-blocking MPI mixed with OpenMP tests | Time of threaded overlap computation |
| 307 | Overlap of OpenMP threaded computation with MPI_Isend (not fully tested; use at own risk) | Overlap potential of MPI_Isend |
| 308 | Overlap of OpenMP threaded computation with MPI_Isend plus overlap of acknowledgement message (not fully tested; use at own risk) | Overlap potential of MPI_Isend |
| 309 | Overlap of OpenMP threaded computation with MPI_Irecv (not fully tested; use at own risk) | Overlap potential of MPI_Irecv |
The preceding table provides only a brief description of the tests and their results. The referenced papers provide further detail. Of course, a complete understanding can only result from careful consideration of the code. For details of the corrections used when the correct for overhead mode is used, consult the code.
Sphinx Independent Variable Scales
| NAME | DESCRIPTION |
|---|---|
| FIXED_LINEAR | Fixed linear scale; use up to MAXTSTEPS values exactly STEPWIDTH apart |
| DYNAMIC_LINEAR | Dynamic linear scale; use values exactly STEPWIDTH apart, then fill in into either exactly MAXSTEPS values or no "holes" |
| FIXED_LOGARITHMIC | Fixed logarithmic scale; use up to MAXTSTEPS values "logarithmically" exactly STEPWIDTH apart |
| DYNAMIC_LOGARITHMIC | Dynamic linear scale; use values "logarithmically" exactly STEPWIDTH apart, then fill in into either exactly MAXSTEPS values or no "holes" |
Linear scales are reasonably intuitive; a STEPWIDTH of the square root of 2 will result in doubling the previous value with a fixed logarithmic scale. The default stepwidth actually varies with the scale since a STEPWIDTH of 1.00 would not result in any variation with logarithmic scales. The default is the ssquare root of two with logarithmic scales.
Sphinx Work Functions
| NAME | DESCRIPTION |
|---|---|
| SIMPLE_WORK | A simple for loop of WORK_AMOUNT iterations, each iteration has a single FMA plus a few branch statements based on mod tests and possibly an integer shift |
| BORS_WORK | Complex set of array operations; duration per WORK_AMOUNT unit is relatively long |
| SPIN_TIMED_WORK | Loop over checks to see if work function has lasted WORK_AMOUNT nanoseconds |
| SLEEP_TIMED_WORK | Loop over checks to see if work function has lasted WORK_AMOUNT nanoseconds followed by sleep and usleep of remaining time |
The effect of varying work functions should be limited to cache effects. A future paper will present results addressing the validity of this expectation.
Sphinx Schedule Values
| VALUE | DESCRIPTION |
|---|---|
| STATIC_SCHED | static |
| DYNAMIC_SCHED | static |
| GUIDED_SCHED | static |
The standard OpenMP names for the scheduling options are actually sufficient since Sphinx uses a non-case specific minimum string mechanism to determine the value. Support for the OpenMP runtime schedule option may be added in the future.
Sphinx Overlap Function Values
| VALUE | DESCRIPTION |
|---|---|
| SEQUENTIAL | sequential work function (i.e. not in an OpenMP parallel region) |
| PARALLEL | work function inside OpenMP parallel region |
| PARALLEL_FOR | work function inside OpenMP parallel for construct |
| PARALLEL_FOR_CHUNKS | work function inside OpenMP parallel for construct with variable chunks |
The log mechanism inherited from SKaMPI allows multiple runs of the same input file to run the full set of test descriptions to completion. If the log file contains and end of run message, then the log file and output file are moved to file names extended with an integer and a new run of the full set is started. Sphinx includes corrections to some bugs in this mechanism. These corrections ensure that each test description is run to completion exactly once, regardless of its name or the status of partial runs.