TotalView 4.0.0-1-LLNL

Release Features

March 28, 2000

TotalView is a product of Etnus Inc.

This documents the release of TotalView 4.0.0-1-LLNL installed into totalview and totalviewcli on blue/s/k/y/snow and compass/forest on March 29, 2000.

Below are the features added to create TotalView 4.0.0-1-LLNL.

§ Etnus Release Features in 4.0
§ Support of Gang Scheduling on IBM
§ Performance Monitoring available on IBM
§ Condensing the root window for processes in the same state
§ Condensing the variable window for array values with the same value
§ Removed the FIND command
§ Better function specification in thread pane of process window
§ Added CLI command dwhatsnew
§ Added CLI command dformat
§ Added CLI command dsave/dcompare
§ Added save/load action points options to CLI command dactions
§ Printing array slices in CLI
§ Added CLI variable DISPLAY_FORMAT to control default display format
§
§ Previous LLNL/LANL Enhancements to TotalView
§ TotalView V4.1.0.0-0-LLNL Release Features

Documentation: TotalView User's Guide (HTML and PDF) and TotalView CLI Guide (HTML and PDF).

Tutorial: TotalView Debugger by Blaise Barney

TotalView Quick Reference Page

TotalView CLI Summary Sheet

For problems/questions, send e-mail to Bor Chan, Karen Warren, Rich Zwakenberg (LLNL) or Laurie McGavran (LANL).













Support of Gang Scheduling on IBM

When using TotalView on the IBM to debug a job, the job must be non-preemptable. This can be done using the llexpress command (see man llexpress) or by using the following TotalView option. Note that initially this will only need to be done for batch jobs, not interactive jobs; however, this may change in the future.

Batch jobs that execute totalview in their script must be submitted to psub using the -np option to run in the non-preemptable mode, or must use the method described above to temporarily make the job non-preemptable.

Jobs can be made non-preemptable for a maximum of two hours, possibly across multiple debug sessions (e.g., one batch job may have two 60-minute debug sessions, or four 30-minute debug sessions, etc.). A database is maintained of a job's non-preemptable time. A system daemon monitors this and returns jobs to a preemptable state after their two-hour time limit is reached.
















Performance Monitoring available on IBM

TotalView allows Performance Monitoring on the IBM with the IBM Performance Monitoring Application Programming Interface (PMAPI).

How to run the performance monitors from TotalView

Start up TotalView on your program. If it is a parallel job, make sure you are beyond the execution of poe and ready to begin debugging your program. Before you start your program, or at any breakpoint within your program, select the command Open Performance Monitor Display, which will open the Performance Monitor Display Window (see picture below).

Indicate what kind of events to count

The menu command in the Performance Monitor Display Window allows for the selection of events to be asssigned to counters: Set Counter 1, Set Counter 2, etc. Only one event can be counted on a counter at a time. The user selects from the counters subcommand the allowable event for that counter. In this way, the user can select events for as many counters as allowed without the problem of selecting more than one event on a counter or selecting an event not available on that counter. You need not place an event on each counter. Some events take more than one counter, so TotalView will not allow you to use the second counter when that event is chosen.

The events to be counted will be the same in all processes, threads and monitoring ranges. We are not allowing different events to be set for counting in different processes, threads or in different monitoring ranges. The user can respecify what events to be counted at any time when all processes and threads are halted.

Starting the performance monitor counters

After the events to be counted have been set, the performance monitor counters are started at the beginning of a monitoring range (see Defining monitoring ranges below).

Display of the performance monitoring results

The Performance Monitor Display Window will display the performance monitoring results when the code reaches any stop point; via breakpoint, is halted, etc.

Defining monitoring ranges

Monitoring ranges are set up in either of two ways:

  • At subsequent breakpoints (or any stop points), selecting the Activate Counters and Stop Counters menu commands respectively from the Performance Monitor Display Window, or
  • Setting count/nocnt action points in your program.

    Count/Nocnt Action Points

    We have added a new kind of action point, Start Performance Monitor Counting, displayed as count, and Stop Performance Monitor Counting, displayed as nocnt. Press the right mouse button on the line number where you want the performance monitoring count/nocnt action point. An action point window will pop up. If you would like monitoring to start at this point, select the count action point. If you would like monitoring to stop at this point, select the nocnt action point. Then press OK or hit return.

    The count/nocnt action points have the same options as do the other action points (breakpoint, barrierpoint, eval point), that being enable/disable, share or don't share with other processes, and are saved when action points are saved.

    The count/nocnt actions are defined by the dynamic scope of the process. Once you begin executing your program the counters will only be active in between the count/nocnt action points. Once a count action is encounterd, subsequent count actions are redundant. The first nocnt action encountered stops the monitoring.










    Condensing the root window for processes in the same state

    The root window is used to display the processes (and threads) of the code being debugged. It also displays the states of each process (and thread), like running (R), trapped (T), at a breakpoint (B1), etc. The new feature is if any two contiguous processes in the list are in the same state, the processes will be collapsed into one in the list. The list can be expanded out by selecting the right facing triangle to the left of the collapsed process list (meaning open up and display the entire collapsed list).

    The first window below is an example of the new root window with 12 processes. The second window below is an example of the way the root window appeared before. Selecting the triangle oposite the collapsed processes will cause TotalView to expand out the process list and it will look like the previously expanded process window.











    Condensing the variable window for array values with the same value

    The variable window is used to display the values of variables for the code being debugged. When TotalView displays arrays, it displays them in a linear list. This list can be quite long for large arrays. If many of the contiguous values of the array are the same, it makes sense to be able to display these values with one line.

    Two new commands have been added to the variable window commands. They are Condense Window and Uncondense Window. Condense window means to display contiguous array values with the same value on one line. Uncondense window means to display contiguous array values with the same value on two lines (the normal default).

    Displayed array subscripted arrays or sub members of a structure can not be condensed.

    The first window below is an example of an uncondensed array valued variable window (the default). The second window below shows the array valued window after the Condense Window menu command has been selected.









    Removed the FIND command

    The Variable Window Command find has been removed. All functions of the find command are now available from the variable window commands in the following ways.









    Better function specification in thread pane of process window

    The process/source window displays the threads for a process in the lower left pane. Additionally the threads can be displayed by selecting the triangle of the process in the root window (meaning let me see what threads are in what state in this process). The change has been to get a better user description of what function the user is in within a thread. For example, within a thread the user might call user_called_lib_function which in turn calls a lot of other library functions, such that the code is really in who knows what. Instead of saying the thread is in who knows what, TotalView now states the thread is via user_called_lib_function, hopefully something more meaningful to the user.

    Below are two examples of the root window for a process using explicit thread calls: the first displays the threads for a process not using this feature and the second root window displays the threads for the same process using this new display feature.









    Added CLI command dwhatsnew

    The CLI command dwhatsnew prints out the new features available in the current release of TotalView. The dwhatsnew command is the CLI counterpart of the root window What's New command.















    Added CLI command dformat

    The CLI command dformat changes the default printing format for subsequently displayed variables. This command is similar to the variable window Format command. The primary difference is that the window Format command only applies to the specific window. The CLI dformat command is in effect till a subsequent dformat command. To get back to the default printing format, you must execute the CLI command dformat native.

    The dformat command options are:

    The dump option prints the value in octal, hex, and string. The nn in the float and scientific options states precision length.

    Execute help dformat for this information.

    Example:















    Added CLI command dsave/dcompare

    Two new commands, dsave and dcompare, have been added to the CLI to give the same capability as the Save/Compare commands in the window version.

    dsave will save the variable data in the file specified by the filename. If no filename is specified, the variable's data will be saved in the file "function.variable-name".

    dcompare will compare the variable with the data in the file specified by the filename. This command is used after a variable has been saved with a dsave command. If no filename is specified, the variable will be compared with the data in the file "function.variable-name".

    Example:
    
    	d1.<> dsave a filez
    
    	d1.<> as a(3,4) 7
    
    	d1.<> dcompare a filez
    	33c33
    	< (3,4)   3.464102e+00 
    	---
    	> (3,4)   7.000000e+00 
    















    Added save/load action points options to CLI command dactions

    The CLI command dactions lists action points for the process in the current focus. For more information execute help dactions.

    The dactions command options save and load save and load action points to a TotalView defined debug file. These are the counter parts to the window commands STOP/BARR/EVAL/ELOG->Save All Action Points and STOP/BARR/EVAL/ELOG->Load All Action Points.

    Example:















    Printing array slices in CLI

    The TotalView CLI now allows printing slices of arrays.

    Example:

















    Added CLI variable DISPLAY_FORMAT to control default display format

    TotalView's normal display format is to display real variables in floating point notation. Our default here at LLNL is to display real variables in scientific notation. Via the new variable DISPLAY_FORMAT, the CLI user can now set the default display format for real variables to display as float or scientific and set the precision to any length they want. This is the CLI equivalent to the root window Setting the Display Precision Globally command.

    The CLI command dset is used to set the DISPLAY_FORMAT variable.

    where nn is the precision length.














    LLNL Disclaimers

    Last revised March 28, 2000