LLNL

EZOUTPUT

EZOUTPUT

Introduction

This manual provides a basic guide for generating output and transferring files on LC computers. It outlines naming and locating tips for local network printers and introduces local tools (such as LISTPQ) for discovering and taking advantage of LC's standardized print-queue names. It explains how to convert older graphics file formats into PostScript for printing with LP, LPR, or APR, and explains how to print files using LP, LPR, or APR, whichever is most appropriate for your situation. EZOUTPUT introduces the utilities SCP, FTP, SFTP and NFT, and Hopper. LC's special open/secure File Interchange Service (FIS) is also discussed.

Detailed technical instructions for the tools and techniques introduced in EZOUTPUT are available for FTP (with SFTP), NFT, HTAR, Hopper, and FIS. You may also want to consult the basic guide for using local directories and for general file-handling software at LC, called EZFILES. If your concern is efficiently managing I/O for science application codes as they run in LC's parallel computing environments, consult the separate I/O Guide for LC.

For help, contact the LC Hotline at 925-422-4531 or via e-mail (OCF: lc-hotline@llnl.gov, SCF: lc-hotline@pop.llnl.gov).


Local Printers

Print Queue Names

Each printer is associated with one basic print queue whose name always has the form pnnn, where nnn is a three digit number that uniquely identifies that printer. Examples are p005 and p123.

Most printers also have additional print queue names that derive from the basic name and that invoke special processing features. The most common feature names include:

Each printer also has one print queue name that reveals its physical location by prefixing the basic name with a building identifier (uppercase B, e.g. B453) and a room identifier (uppercase R, e.g. R1000). Examples of location names are B453_R3040_p294 and B453_R3152_282. Color printers have the extra prefix C_ inserted between the room number and the basic name in their location name (e.g., B453_R3040_C_p294). Note: This printer identification typography is typical for the UNIX or Linux command-line interface. Windows and Mac desktop systems will use slightly different typography.

A single physical LC printer therefore typically has several associated print-queue names, for example:

p302
B453_R3152_C_p302
p302_dup
p302_trans
tsfptr3


Locating Printers

On LC machines, the command lpstat -a will list the names of all active print queues. You can use this command, in combination with the grep command, to find the names of printers in your building, For example, the command

lpstat -a | grep -i b217

returns the following list of 6 print queues for printers in B217:

B217_R144_C_p102 accepting requests since Fri May 23 13:39:27 2008
B217_R144_p222 accepting requests since Fri May 23 13:27:33 2008
B217_R184_p035 accepting requests since Fri May 23 13:03:56 2008
b217c accepting requests since Fri May 23 13:39:27 2008
b217c-d accepting requests since Fri May 23 13:39:27 2008
b217c-t accepting requests since Fri May 23 13:39:27 2008

More details about and options for lpstat are available on the lpstat man page.

Environment Variables for Default Printers

On machines with LPR or APR (see Printing Files for information about LPR and APR), you can set the UNIX environment variable PRINTER (always uppercase) to specify the name of a default printer. Then, if you run lpr or apr without the -P option, output will go to the default printer you specified.

Likewise, on machines with LP, you can set the UNIX environment variable LPDEST (uppercase) to specify the name of a default printer. Then, if you run lp without the -d option, output will go to the default printer you specified.

To set either of these environment variables for your current login session while you run the C shell on any UNIX machine, type

setenv PRINTER printername

or

setenv LPDEST printername

To automatically set either environment variable whenever you log in, include the appropriate one of these lines in the .cshrc file that resides in your home directory.

To report the current value of an environment variable (such as PRINTER) while in the C shell, use PRINTENV:

printenv PRINTER

Printing Files

Some graphics files and other file formats must be converted to a text or PostScript file for printing. A few conversion tools are discussed in "Converting File Formats for Printing".

Output Software Compared

The two standard UNIX tools for printing files are LP and LPR. Both have their own distinct related routines for monitoring print queues and for deleting pending print jobs. However, because neither LP nor LPR meets local needs for automatically labeling each page of printed output in a secure environment, a locally enhanced tool called APR adds this feature (among others) on secure systems at LC.

Details about the syntax and options for LP and LPR are available. Type

man lp

or

man lpr

The table below summarizes some user-relevant differences between LP, LPR, and APR. (See Special Cases for details on using APR in some unusual ways and for invoking extra options to control the details of output format.)

LP LPR APR
Related program to show print queue lpstat lpq lpq
Related program to delete print job cancel lprm lprm
Environment variable checked for default printer LPDEST PRINTER PRINTER
Supports security page labels no no yes
Accepts PR preprocessing yes yes yes
Typical command lines lp -d printername filename
lpstat
lpr -P printername filename
lpq -P printername
apr -P printername -S level filename
lpq -P printername
Specify printer -d printername -P printername -P printername
Label banner page -t string -J string -- -J string
Make num copies -n num -# num -# num
Add security labels none none -S level
Control output format -o feature -o feature none


Security Levels for APR

APR is the only output routine that labels the top and bottom of every printed page with a security-level banner of the form

--- XXX --- Mon May 11 10:30:45 PDT 2008 --- XXX ---

which includes the date printed and a standard security label XXX. You specify the security level of the file(s) you print by using APR's -S level option, which has no default value so you must explicitly state the level, and APR assigns a corresponding security label for XXX according to this chart:

Security Level Specified Label Output
s, S, srd, 5 Secret Restricted Data
c, C, crd, 4 Confidential Restricted Data
u, U, uncl, UNCL, 1 Unclassified

If you direct APR's output to a file (using the -p option), each PostScript page in that file also bears the security-level banner that you specified with -S level.


Special Cases

Printing to a File

On a secure machine offering APR, you have the option to "print to a file." This lets you save a document as a PostScript file with your specified security banner at the top and bottom of each page. Use the lowercase -p option to name the output file:

apr -p myout.ps -S s project37

You can then preview this output file with PostScript tools before printing it with LPR (because it is already security-labeled) or storing it. Neither LP nor LPR offers a corresponding option to "print to a file."


More Output Control

APR Features

On a secure machine offering APR, you can specify additional output features by using any of these additional APR options:

APR Option Output Feature
-L rotate output pages to landscape (132- characters/row)
-2 print output in two columns
-W truncate lines too wide for the page (default wraps them onto next line)

On LC SCF machines, you can specify which text-to-Postscript converter APR will invoke privately when you run it. Set the environment variable XENSCRIPT to the path name of the converter that you prefer, where the current choices are:

/usr/bin/enscript
/usr/bin/nenscript [default]

Preprocessors

A variety of standard preprocessing tools can be used to control the output characteristics of a text file that you then pipe ( | ) to LP, LPR, or APR for printing. Among the most commonly needed preprocessing sequences are:

Command Line Output Feature
nl -ba file | lp ... number each line
pr -d file | lp ... double space all lines
pr -l n file | lp ... set the lines/page to n

Test these locally on a small file when you first use them, since implementation details and command interactions often vary by vendor, version, and platform. Note: Preprocessed files can be piped into APR by specifying "-" as the file name. For example,

pr –d file | apr –P myprinter –Su -


Using -o Settings

On LC OCF machines, you can activate latent output-control features of the Common UNIX Print System (CUPS) by specifying them with the -o option on the execute lines of LP, LPR, and LPOPTIONS. You can control output with -o in three (related) ways:

The examples above reveal that each use of -o takes a single argument (although it may in turn have several subarguments), so you must overtly invoke -o several times to specify several different output features on one execute line. Additional information and documentation about CUPS is available on LC machines at http://localhost:631/help or on the Internet at http://www.cups.org/.

Arguments for -o include:

cpi=n
sets horizontal characters per inch (default is 10).
lpi=n
sets vertical lines per inch (default is 6).
fitplot
scales the print file to fit on the page.
landscape
rotates the text 90 degrees [see also sides below].
media=size
specifies a paper size (choices are a4, letter [default], and legal).
number-up=count
specifies 1, 2, 4, 6, 9, or 16 page images per sheet.
page-left=n
sets the left margin at n points (where each point is 1/72 inch or 0.35 mm).
page-right=n
sets the right margin at n points.
page-top=n
sets the top margin at n points.
page-bottom=n
sets the bottom margin at n points.
scaling=number
scales image files to use up to number percent of the page. Values greater than 100 cause the image files to be printed across multiple pages.
sides=duplexflavor
activates duplex (two-sided) printing in either of two ways:
   two-sided-short-edge [duplex landscape]
   two-sided-long-edge [duplex portrait]

Converting File Formats

Print File Conversions

Some graphics file formats may need to be converted into another file format before they can be printed, including:

After conversion, you can send the file to the current network printers (with LP, LPR, or APR, depending on where the files reside).

Software tools exist to convert each file format, including text to PostScript using ENSCRIPT. For example, ImageMagick is a popular open-source tool that converts most image files.

CRAY File Conversions

Using data files from CRAY or CDC machines on LC's IBM computers often requires that you run a conversion program that understands the older file formats. TRANS on LC's IBMs is one such conversion utility, while LFT is another. This section explains how to use TRANS (on IBMs) appropriately, and it refers you to LFT instructions elsewhere.

TRANS

In most cases, TRANS can convert text formats and it can also convert numeric values to the IEEE format used on the IBM machines as 8-byte real and integer values. (Note that TRANS cannot deal with real or integer values that do not occupy full words on the older machines. It cannot deal, for example, with byte arrays or bytes packed into structures.) Simple programs can be written to then further convert data to other formats (such as 4-byte integers or reals) as needed.

In many cases, TRANS can figure out what the format of an older file is on its own, but in some cases you must supply extra hints by using TRANS options. The chart below suggests for different source machines and operating systems which TRANS execute line and special options are most likely to work best for specific types of files. Once a file has been converted to IBM format by using TRANS, you can move it to other machines with FTP, and FTP will automatically handle any futher conversions needed.

For a complete list of all TRANS options (many now obsolete), see the MAN page for TRANS on any LC IBM machine or visit the TRANS Web page.

For files from a CRAY running LTSS, NLTSS, or CTSS

Text editor output,
Fortran sequential formatted write (and namelist and list directed),
Fortran sequential unformatted write,
Fortran direct access formatted write,
Fortran direct access unformatted write,
Fortran two-argument buffer out to a file connected for unformatted I/O,
..........use this TRANS line:
trans -i infile -o outfile

Fortran two-argument buffer out to a file connected for formatted I/O,
Fortran three-argument buffer out,
..........use this TRANS line:
trans -i infile -o outfile abs

For files from a CRAY running UNICOS

Text editor output,
Fortran sequential formatted write (and namelist and list directed),
Fortran direct access formatted write,
Fortran direct access unformatted write of character data,
..........use this TRANS line:
[NO translation needed here]

Fortran sequential unformatted write of numeric data,
Fortran two-argument buffer out to file connected for unformatted I/O,
..........use this TRANS line:
trans -i infile -o outfile

Fortran direct access unformatted write of numeric data,
Fortran two-argument buffer out to a file connected for formatted I/O,
Fortran three-argument buffer out,
..........use this TRANS line:
trans -i infile -o outfile abs

Fortran sequential unformatted write of character data,
..........use this TRANS line:
trans -i infile,cos -o outfile

For files from a CDC 7600 or CDC 6600

Most text editor output,
Fortran sequential formatted write (and namelist and list directed),
Fortran sequential unformatted write,
Fortran direct access formatted write,
Fortran two-argument buffer out,
..........use this TRANS line:
trans -i infile,7600 -o outfile

TRIX Red output,
..........use this TRANS line:
trans -i infile,7600,red -o outfile

Fortran direct access unformatted write,
Fortran three-argument buffer out,
..........use this TRANS line:
trans -i infile,7600 -o outfile abs

LFT

Unlike IBM's TRANS, LFT ("list file types") was written at LC to help manage CRAY and CDC 7600 legacy files. LFT (on all LC production machines) reports and, if you request, converts to UNIX format all specified CRAY and CDC 7600 text files (but leaves all other formats unchanged). See the "Tools for Obsolete File Types" section of EZFILES for details on LFT and the related tool LIB76.


Transferring Files Between Machines

File-Transfer Utilities

FTP, SCP

Both FTP and SCP use the TCP/IP network protocols to manage file transfers, and both let you prevent accidental overwriting of existing files (only files with write permission for the owner are overwritten). Neither offers queued transfer of multiple files. Details are available in the FTP Usage Guide. SFTP (secure FTP) behaves much like standard FTP but it encrypts the files that it transfers.

NFT

NFT was locally developed and runs only on LC machines where its clients have been specifically installed. Like FTP clients, NFT clients use standard FTP daemons (servers) on remote machines to handle their file transfers. To improve reliability for session tracking and command persistence, however, NFT also employes a third machine running special support software to manage NFT jobs. Unlike the other alternatives, NFT offers the convenience of preauthenticated (passwordless) file transfer, and it can be used to transfer files to storage by default and between two remote hosts without logging on to either one. Full details on NFT's features appear in the NFT Reference Manual.

HTAR

Unlike FTP, NFT, or SCP, HTAR is not a general-purpose file transfer program. It was developed by LC and installed on LC production machines specifically to transfer member files into or out of remote TAR-format archive files. The default location for those archives is HPSS. HTAR is fast and efficient, even when there are thousands of member files. Consult the HTAR Reference Manual for usage suggestions, annotated examples, technical tips, and full option details.

SFTP

Unlike FTP, NFT or SCP (but somewhat like HTAR), Secure FTP (SFTP) is not a general-purpose, general-destination file transfer program. Standard FTP clients do not encrypt the data that they send to remote hosts. The SFTP clients behave generally like FTP, but it does encrypt all of the files that it sends for greater protection against third-party eavesdropping. See the Secure FTP (SFTP) section of LC's FTP Reference Manual for a more complete discussion of SFTP features and limitations, for a comparison of FTP and SFTP user dialogs, for suggested ways to work around the missing SFTP control options, and for information on using "DSA keys" instead of your one-time password (OTP) to authenticate SFTP sessions (to FIS).

Hopper

Hopper offers a graphical interface to storage and other LC resources, including support for simple drag-and-drop file-transfer services using FTP, NFT, HSI, HTAR, etc. By invoking Hopper, you can do many file-management tasks graphically, including:

More general background information on Hopper is available at the Hopper Web site. See "Getting Started" for instructions on how to download Hopper to your local desktop machine.


Open/Secure File Interchange Service (FIS)

FIS Overview

Although LC's unclassified (Collaboration Zone/Restricted Zone or CZ/RZ) and secure networks are physically isolated from each other, you can transfer files between them. NFT and SCP will not transfer files to or from FIS. SFTP ("secure FTP") clients on open-network LC machines can transfer files to a special SFTP server on the open-network FIS node. The interaction is much like using standard FTP except that the transferred data is encrypted. But many fewer options are available to control the transfer. SFTP is not available on the secure network. Hopper offers a graphical user interface for FTP transfers to or from FIS on LC production machines.

Two implementations of FIS are available. One implementation (called FastFIS) uses an electronic One Way Link (OWL), and the other (called TapeFIS) uses tape technology.  The OWL implementation is unidirectional and is used for transfers from the OCF CZ and RZ to the SCF. The tape technology FIS is bidirectional and is primarily used for transfers from the SCF to the OCF.


Authorization

To use FIS you must already have an account and valid password for at least one open and one secure machine. You must also complete the FIS form. Any user can receive authorization to move files from the open to the secure network. See the instructions on the form and in the FIS Reference Manual for full details. Once you are authorized, your authenticator-generated one-time password (OTP, used for any production open computing cluster) will allow access to the open FastFIS node as well.


File-Transfer Process

The File Interchange Service (FIS) Manual provides usage details for FastFIS (and TapeFIS).