LLNL

File Interchange Service Manual

File Interchange Service (FIS) Manual

Introduction

The Livermore Computing (LC) file interchange service (FIS) allows LC users to transfer files between the unclassified Open Computing Facility (OCF)—both the Collaboration Zone (CZ) and the Restricted Zone (RZ)—and the classified Secure Computing Facility (SCF). This document describes how FIS works and how to use it effectively.

Two implementations of FIS are available. One implementation uses an electronic One Way Link (OWL), and the other uses tape technology.  The OWL implementation is unidirectional and is used for transfers from the OCF CZ and RZ to the SCF. The tape technology FIS is bidirectional and is primarily used for transfers from the SCF to the OCF. On the iSNSI network, an OWL implementatin is available (as of February 2015) for transfer from the OCF to the iSNSI, as well as a tape implementation for transfer between the OCF and the Pinot cluster .

Note: Throughout this document, the OWL implementation of FIS is referred to as FastFIS (server is fastfis.llnl.gov), and the tape implementation of FIS is referred to as TapeFIS (server is tapefis.llnl.gov). Also, the server name fis.llnl.gov is an alias for fastfis.llnl.gov.

For help, contact the LC Hotline at 925-422-4531 or via e-mail (OCF: lc-hotline@llnl.gov, SCF: lc-hotline@pop.llnl.gov).


Overview

There are three FIS servers, one in each environment (i.e., CZ, RZ, and SCF). For the user, the each server represents the place to which files are submitted for transfer and the place from which to retrieve files that have been transferred. FIS is also available to move files to and from the OCF CZ and RZ to Pinot on the iSNSI network.

To utilize the OWL FIS, users should connect to fastfis.llnl.gov. To utilize the tape FIS system, users should connect to tapefis.llnl.gov.

FastFIS service is unidirectional from the OCF to the SCF; there is no SCF to OCF FastFIS. Because files will be queued for transfer within 5 minutes of submission to FastFIS, no user notification will be sent when the files reach their destination. TapeFIS service is bidirectional, and although the transfer mechanism is the same in both directions, the user interface and operational aspects are different for SCF to OCF transfers because of the asymmetric need (met by having your organization's Derivative Classifiers [DCs] inspect the transferred files) to verify that only unclassified content moves from the secure to the open network. For users on the iSNSI network, both dedicated TapeFIS and OWL FIS are available. See the section FIS on the iSNSI Network for more information.

Users cannot have accounts on both CZFastFIS and RZFastFIS. To protect security, general login capability is disabled. To connect to a FIS server, an FTP or SFTP/SCP client must be executable on your local machine.

The table below provides connection guidelines. Use TapeFIS only for very large one-time data transfers from the OCF to the SCF or when moving files from the SCF to the OCF. For details about file transfers from the SCF to the OCF using TapeFIS, see DC Support for Secure-to-Open Transfers.

Connecting to FastFIS
Transfer Originating From Server Name Alias
OCF CZ fastfis.llnl.gov fastfis
fis
OCF RZ rzfastfis.llnl.gov rzfastfis
rzfis
SCF retrieval fastfis.llnl.gov fastfis
fis
Connecting to TapeFIS *
Transfer Originating From Server Name Alias
OCF CZ tapefis.llnl.gov tapefis
OCF RZ rztapefis.llnl.gov rztapefis
SCF retrieval tapefis.llnl.gov tapefis
* Use TapeFIS only for very large one-time data transfers from
the OCF to the SCF or when moving files from the SCF to the OCF.

The diagram below identifies the FIS servers and the authentication and login methods on the OCF.

Which FIS server to use on the OCF depends upon whether a user is CZ-only, RZ-only, or CZ/RZ.

CZ-only Users

RZ-only Users or CZ/RZ Users

Usage Guidelines

  1. You cannot use NFT to connect to FastFIS.
  2. Connecting to FastFIS via SFTP/SCP is not available on the SCF.
  3. LC uses its hardware/software security "firewall" to block direct FTP connections from machines outside the llnl.gov domain to LC machines within llnl.gov (including FIS). Such FTP blocking means that you must start your FTP client on a within-llnl.gov machine (or one with a VPN virtual llnl.gov address).
  4. The maximum number of simultaneous FTP connections to FastFIS is 25 (with SFTP, no maximum). While often an invisible limit, this may prevent your reaching the FastFIS server during busy file-exchange periods.
  5. FIS automatically changes some characters in a file's name (not body) during transfer. See the section on "FIS and File Names" for details and a work-around.
  6. The total capacity of the disks on both the open and secure FIS nodes is 70 GB to enable the simultaneous exchange of many large files. There is no per-file FIS size limit, but there are system limitations (e.g., available disk, checksumming time).
  7. FIS cannot process a directory. Make a TAR file instead.

FIS Passwords and Authorization Requests

To use FastFIS, you must already have an account and valid password for at least one open and one secure machine. Before your first use of FastFIS, you must request access via the File Interchange Service form. This form creates your FIS account. It needs Computer Coordinator approval for open-to-secure transfers, and it needs division or department head approval for secure-to-open transfers (which a DC always monitors). OCF (CZ/RZ)-to-SCF FIS accounts do not expire once approved; however, SCF-to-OCF FIS accounts must be renewed annually, and failure to promptly renew them will cause LC to close the account.

Any user can receive authorization to move files from the open to the secure network. To receive authorization to move files from the secure to the open network, you must specify the kinds of files to be moved and obtain the approval of your division leader or department head (in the appropriate places on the FIS form). The instructions on the FIS form remind you of these requirements in relation to each blank.

Once you are authorized, you will be able to use your current authenticator-generated one-time password (RSA SecurID OTP for CZ-only users; CRYTPOCard OTP for RZ-only and CZ/RZ users) to access the corresponding FastFIS server. On the SCF, the FIS server uses the password you use for SCF production machines. Each user of FastFIS has an account on both OCF and SCF servers. A prerequisite for this is an established user record for LC's SCF. The user name for your FIS account is the same as your user name for any LC machine.

How To Use FIS

All users of FastFIS are permitted to transfer from the OCF (CZ/RZ) to the SCF. On the FastFIS server, each user has a private work space. The work space on the CZ and RZ FIS servers consists of two subdirectories named TO and FROM. The TO directory is where a user places files to be transferred to the SCF.

Only the procedure for a FastFIS transfer from the OCF (CZ/RZ) to the SCF is described in this section. The TapeFIS transfer from the SCF to the OCF follows a similar pattern but requires DC review. See Secure-to-Open Transfers for more information.

FIS and File Names

When FastFIS moves files from one server to another, it automatically changes some characters in each file name (not in the body of the file, just in the file name) to avoid characters troublesome to some UNIX file-handling utilities. The table below shows which file name characters are changed during a transfer.

File Name Character Changes To
alphabeticno change
numericno change
internal . (dot)no change
leading . (dot)_ (underscore)
All others (includes
space, hyphen, quote)
_ (underscore)

Users' file-handling scripts and commands need to take account of these changes in file name characters (on the receiving side) to avoid losing or omitting some FastFIS-transferred files. To preserve the special characters in a file's name unchanged, use the UNIX TAR utility to embed the file inside a TAR output file, transfer the TAR file in binary mode with FTP to FastFIS, then run TAR again on the receiving side to extract the original file with its original name. (Changing your FastFIS access software from FTP to SFTP or Hopper has no effect on the file name changes described here.)


Depositing Open-to-Secure Files

FastFIS does not accept file transfers using NFT; you must use FTP or SFTP/SCP. Hopper may also be used. (Use the "Connect to Fastfis..." or "Connect to Rzfastfis..." options in Hopper's Connect menu.) If your files are not already on an llnl.gov machine, you must run FTP on an llnl.gov machine to first get your files from outside. Files with blanks (spaces) or nonASCII characters in their names (such as some Macintosh files) will not be handled properly on UNIX machines. See the FIS and File Names section above on how FIS handles all file names. The maximum number of simultaneous FTP connections with FIS is 25; SFTP connections have no maximum.

If you have one or more files you wish to transfer from the OCF (CZ/RZ) to the SCF, first gain access to the machine that has the file(s) and cd to the directory that contains the file(s). You submit files by using FTP or SFTP/SCP to make a copy of a file from your local machine onto the transfer machine. Initiate the FTP or SFTP/SCP client software or Hopper and connect to fastfis.llnl.gov (CZ-only users), authenticating via RSA SecurID OTP, or connect to rzfastfis.llnl.gov (RZ/CZ-RZ users), authenticating via CRYTPOCard OTP. After you have connected to fastfis.llnl.gov/rzfastfis.llnl.gov, change to your TO directory so that you can submit files for transfer. A binary transfer will copy the file from the local machine to the transfer machine unchanged. In most cases you will want to select the binary transfer mode. FIS treats all files as binary files and does not do any data translation.

Note: It is the responsibility of each user to be mindful of the data transferred and to safeguard against viruses, worms, trojan horses, and other hazards. In addition, the user should ensure that the file content faithfully represents that which the user is attempting to copy.

After copies of your local file(s) are put into the TO directory, they will be queued for transfer within 5 minutes of submission to FastFIS. No user notification will be sent when the files reach their destination. On FastFIS, the individual TO directories are scanned periodcally and submitted file(s) are moved as a batch to a transfer area. The files are then automatically transferred across the OWL to the server on the SCF and desposited in the FROM directory. The file transfer time depends on the size of your files and the size of the files in the queue ahead of yours. Batches are transferred in a FIFO (first-in-first-out) manner, but within a batch, files may be transferred in any order.

If transferring files using TapeFIS, the individual TO directories are scanned periodically and your submitted file(s) are moved to a central collection area. The LC Operations staff monitors this collection area, and they will write a transfer tape (depending on the age and amount of accumulated files). (The separation between open and secure systems is maintained while files are transferred across it by copying the files to tape and physically moving those tapes from the open TapeFIS server to its counterpart on the SCF.) Files are generally transferred within 4 hours. You will receive a transfer notification via e-mail message on the OCF confirming that your files have arrived on the SCF.


Claiming Open-to-Secure Files

FastFIS does not accept file transfers using SFTP/SCP or NFT; you must use FTP or Hopper. (Access via SFTP/SCP is not available on SCF machines.)

Note: If using TapeFIS or RZTapeFIS, retrieval on the SCF or SNSI is by FTP only. SFTP will not work to retrieve files transferred by the TapeFIS systems.

Retrieve files on the SCF by using SFTP/SCP, FTP or Hopper to make a copy of a file from the transfer server to your local machine. Initiate the FTP client software, connect to fastfis.llnl.gov, and complete the authentication process (by specifying a user name and SCF password). After you have connected to to fastfis.llnl.gov, change to your FROM directory and list the file(s) in the directory. Use FTP to get a copy of the file from the transfer server and store it on your local machine. A binary transfer will copy the file from the local machine to the transfer machine unchanged. (In most cases, you will want to select the binary transfer mode.)

Because the transfer server has a finite amount of space and transfers will be impacted if file space is low, it is a good idea to delete the files from the FROM directory once you have retrieved and stored them on your local machine. After several days, files left in the FROM directory will be automatically purged.

Example Using OWL/FastFIS/FIS

CZ-only Users RZ Users SCF or SNSI

sftp fis (PIN and RSA SecurID)

cd TO

put <filename>

exit

sftp rzfis (PIN and CRYPTOCard)

cd TO

put <filename>

exit

sftp fis (PIN and RSA SecurID)

cd FROM

get <filename>

exit


FIS on the iSNSI Network

Both tape FIS and OWL FIS are available on the iSNSI network. iSNSI TapeFIS is bidirectional and requires Derivative Classifier (DC) intervention for file transfers from Pinot to the OCF. Login examples are provided below. OWL transfers are unidirectional for transfers from CZ and RZ to the SCF.

Note: User authentication to iSNSI TapeFIS on the OCF via the CZ (ofis) or RZ (rzofis) requires your OCF LC user name. Authentication to iSNSI TapeFIS from Pinot (sfis) requires your official user name (OUN), which may differ from your OCF LC user name.

CZ Access to iSNSI FIS
Tape OWL

sftp snsitapefis (authenticate with OCF PIN and RSA SecurID token code)

cd TO

put filename

sftp snsifis (authenticate with OCF PIN and RSA SecurID token code)

cd TO

put filename

RZ Access to iSNSI FIS
Tape OWL

sftp rzsnsitapefis (authenticate with PIN and CRYPTOCard token code)

cd TO

put filename

sftp rzsnsifis (authenticate with PIN and CRYPTOCard token code)

cd TO

put filename

Retrieving Files from iSNSI FIS
Tape OWL

From an iSNSI desktop or Pinot

sftp tapefis (authenticate with OUN and SNSI PIN and RSA SecurID token code)

cd FROM

get filename

sftp fastfis (authenticate with OUN and SNSI PIN and RSA SecurID token code)

cd FROM

get filename

DC Support for Secure-to-Open Transfers

Review by a DC is part of every secure-to-open—i.e., SCF to OCF—TapeFIS file transfer. (Note: A special visualization FIS-DC is needed for secure-to-open image and movie file transfers.) To help DCs carry out this review role, TapeFIS provides a special set of directories dedicated to managing files undergoing review and a special software tool (ADCTOOL) for conducting the review online.

DC Review Area

In addition to the TO and FROM directories seen by each regular user of the secure FIS server, DCs have access to a separate review area (where files wait hidden from users and protected from outside changes). This review area is organized by DC pool and by user and managed when DCs run the ADCTOOL utility.


DC Pools

Each organization that participates in TapeFIS secure-to-open file transfers has established one or more "DC review pools" responsible for inspecting candidate files submitted from users in that organization. A DC review pool consists of one or more DCs and follows these rules:

The secure-to-open review process, based on these DC pools, is simple:

  1. A user submits one or more files for transfer from the secure to the open network (by FTPing them to their TO directory on the secure TapeFIS server).
  2. The user then contacts a DC from their review pool and alerts them that files await inspection.
  3. The authorized DC runs ADCTOOL on the secure TapeFIS server to list the submitting user's queued files, move some (or all) of them to a special area (inaccessible to the user) for formal review, and pass (or fail) the files for transfer once reviewed.
  4. The user checks their README file (/users/username/README) on the secure TapeFIS server about the outcome of each file review and claims the file on the TapeFIS node if it is transferred.

ADCTOOL Explained (DCs Only)

ADCTOOL Quick Reference Guide

If you are a DC, you can log into the secure TapeFIS server (using SSH) to manipulate files that users have submitted for review. A utility called ADCTOOL runs as a shell as soon as you log in and offers commands designed to select and (dis)approve (and otherwise manage) user-submitted files and share DC duties among those in your DC pool. ADCTOOL offers the prompt

ADC-yourusername>

and accepts these commands:

list [target]
reports all files currently submitted for transfer or held for review by the DC running ADCTOOL, or (with target options) selectively reports only submitted files (queue), review-held files (held), the users for whom this DC can perform reviews (users), or this DC's peer reviewers and their expiration dates (adcs).
review [username [filelist]]
moves the specified file(s) for the specified user into the review area of the DC running ADCTOOL.
pass [username [filelist]]
approves the specified file(s) for the specified user, prompts for a description of the file type(s), moves the file(s) into the collection area for tape transfer to the open network, and alerts the user by appending a message to their README file.
fail [username [filelist]]
disapproves the specified file(s) for the specified user, deletes the file(s) and overwrites the space, and alerts the user by appending a message to their README file.
assume [username [adcname]]
enables the DC running ADCTOOL to "assume" review duties from the specified DC (adcname) for all currently held files of the specified user.
ftp [username]
starts an FTP session so you can move any currently held files for the specified user to another machine where you can examine their contents to determine their classification status.
help [command]
displays general ADCTOOL help, or help on the specified command.
quit
ends ADCTOOL and logs out of the secure (SCF) FIS node.

ADCTOOL Commands (By Function)

Full technical details on the ADCTOOL commands, grouped by function, are provided below.

File-Review Commands

list [queue | held | both [username]]
list [poolname | adcs | users]
reports the names of available files on the secure FIS node that meet various criteria that you specify, or lists the users or DCs in your DC pool if requested. The alternative arguments for list are (where username and poolname are variables and all the other options are literals):
queue [username]
lists all files currently in the submission (TO) directory of the specified user, or, without username, lists all files in the submission (TO) directories of all FIS users ordered and labeled alphabetically by user name (users with no files are omitted).
held [username]
lists all files that you as DC are currently holdling in your review area for the specified user, or, without username, lists all files held for any user in your review area (users with no files are reported as NONE).
both [username]
lists all files currently in the submission (TO) directory of the specified user or held by you as DC in your review area for the specified user, or, without username, lists all files in the submission (TO) directories of all FIS users ordered and labeled alphabetically by user name (users with no files are omitted) followed by all files held for any user in your review area ordered by user name (users with no files reported as NONE).
both [poolname]
lists all files currently held in the review area by any member of your DC pool (useful if you plan to exchange file-review duties).
adcs
lists all DCs in your pool by user name.
users
lists all users in your pool by user name.
review [username [filelist]]
for a specified user (called username) who belongs to your DC review pool, moves all files named in the space-delimited filelist from the user's submission (TO) directory into a private (separated by user) review area for your inspection. You (and your peer DCs, if any) can see such files moved for review, but the submitting user can no longer see or change them. They remain in the user's separate portion of your review area until you pass or fail them (below, usually after you use the FTP option to allow inspection elsewhere). If you specify username but no filelist, all pending files for that user move from the TO directory into the corresponding review area. If you omit both username and filelist, ADCTOOL prompts you for the user and then the list of files (pressing 'enter' selects all pending files for that user).
pass [username [filelist]]
for a specified user (called username),
  1. Prompts you for the "type of data" you are approving (supply any string up to 8 characters, such as ASCII, which will be logged along with each file's name and owner).
  2. Declares the file(s) named in the space-delimited filelist as approved for transfer to the open network.
  3. Moves those held file(s) from your DC review area into the general collection area for tape transfer (and assigns all information needed to deliver each file to the right user on the open FIS node).
  4. Appends to the user's (SCF FIS) README file a message stating that the specified file(s) have passed review.
If you specify a username but no filelist, all held files for that user move from the review area into the collection area for transfer. If you omit both username and filelist, ADCTOOL prompts you for the user and then the list of files (pressing 'enter' selects all held files for that user).
fail [username [filelist]]
for a specified user (called username),
  1. Declares the file(s) named in the space-delimited filelist as disapproved for transfer to the open network.
  2. Removes those held file(s) from your DC review area and overwrites their disk
  3. Appends to the user's (SCF FIS) README file a message stating that the specified file(s) have failed review.
If you specify username but no filelist, then all held files for that user are removed from the review area. If you omit both username and filelist, ADCTOOL prompts you for the user and then the list of files (pressing 'enter' selects all held files for that user).

File-Management Commands

assume [username [dcname]]
enables a second DC to "assume" review duties from a first DC (called dcname) for all (and only the) currently held (under-review) files submitted by username. If one DC transfers username's file(s) to their review area to begin the classification review but does not complete the process, this command lets any other DC in the same review pool (only) move the already-held files to their second review area to resume review. If you specify username but no dcname, ADCTOOL prompts for the missing DC's name. If you omit both names, ADCTOOL prompts for each one.
ftp [username]
enables you to move held files to another machine (this is the only way you can examine their content in detail to confirm their classification status). This command:
  1. Starts an FTP client on the secure FIS node.
  2. Changes local directories so that (only) file(s) submitted by username are available for transfer.
  3. Lets you OPEN a connection to another secure machine, PUT files, and then QUIT the FTP session.
  4. Resumes your ADCTOOL session when FTP ends.
While FTP runs, you get its prompt directly (and you can use any of its options). If you omit username, ADCTOOL prompts for it. Only a file copy is transferred to the remote machine; each file's reference version remains in the FIS node's review area until you pass or fail it with ADCTOOL.

Housekeeping Commands

help [command]
displays a brief descriptive list of available ADCTOOL commands, or, if you specify command, provides a brief explanation of that command's role.
quit
ends ADCTOOL and logs you out of your current interactive session on the secure FIS node.

Authorization Levels: The Classification Review Categories

Most users are prohibited from transferring any data from the SCF to the OCF. The use of the secure transfer server is limited to retrieving files transferred from the open environment. Users have a TO directory as part of the standard work space, but any files placed in that directory will not be transferered.


NEED Category

For users that have a need and are authorized to transfer files from SCF to OCF, you will have the standard work space (TO and FROM directories) for submitting and retrieving files. Files that you place in the TO directory will be held in the TO directory awaiting DC review and approval. Typically, you submit files into the TO directory and then seek out a DC within your organization that is capable of reviewing your data. Your completed user request form contains the name of a DC pool; the DCs assigned to this pool are capable of reviewing your data. Ask your computer coordinator for the names of the DCs assigned to this pool. Each department or division determines its own file review policy for its own DCs. LC, for example, expects a "cognizant system administrator" DC to review system data before a second, routine review by a "FIS DC" takes place.

You and your DC(s) would normally begin by discussing the content of your submitted files. The DC is also able to select and copy your files for review and examine them on his or her local computer (assisted by a special program called ADCTOOL). The DC can accept the files as approved data and release them back into the transfer path or reject the files as disapproved data and purge the files from FIS.

When a file is selected for review by a DC, it is copied from your TO directory into a review area which is inaccessible by the user. (All files should be submitted in the TO directory, where they await DC inspection.)

So how do users find out if their files have completed review? One way is to ask the DC; another is to look at the README file found in your top-level directory on the secure FIS node (/users/yourname/README). Each time a file has been reviewed (pass or fail), an entry is added to README; you can examine the tail end of this file for the results of the review or simply examine the modification time of the README file to determine if the review action has occurred.