This chapter contains background information intended for anyone installing any LSF software product on any platform. These concepts should be read and understood before installing LSF software. Understanding this information will allow you to make the informed decisions that lead to a smooth LSF installation.
The products that make up LSF Suite 3.2 are described in the preface of this book. The LSF products are all packaged in the same distribution file, and the installation program requires you to specify which products are to be installed. LSF Batch is installed by default.
LSF Base is a prerequisite for all other LSF products. LSF Batch is prerequisite for LSF JobScheduler, LSF Analyzer, and LSF Parallel.
Individual hosts can be configured to run as LSF Batch servers or LSF JobScheduler servers within the same cluster. LSF MultiCluster is licensed on a cluster-wide basis i.e. the entire cluster is either enabled or disabled for multicluster operation.
Hosts should be chosen so that users on any host in the cluster have shared access to the computing resources on all hosts. LSF includes sophisticated controls to prevent overloading hosts, so interactive workstations can be configured as LSF servers without degrading performance for the owner.
LSF works best when users' home directories are shared across all hosts in the cluster by NFS, by the Andrew File System (AFS), or by the Distributed File System (DCE/DFS).
On Windows NT systems, users' home directories can be mapped to NT shared directories (on an NTFS file system) which can be accessible from other machines via UNC (Universal Naming Convention) path names. Interactive and batch jobs can then access files just as they would on the local host.
LSF can also be used on systems without shared file space, using built-in remote file access to move job input and output. Additionally, batch jobs can also be run on systems without shared user accounts by using LSF's account mapping facility.
Almost all LSF administrative tasks can be done from a non-root account, so LSF can be used on groups of hosts where other system administration tasks are not shared.
If you have more than one type of host, you should put all available host types together in a single cluster. If you have applications that require specific host types, you can configure resource requirements to select the correct host type for each job. This gives users transparent access to applications, regardless of the host(s) to which they are logged in.
LSF is known to operate effectively in clusters of several hundred hosts, clusters supporting a heavy workload, and clusters with a wide range of system types and sizes. There is no built-in limitation to cluster scalability. The maximum size of an LSF cluster, while maintaining good performance, is determined by the system and network environment, as well as the load on your network and the memory available on your hosts.
The cluster size you select depends on the number of hosts you have available, the administrative organization of the hosts, and the tasks you wish to run. Larger clusters usually allow better load sharing, at the cost of slightly more processing overhead in the LSF servers.
Hosts which have multiple network interfaces, each with its own network address, may require special consideration. If such multi-homed hosts will be installed in the cluster, see Appendix E, `Host Naming', beginning on page 117 to determine if additional configuration is required.
All hosts in an LSF cluster can send jobs to other hosts. A server host is a host where LSF sends jobs to run. A client host is a host that only sends jobs out to other hosts to run. Client hosts do not run any LSF daemons and do not run jobs from other hosts.
You should use as many hosts as possible as servers. The more resources you have available for load sharing, the better performance you can get from your cluster. Client hosts can be used when administrative or resource constraints prevent you from using some hosts as servers, or the hosts are too slow or do not have enough resources to run jobs.
One of the server hosts in each cluster is designated as the LSF master host and acts as the overall coordinator for that cluster.
Because LSF products have the flexibility to operate in network environments consisting of hosts of a variety of hardware/operating system combinations, you should choose to install them to a location which simplifies setup and administration as much as possible. A single LSF cluster can include hosts of a single type (homogeneous) or hosts of more than one type (heterogeneous).
There are a number of factors involved, including host type(s), file system type(s), and cluster size(s). There are many possible combinations, and a number of variable elements are involved--each site will be different.
The specific installation directory you choose will depend upon the machines on which you want to install and run LSF products, and on the type of shared file systems you are using.
The components of LSF are divided into those files that do not depend on the host type and those that do. The LSF custom installation procedure allows you to choose where to install all parts of the distribution. Using the default installation mode, all parts of LSF are installed under a single top-level directory--LSF_TOP. The file server will contain the LSF executables for all host types. LSF_TOP must be a local directory.
Figure 1 LSF Directory Structure
For example, LSF_TOP may be /usr/local/lsf.
For example, LSF_TOP can be the LSF share on a server referenced with the UNC name \\SERVER\LSF.
All the host-type specific files are placed in a subdirectory of LSF_TOP/mnt. In the diagram, there are directories for the HP-UX and SOLARIS versions of the software.
LSF_TOP/mnt must be mounted on every LSF host. The default installation procedure creates the mnt subdirectory if it doesn't already exist. Symbolic links are used so that each machine sees the correct host type-specific files. In the example in the diagram, the file server should export/usr/local/lsf/mntand all other hosts should mount this as/usr/local/lsf/mnt. In order for every host type to find the LSF executables and configuration files under the same file name, the following symbolic links are added on each HP-UX host:
/usr/local/lsf/bin to /usr/local/lsf/mnt/hppa/bin
/usr/local/lsf/lib to /usr/local/lsf/mnt/hppa/lib
/usr/local/lsf/etc to /usr/local/lsf/mnt/hppa/etc
and similarly for the Solaris hosts, replacing hppa with sparc-sol2. Now users can put /usr/local/lsf/bin in their PATH environment variable, and their PATH is valid on all hosts.
In Windows environments, symbolic links are not supported. You can use login scripts to set the PATH variable differently based on the type of the host.
The installation examples use /usr/local/lsf as the LSF_TOP directory. If you install LSF under another directory name, make sure that LSF_TOP is local to the host type, and that LSF_TOP/mnt is mounted on all LSF hosts.
Some of the variables you will encounter during the installation procedure are listed below, along with their default values.
This variable specifies the top-level directory under which all LSF files are installed. LSF_TOP must be local to the host type (i.e. it could be a local directory similar to /tmp on each host) or it could be a partition on a file server shared by a set of host types.
In the installation depicted in Figure 1 on page 4, LSF_TOP is local to every host in the cluster, and the mnt subdirectory is mounted on each host from the file server. Host type-specific directories are created on the file server and exported as part of the LSF_TOP mount.
Default:
/usr/local/lsf
Default:
\\SERVER\LSF
Specifies the directory where host type dependent files are installed. In clusters with a single host type (homogeneous clusters), LSF_MACHDEP is usually the same as LSF_INDEP. The machine dependent files are the user programs, daemons, and libraries.
Specifies the default top-level directory for all host type independent LSF files. This includes manual pages, configuration files, working directories, and examples. For example, defining LSF_INDEP as /usr/local/lsf places manual pages in /usr/local/lsf/man, configuration files in /usr/local/lsf/conf, and so on.
Directory where all user commands are installed (default: LSF_MACHDEP/bin).
The directory where all LIM configuration files are installed. These files are shared throughout the system and should be readable from any host. This directory can contain configuration files for more than one cluster (default: LSF_INDEP/conf).
This variable specifies the directory in which the lsf.conf file can be found.
By default, lsf.conf is installed by creating a shared copy in
LSF_CONFDIR and adding a symbolic link from /etc/lsf.conf
to the shared copy. If LSF_ENVDIR is set, the symbolic link is
installed in LSF_ENVDIR/lsf.conf.
This is an optional definition.
If LSF_LOGDIR is defined, error messages from all servers are logged into files in this directory. If a server is unable to write in this directory, then the error logs are created in /tmp.
If LSF_LOGDIR is not defined, then syslog is used to log everything to the system log using the LOG_DAEMON facility. The syslog facility is available by default on most systems. The /etc/syslog.conf file controls the way messages are logged, and the files they are logged to. See the manual pages for the syslogd daemon and the syslog function for more information (default: log messages go to syslog).
If a server is unable to write in the LSF_LOGDIR, then the error logs are created in C:\temp.
Directory where all server binaries are installed. These include lim, res, nios, sbatchd, mbatchd, eeventd (for LSF JobScheduler only). If you use elim, eauth, eexec, esub, etc, they should also be installed in this directory (default: LSF_MACHDEP/etc).
LSF Batch and LSF JobScheduler configuration directories are installed under LSB_CONFDIR. Configuration files for each LSF cluster are stored in a subdirectory of LSB_CONFDIR. This subdirectory contains several files that define the LSF Batch user and host lists, operation parameters, and batch queues.
All files and directories under LSB_CONFDIR
must be readable from all hosts in the cluster. LSB_CONFDIR/cluster/configdir
must be owned by the LSF administrator.
You should not try to redefine this parameter once LSF has been installed. If you want to move these directories to another location, use lsfsetup utility and choose the Product Install option to install configuration files (default: LSF_CONFDIR/lsbatch).
LSF Batch and LSF JobScheduler keep job history and accounting log files for each cluster. These files are necessary for correct operation of the system. Like the organization under LSB_CONFDIR, there is one subdirectory for each cluster.
The LSB_SHAREDIR/cluster/logdir
directory must be owned by the LSF administrator (default: LSF_INDEP/work).
LSF is available on a variety of distribution media. Each LSF distribution medium is shipped with instructions for reading the software on your system. This procedure creates a directory on your disk, which is the distribution directory. The directory name starts with lsf and includes the LSF version number and host type. For example, the distribution directory for the Solaris 2.x version of LSF 3.1 is called lsf3.1_sparc-sol2.
If you obtained LSF by download from Platform's WWW or FTP sites, you can go directly to the next section, `Loading LSF on UNIX'.
Loading LSF on UNIXWhen LSF is distributed on tape, the tape is in tar format, and the contents are compressed tar archives for each host type supported by LSF. To see the contents of the tape, run the command:
% tar tvf /dev/tape_drive_device
You may need to use the tar f option and specify the name of your tape device.
After you identify the binary versions you want, extract them from the tape with the command
% tar xvf /dev/tape_drive_device filename ...
% tar xvf /dev/rst1 lsf3.1_aix4.Z lsf3.1_alpha.tar.Z
LSF is distributed as a compressed tar archive for each host type. The LSF distribution is named by version and host type e.g. lsf3.1_sparc-sol2.tar.Z.
To uncompress the distribution, move the compressed tar file to a temporary directory with at least 100MB of available space and run the command:
% zcat lsf3.1_sparc-sol2.tar.Z | tar xvf -
This creates a distribution directory called lsf3.1_sparc-sol2 under the working directory.
Loading LSF on Windows NTLSF for Windows NT and Windows 95 are distributed as self-extracting executables (.exe). To uncompress the distribution, run the executable by double-clicking on it from the File Manager or Windows Explorer. A prompt will allow for selecting the distribution directory in which the LSF binaries will be uncompressed. The distribution directory should contain the setup.exe which performs the installation.
On all supported platforms except Cray, and regardless of which or how many LSF products are being installed, LSF requires less than 100MB of disk space for a complete installation.
On Cray platforms, LSF may require as much as 150MB of disk space.
You must have superuser privileges (root access) to install LSF products.
You must use a domain account, which is in the domain administrator's group.
You must choose a user account to act as the primary LSF administrator. This account owns the LSF configuration files and has permission to reconfigure LSF and to control batch jobs submitted by other users. You can use an existing user account, or you can create a new account. By creating a separate LSF administrator account you can allow more than one person to modify the configuration files; if you assign only one individual as the administrator then no one else can modify the configuration.
Once you have installed LSF, you can configure secondary LSF administrators. Secondary LSF administrators have the same privileges as the primary administrator except that they do not have permission to update the LSF configuration files as these files are owned by the primary administrator.
You should not configure the root account as the LSF administrator.
Many network installations restrict the root account so that it does not have permission to write to NFS or AFS mounted directories.
If you are installing LSF on a Network File System (NFS) mounted directory then the NFS server must allow the root account on the client to write to the mounted file system (it must be exported with setuid enabled).
The primary administrator is also used as the account to run the LSF daemons. It is assigned certain privileges necessary to run jobs during the installation.
You must have a software license key to run LSF. You can get a license key from your LSF vendor, or from Platform Computing. There are two types of LSF license keys: temporary and permanent.
A temporary (DEMO) license key will allow you to use LSF for a limited time only, and may easily be replaced with a permanent license key later on.
A permanent license key cannot be created for you until you supply information identifying the host(s) on which the LSF license server will run at your site. The LSF installation utility can help you retrieve this information and install your permanent license key. Instructions for obtaining a permanent license key are contained in `Getting License Key Information' on page 34.
If you received a DEMO license key, you can proceed directly with the installation. To get a permanent license from your LSF vendor, see the Release Notes or `Getting License Key Information' on page 34. You can install LSF with a DEMO license key and change to a permanent license later with no interruption in service.
Store the license key in a file. lsfsetup automatically finds your license key if you store the license key in a file named license.dat in the distribution directory. Otherwise, you must enter the path name of the file during the installation.
You should back up the root partition disk on each host and the file server disks where LSF products will be placed before installing LSF. Use the normal backup procedure for your site.
Default: 
Default: 
Default: 
This is required.