This section describes how LSF can be configured to support specific systems. Several systems are discussed: IRIX 6 processor sets, Solaris 2.6 processor sets, IBM SP-2, Cray Research NQS, and Atria Clearcase.
IRIX 6 allows the processors in a multiprocessor system to be divided into groups of processors call processor sets. IRIX 6 provides facilities to allow a user to define processor sets, and to run processes onto specific processor sets.
The pset(1M) command allows administrators to set up and manipulate processor sets and associate processes with sets. Once a process is associated with a processor set, the process and all its children will be scheduled only on the processors in that set. The definition of the processor set can be dynamically changed to increase or reduce the number of processors a process can be scheduled on.
LSF can be made to interface with processor sets by using pset(1M) in the queue-level pre- and post-execution commands (see `Queue-Level Pre-/Post-Execution Commands' on page 224). This allows batch jobs to be assigned to certain processors.
Since the pset command must be run as root but, by default, queue-level pre- and post- execution commands are run as the user that submitted the command, you must define LSB_PRE_POST_EXEC_USER=root in /etc/lsf.sudoers. See `The lsf.sudoers File' on page 189 for details.
The following gives examples on how to handle different processor allocation situations for an 8-processor machine.
During the day you want to ensure that batch jobs only use four processors with the remaining four dedicated for interactive users. During the night you want the batch jobs to be able to use all processors.
/etc/pssettab
file as follows:
# <symbolic name> <pset id> <processor list>
#
batch 100 0,1,2,3
interactive 101 4,5,6,7
Run the pset command as root to tell the operating
system to read the pssettab file:
# pset -i -v
PRE_EXEC
parameter:
PRE_EXEC = pset -p $LS_JOBPID batch
The LS_JOBPID will be set to the process identifier of the
batch job process. The pset command will tell IRIX to schedule
that process and any of its children in the batch processor set.
# Move processors 4,5,6,7 out of the batch processor set
pset -s batch \!4,5,6,7
The cron job running at 6 p.m. should execute the command:
# Move processors 4,5,6,7 into the batch processor set.
pset -s batch +4,5,6,7
You want to give a particular user (Jane) exclusive access to one processor if she has jobs to run. Otherwise, users should be able to use all eight processors for batch jobs.
Begin Queue
QUEUE_NAME = exclusive
PRIORITY = 43
USERS = jane
# Move processor 0 from batch processor set to excl processor set
# Associate the job with the excl set.
PRE_EXEC = pset -s excl +0; pset -s batch -0; pset -p $LS_JOBPID excl
#Move processor 0 back to the batch processor set
POST_EXEC = pset -s excl -0; pset -s batch +0
DESCRIPTION = Provides exclusive access to a processor for Jane's jobs
End Queue
Other queues should have their pre-execution command set to:
PRE_EXEC = pset -p $LS_JOBPID batch
More complicated situations can be handled by using scripts in the pre- and post-execution commands which check for other conditions. For example in the above `User-Based Processor Allocation' case, if you wanted to give Jane up to four processors to use (but not more), the pre-execution script could use pset to determine how many processors were already in the `excl' set and move an additional processor from the `batch' set into the `excl' set until the `excl' set has four processors.
Solaris 2.6 allows the processors in a multiprocessor system to be divided into groups of processors call processor sets. It provides facilities to allow a user to define processor sets, and to run processes on specific processor sets.
The psrset(1M) command allows administrators to set up, manipulate, and associate processes with processor sets. Once a process is associated with a processor set, the process and all its children will be scheduled on the processors in that set. The definition of the processor can be dynamically changed to increase or reduce the number of processors on which a process can be scheduled.
LSF can be made to interface with processor sets by using psrset(1M) in the queue-level pre- and post-execution commands (see `Queue-Level Pre-/Post-Execution Commands' on page 224 for detailed information). This allows batch jobs to be dispatched onto certain processors.
Since the psrset (1M) command must be run as root but, by default, queue-level pre- and post-execution commands are run as the user that submitted the command, you must define LSB_PRE_POST_EXEC_USER=root in /etc/lsf.sudoers. See `The lsf.sudoers File' on page 189 for more details.
The following gives examples of how to handle different processor allocation situations for an 8-processor machine.
Suppose that during the day, you want to ensure that batch jobs only use 4 processors with the remaining 4 dedicated for interactive users. During the night, you want the batch jobs to be able to use all processors. This can be accomplished by using the following procedure:
%psrset -c 0 # create a processor set. check the processor set id from the print out.
processor 0: was not assigned, now 1
%psrset -a 1 1 # put processor 1 into processor set 1.
processor 1: was not assigned, now 1
%psrset -a 1 2 # put processor 2 into processor set 1.
processor 2: was not assigned, now 1
%psrset -a 1 3 # put processor 3 into processor set 1.
processor 3: was not assigned, now 1
%psrset -c 4 # create a processor set. check the processor set id from the print out.
processor 4: was not assigned, now 2
%psrset -a 2 5 # put processor 5 into processor set 2.
processor 5: was not assigned, now 2
%psrset -a 2 6 # put processor 6 into processor set 2.
processor 6: was not assigned, now 2
%psrset -a 2 7 # put processor 7 into processor set 2.
processor 7: was not assigned, now 2
PRE_EXEC = psrset -b 1 $LS_JOBPID
The LS_JOBPID will be set to the process identifier
of the batch job process. The psrset command will tell the
system to schedule that process and any of its children in the batch processor
set.
PRE_EXEC = psrset -b 2 $LS_JOBPID
The LS_JOBPID will be set to the process identifier
of the interactive job process. The psrset command will tell
the system to schedule that process and any of its children in the interactive
processor set.
# Move processors 4,5,6,7 out of the batch processor set.
psrset -r 4
psrset -r 5
psrset -r 6
psrset -r 7
The cron job running at 6 p.m. should execute the commands:
# Move processors 4,5,6,7 into the batch processor set.
psrset -a 1 4
psrset -a 1 5
psrset -a 1 6
psrset -a 1 7
Alternatively, you can use LSF JobScheduler to run this periodic job.
Suppose you want to give a particular user (Bob) exclusive access to one processor if he has jobs to run. Otherwise, users should be able to use all 8 processors for batch jobs. This can be accomplished by:
%psrset -c 0 # create a processor set. check the processor set id from the print out.
processor 0: was 1, now 3
%psrset -a 1 0 # release processor 0 from processor set 3, and put it back to the processor set 1.
processor 0: was 3, now 1
Begin Queue
QUEUE_NAME = exclusive
PRIORITY = 43
USERS = bob
# Move processor 0 from batch processor set 1 to exclusive processor set 3
# associate the job with the exclusive set
PRE_EXEC = /usr/local/lsf/etc/procset $LSB_JOBPID
# Move processor 0 back to the batch processor set
POST_EXEC = /usr/local/lsf/etc/procset
DESCRIPTION = Provide an exclusive access to a processor for Bob's jobs
End Queue
The sample script procset is included
in examples/solaris-pset directory of the distribution.
More complicated situations can be handled by using scripts in the pre- and post-execution commands which check for other condition. For example in the above 'User-Based Processor Allocation' case, if you wanted to give Bob up to 4 processors to use (but not more), the pre-execution script could use psrset (1M) to determine how many processors were already in processor set 3 and move an additional processor from the batch processor set 1 into processor set 3 until processor set 3 has 4 processors. Remember to release the processors to the batch processor set 1.
An IBM SP-2 system consists of multiple nodes running AIX. The system can be configured with a high-performance switch to allow high bandwidth, low latency communication between the nodes. The allocation of the switch to jobs as well as the division of nodes into pools is controlled by the SP-2 Resource Manager.
IBM's Parallel Operating Environment (POE) interfaces with the Resource Manager to allow users to run parallel jobs requiring dedicated access to the high performance switch.
The following are provided in LSF to support POE jobs running under LSF:
poejob script which translates the nodes allocated by LSF Batch into an appropriate host list and environment variables for use by POE. The poejob script also causes the job to be requeued, if the request for dedicated access to the switch fails.
pool and lock, are introduced to represent the pool the node is in and whether the switch is locked or not, respectively. The SP-2 ELIM uses the jm_status command to collect information from the Resource Manager.
The `poejob' script is installed as part of the standard installation procedure. The SP-2-specific ELIM can be found in the examples directory of the distribution.
The following steps should be followed to allow POE jobs to run under LSF:
jm_status should return the same names for the nodes that lsload
returns.
README.sp2 file is provided
in the examples directory with specific instructions.
lsf.shared file in the LSF_CONFDIR
directory to pick up the indices reported by the ELIM
Begin Resource
NAME TYPE INTERVAL INCREASING DESCRIPTION
lock Numeric 60 Y (IBM SP2 Node lock status)
pool Numeric 60 N (IBM SP2 POWER parallel system pool)
End NewIndex Resource
lock index. Begin Queue
NAME=poejobs
lock=0
REQUEUE_EXIT_VALUES = 133
...
End Queue
The poejob script will exit with 133
if it is necessary to requeue the job. Note that other types of jobs should
not be submitted to the same queue. Otherwise, they will get requeued if
they happen to exit with 133. The scheduling threshold on the lock
index prevents dispatching to nodes which are being used in exclusive mode
by other jobs.
Note that it is only necessary to enable requeuing of POE jobs if some users submit jobs requiring exclusive access to the node.
The HP Exemplar Technical Server is a high performance cache coherent Non Uniform Memory Access (ccNUMA) computer system. The Exemplar system supports the partitioning of the computing resources into subcomplexes which are collections of processors and memory from one or hypernodes in the system. The Subcomplex Manager (SCM) enables administrators to configure processor and memory resources into subcomplexes.
The following are provided in LSF to support the Exemplar:
LSF does not currently support dynamic load balancing between subcomplexes.
The following steps should be taken to setup an Exemplar system to run LSF.
README file in the examples directory of the distribution.
lsf.shared file.
lsb.queues file.
Edit the LSF_CONFDIR/lsf.shared configuration. Add the definitions for the load indices for each subcomplex. For example, if you have two subcomplexes, you need to configure 12 indices as follows:
Begin Resource
NAME TYPE INTERVAL INCREASING DESCRIPTION
sc1Pme Numeric 60 N (Subcomplex one private memory)
sc1Gme Numeric 60 N (Subcomplex one global memory)
sc1cpu Numeric 60 N (Subcomplex one number cpu)
sc1r5s Numeric 60 Y (Subcomplex one five sec runQ)
sc1r30 Numeric 60 Y (Subcomplex one thirty sec runQ)
sc1r1m Numeric 60 Y (Subcomplex one one minute runQ)
sc2Pme Numeric 60 N (Subcomplex two private memory)
sc2Gme Numeric 60 N (Subcomplex two global memory)
sc2cpu Numeric 60 N (Subcomplex two number cpu)
sc2r5s Numeric 60 Y (Subcomplex two five sec runQ)
sc2r30 Numeric 60 Y (Subcomplex two thirty sec runQ)
sc2r1m Numeric 60 Y (Subcomplex two one minute runQ)
End NewIndex
The index names should be of the form scNxxx where N is the subcomplex number. The name of the subcomplexes defined on the system can be obtained by running the following command
% sysinfo -ls
System load average: 4.30 4.28 4.09
largeGlbMem load average: 3.20 2.18 2.07
The subcomplex number corresponds to its position in the list. In the above example, System is subcomplex 1 and largeGlbMem is subcomplex 2. The names of the indices can be modified if appropriate changes are made to the supplied ELIM.
It is possible to change the name of the indices to include the subcomplex name instead of using a number. This requires changes to the supplied Exemplar ELIM.
The built-in load indices reported by the LIM on the Exemplar are implemented as follows:
r15s, r1m, and r15m are the total run queue lengths of all nodes in all subcomplexes
ut and pg are the CPU utilization and paging rate averaged over all nodes in all subcomplexes
mem is the amount of global memory available
swp index is not currently available on the Exemplar. The swp index should not be used as a threshold to control scheduling
Edit the queue definitions in LSB_CONFDIR/cluster/configdir/lsb.queues
to add queue definitions for each subcomplex. A Job Starter should be specified
for each queue to control which subcomplex jobs from the queue will run on.
Begin Queue
QUEUE_NAME = long
JOB_STARTER = mpa -sc largeGlbMem
.
.
DESCRIPTION = Long jobs on the largGlbMem subcomplex
End Queue
Begin Queue
QUEUE_NAME = short
JOB_STARTER = mpa -sc System
.
.
DESCRIPTION = Short jobs on the System subcomplex
End Queue
The JOB_STARTER parameter uses the mpa(1) command to start the job script file onto the subcomplex specified with the -sc option. LSF sets the LSB_JOBFILENAME environment variable, which specifies a shell script containing the user's commands.
You can use the load indices for each subcomplex to control the scheduling or suspension of jobs on that subcomplex. For example:
Begin Queue
QUEUE_NAME = idle
JOB_STARTER = mpa -sc System
sc1r1m = 2.0/6.0
.
End Queue
would only start jobs on the System subcomplex if the 1-minute run queue was below 2.0 and suspend jobs if it goes above 6.0. Note that the load index specified in the scheduling constraints should correspond to the subcomplex specified in the JOB_STARTER parameter.
It is possible to make use of the queue level pre-/post-execution commands to move CPUs between subcomplexes on a per-job basis. For example, if an exclusive subcomplex has been set up, CPUs can be moved from the system subcomplex before job execution by the pre-exec command and moved back to the system subcomplex after job execution by the post-execution command.
Begin Queue
QUEUE_NAME = exclusive
JOB_STARTER = mpa -sc Exclusive
.
PRE_EXEC = /usr/spp/moveCpuToEx
POST_EXEC = /usr/spp/moveCpuToSys
End Queue
The Exemplar supports kernel level checkpointing using the chkpnt(1) and restart(1) commands. To enable checkpointing on Exemplar systems, copy erestart and echkpnt to the LSF_SERVDIR.
Users are not required to take any special actions for submitting jobs on a Exemplar system. If an Exemplar system is integrated into a larger cluster of machines, it is possible to set up queues that can dispatch to all machines. You need to specify a Job Starter script, which runs the job file through the mpa(1) for the Exemplar, and just executes the job file on non-Exemplar systems. Also scheduling constraints should be specified using the queue-level RES_REQ parameter to distinguish between Exemplar and non-Exemplar systems. For example:
RES_REQ= (type==Exemplar && sc1r1m < 2.0) || (type != Exemplar && r1m < 2.0)
NQS (Network Queuing System) is a UNIX batch queuing facility that allows users to queue batch jobs to individual UNIX hosts from remote systems. This section describes how to configure and use LSF to submit and control batch jobs in NQS queues.
If you are not going to configure LSF to interoperate with NQS, you do not need to read this section.
While it is desirable to run LSF on all hosts for transparent resource sharing, this is not always possible. Some of the computing resources may be under separate administrative control, or LSF may not currently be available for some of the hosts.
An example of this is sites that use Cray supercomputers. The supercomputer is often not under the control of the workstation system administrators. Users on the workstation cluster still want to run jobs on the Cray supercomputer. LSF allows users to submit and control jobs on the Cray system using the same interface as they use for jobs on the local cluster.
LSF queues can be configured to forward jobs to remote NQS queues. Users can submit jobs, send signals to jobs, check the status of jobs, and delete jobs that are forwarded to the remote NQS. Although running on an NQS server outside the LSF cluster, jobs are still managed by LSF Batch in almost the same way as jobs running inside the LSF cluster.
This section describes how to configure LSF and NQS so that jobs submitted to LSF can be run on NQS servers. You should already be familiar with the administration of the NQS system.
NQS uses a machine identification number (MID) to identify each NQS host in the network. The MID must be unique and must be the same in the NQS database of each host in the network. LSF uses the NQS protocol to talk with NQS daemons for routing, monitoring, signalling and deleting LSF Batch jobs that run on NQS hosts. Therefore, you must assign a MID to each of the LSF hosts that might become the master host.
To do this, perform the following steps:
nmapmgr command to create MIDs for each LSF host that can possibly become the master host. List all MIDs available. See the NQS nmapmgr(1) manual page for a description of this command.
NQS uses a mechanism similar to ruserok(3) to determine whether access is permitted. When a remote request from LSF is received, NQS looks in the /etc/hosts.equiv file. If the submitting host is found, requests are allowed as long as the user name is the same on both hosts. If the submitting host is not listed in the /etc/hosts.equiv file, NQS looks for a .rhosts file in the destination user's home directory. This file must contain the names of both the submitting host and the submitting user. Finally, if access still is not granted, NQS checks for a file called /etc/hosts.nqs. This file is similar to the .rhosts file, but it can provide mapping of remote user names to local user names. Cray NQS also looks for a .nqshosts file in the destination user's home directory. The .nqshosts file has the same format as the .rhosts file.
NQS treats the LSF cluster just as if it were a remote NQS server, except that jobs never flow to the LSF cluster from NQS hosts.
For LSF users to get permission to run jobs on NQS servers, you must make sure the above setup is done properly. Refer to your local NQS documentation for details on setting up the NQS side.
The lsb.nqsmaps file in the LSB_CONFDIR/cluster/configdir
directory is for configuring inter-operation between LSF and NQS.
LSF must use the MIDs of NQS hosts when talking with NQS
servers. The Hosts section of the LSB_CONFDIR/cluster/configdir/lsb.nqsmaps
file contains the MIDs and operating system types of your NQS hosts.
Begin Hosts
HOST_NAME MID OS_TYPE
cray001 1 UNICOS #NQS host, must specify OS_TYPE
sun0101 2 SOLARIS #NQS host
sgi006 3 IRIX #NQS host
hostA 4 - #LSF host; OS_TYPE is ignored
hostD 5 - #LSF host
hostB 6 - #LSF host
End Hosts
Note that the OS_TYPE column is required
for NQS hosts only. For hosts in the LSF cluster, OS_TYPE is ignored;
the type is specified by the TYPE field in the lsf.cluster.cluster
file. The `-' entry is a placeholder.
LSF assumes that users have the same account names and user IDs on all LSF hosts. If the user accounts on the NQS hosts are not the same as on the LSF hosts, the LSF administrator must specify the NQS user names that correspond to LSF users.
The Users section of the lsb.nqsmaps file contains entries for LSF users and the corresponding account names on NQS hosts. The following example shows two users who have different accounts on the NQS server hosts.
Begin Users
FROM_NAME TO_NAME
user7 (user7l@cray001 luser7@sgi006)
user4 (suser4@cray001)
End Users
FROM_NAME is the user's login name in the LSF cluster, and TO_NAME is a list of the user's login names on the remote NQS hosts. If a user is not specified in the lsb.nqsmaps file, jobs are sent to the NQS hosts with the same user name.
You must configure one or more LSF Batch queues to forward jobs to remote NQS hosts. A forward queue is an LSF Batch queue with the parameter NQS_QUEUES defined. `Adding a Batch Queue' on page 90 describes how to add a queue to an LSF cluster. The following queue forwards jobs to the NQS queue named pipe on host cray001:
Begin Queue
QUEUE_NAME = nqsUse
PRIORITY = 30
NICE = 15
QJOB_LIMIT = 5
UJOB_LIMIT = ()
CPULIMIT = 15
NQS_QUEUES = pipe@cray001
DESCRIPTION = Jobs submitted to this queue are forwarded to NQS _QUEUES
USERS = all
End Queue
You can specify more than one NQS queue for the NQS_QUEUES parameter. LSF Batch tries to send the job to each queue in the order they are listed, until one of the queues accepts the job.
Since many features of LSF are not supported by NQS, the following queue configuration parameters are ignored for NQS forward queues: PJOB_LIMIT, POLICIES, RUN_WINDOW, DISPATCH_WINDOW, RUNLIMIT, HOSTS, MIG. In addition, scheduling load threshold parameters are ignored because NQS does not provide load information about hosts.
Cray NQS is incompatible with some of the public domain versions of NQS. Different versions of NQS on Cray may be incompatible with each other. If your NQS server host is a Cray, some additional steps may be needed in order for LSF to understand the NQS protocol correctly.
If the NQS version on a Cray is NQS 80.42 or NQS 71.3, then no extra setup is needed. For other versions of NQS on a Cray, you need to define NQS_REQUESTS_FLAGS and NQS_QUEUES_FLAGS in the lsb.params file.
NQS_REQUESTS_FLAGS = integer
If the version is NQS 1.1 on a Cray, the value of this flag is 251918848.
For other versions of NQS on a Cray, do the following to get the value for this flag. Run the NQS command:
% qstat -h CrayHost -a
on a workstation, where CrayHost is the host name of the Cray machine. Watch the messages logged by Cray NQS (you need access to the NQS log file on the Cray host):
03/02 12:31:59 I pre_server(): Packet type=<NPK_QSTAT(203)>.
03/02 12:31:59 I pre_server(): Packet contents are as follows:
03/02 12:31:59 I pre_server(): Npk_str[1] = <>.
03/02 12:31:59 I pre_server(): Npk_str[2] = <platform>.
03/02 12:31:59 I pre_server(): Npk_int[1] = <1392767360>.
03/02 12:31:59 I pre_server(): Npk_int[2] = <2147483647>.
03/02 12:31:59 I show_qstat_flags(): Flags=SHO_R_ALLUID SHO_R_SHORT
SHO_RS_RUN SHO_RS_STAGE SHO_RS_QUEUED SHO_RS_WAIT SHO_RS_HOLD SHO_RS_ARRIVE
SHO_Q_BATCH SHO_Q_PIPE SHO_R_FULL SHO_R_HDR
The value of Npk_int[1] in the above output is the value you need for the parameter NQS_REQUESTS_FLAGS.
NQS_QUEUES_FLAGS = integer
To get the value for this flag, run the NQS command:
% qstat -h CrayHost -p -b -l
on a workstation, where CrayHost is the host name of the Cray machine. Watch the messages logged by Cray NQS (you need to have access to the Cray NQS log file):
03/02 12:32:57 I pre_server(): Packet type=<NPK_QSTAT(203)>.
03/02 12:32:57 I pre_server(): Packet contents are as follows:
03/02 12:32:57 I pre_server(): Npk_str[1] = <>.
03/02 12:32:57 I pre_server(): Npk_str[2] = <platform>.
03/02 12:32:57 I pre_server(): Npk_int[1] = <593494199>.
03/02 12:32:57 I pre_server(): Npk_int[2] = <2147483647>.
03/02 12:32:57 I show_qstat_flags(): Flags=SHO_H_ACCESS SHO_H_DEST
SHO_H_LIM SHO H_RUNL SHO_H_SERV SHO_R_ALLUID SHO_Q_HDR SHO_Q_LIMITS
SHO_Q_BATCH SHO_Q_PIPE SHO_Q_FULL
The value of Npk_int[1] in the above output is the value you need for the parameter NQS_QUEUES_FLAGS.
If you are unable to get the required information after running the above NQS commands, make sure that your Cray NQS is configured properly to log these parameters. To do this, run:
% qmgr
and enter show all to get all information. The parameters related to the logging of the information you need are:
Debug level = 3
MESSAGE_Header = Short
MESSAGE_Types:
Accounting OFF CHeckpoint OFF COMmand_flow OFF
CONfig OFF DB_Misc OFF DB_Reads OFF
DB_Writes OFF Flow OFF NETWORK_Misc ON
NETWORK_Reads ON NETWORK_Writes ON OPer OFF
OUtput OFF PACKET_Contents ON PACKET_Flow ON
PROTOCOL_Contents ON PROTOCOL_Flow ON RECovery OFF
REQuest OFF ROuting OFF Scheduling OFF
USER1 OFF USER2 OFF USER3 OFF
USER4 OFF USER5 OFF
Many sites use Atria's ClearCase environment for revision source control and development. A user uses the cleartool command to startup a ClearCase view. After the view is created, the user is presented with a file system containing the user's sources and binaries. The file system is not accessible outside the view. cleartool has an option to start up a view and run a command under the view.
LSF's job starter can be used to set up a view then run the command (see `Job Starter' on page 227 for further information). For example, if you create a script "clearstarter" similar to the following:
#!/bin/sh
if [ x_$CLEARCASE_ROOT = x_ ]; then
cd $LS_SUBCWD
$*
else
/usr/atria/bin/cleartool setview \
-exec "cd $LS_SUBCWD;$*" \
`basename $CLEARCASE_ROOT`
fi
And specify it as a job starter, the user's job will run by LSF using the command line:
clearstarter myjob
which sets up a view the same as the user's on submission host, changes directory to the same as on submission host, then runs the job. The remote job runs in the same view as on local host.
For interactive jobs, the user sets the environment variable LSF_JOB_STARTER to the ClearCase job starter. The RES on the remote host then will run the user's job via the job starter. After the job starter is set, lsmake can run transparently in ClearCase view.
There are three steps to run an interactive job through the RES in a ClearCase view:
LSF_JOB_STARTER environment variable. This can be done
by each user or as part of the login process. For example:
% setenv LSF_JOB_STARTER clearstarter
% lsmake -j 4 -V -f foo.mak
To run a batch job in ClearCase view, the csub
command should be used instead of bsub. With csub,
no job starter needs to be used (look in the $LSF_INDEP/misc/examples/clearcase
directory for the files clearstarter and csub.
LSF_INDEP is defined in the lsf.conf file).
csub checks whether the environment variable
CLEARCASE_ROOT is set. If it is set, which means the job is
submitted from a view, it wraps the user's job as following:
cleartool setview - exec "cd $LS_SUBCWD;job" `basename $CLEARCASE_ROOT`
and passes all options to bsub, except
-i, -o and -e. These three options
will be translated to shell I/O redirection. For example, suppose CLEARCASE_ROOT=/view/myview
and the user enters:
% csub -q myqueue -o myout -i myin myjob
csub will translate this into:
bsub -q myqueue cleartool setview - exec "cd $LS_SUBCWD; myjob < myin > myout \ 2>&1" myview
An alternative way is to configure a queue-level job
starter (define JOB_STARTER in the file lsb.queues;
see `Queue-Level Job Starters' on
page 129 for details), then use bsub to submit
the job.
Some networks do not share files between hosts. LSF can still be used on these networks, with reduced fault tolerance.
You must choose one host to act as the LSF master host.
The LSF configuration files and working directories must be installed on this
host, and the master host must be listed first in the lsf.cluster.cluster
file.
To install on a cluster without shared file systems, follow the complete installation procedure on every host to install all the binaries, manual pages, and configuration files. After you have installed LSF on every host, you must update the configuration files on all hosts so that they contain the complete cluster configuration. The configuration files must be the same on all hosts.
Some fault tolerance can be introduced by choosing more
than one host as possible master hosts, and using NFS to mount the LSF Batch
working directory on only these hosts. All the possible master hosts must be
listed first in the lsf.cluster.clustername
file. As long as one of these hosts is available, LSF Batch continues to operate.