LSF Make is not supported on Windows NT. This chapter describes how to use LSF Make to perform parallel software builds and similar tasks.
LSF Make is based on GNU make, and supports all GNU make features. Additional command line options control parallel execution. GNU make is upwardly compatible with the make programs supplied by most UNIX vendors.
To use LSF Make you do not need to change your makefile, although reorganizing the contents of the makefile might increase the parallelism and therefore reduce the running time.
The lsmake(1) manual page describes the command line options that control load sharing. The gmake(1) manual pages describes the other command line options.
LSF Make is not supported on Windows NT.
Many tasks consist of many subtasks, with dependencies between the subtasks. For example, compiling a software package requires compiling each file in the package and then linking all the compiled files together.
In many cases most of the subtasks do not depend on each other. For a software package, the individual files in the package can be compiled at the same time; only the linking step needs to wait for all the other tasks to complete.
In an LSF cluster you can use LSF Make to select a group of hosts and run parts of your make in parallel.
LSF Make supports all the GNU make command line options.
The lsmake -j num_processors option tells LSF Make to ask the LIM for num_processors processors. If fewer processors are available, LSF Make uses all the available processors. If no processors are available and the local host meets all resource requirements specified using the lsmake -R option, all make tasks are run on the local host.
By default LSF Make selects the same host type as the submitting host. This is necessary for most compilation jobs; all components must be compiled on the same host type and operating system version to run correctly. If your make task requires other resources you can override the default resource requirements with the lsmake -R resreq option.
For example, to build your software in parallel on 10 processors, enter:
% lsmake -j 10
If you want to take advantage of parallelism between the CPU and I/O on a powerful host, you can also specify the number of concurrent jobs for each processor using the lsmake -c option, as follows:
% lsmake -j 10 -c 2
This selects up to 10 processors and starts two tasks on each processor.
LSF Make can significantly reduce the response time of your make; however, it may also overload your file server or network if the jobs you are running are I/O intensive.
You can specify a threshold load so that parallelism is automatically reduced when the file server load is above a threshold and expanded when the file server load is below the threshold.
% lsmake -j 10 -F "r15s < 5 && pg < 20"
This LSF Make job uses up to 10 hosts, and reduces the parallelism if the file server CPU load r15s goes beyond 5, or if the file server paging rate goes beyond 20 pages per second. LSF Make automatically determines the file server for the current working directory.
The smallest unit that LSF Make runs in parallel is a single make rule. If your makefile has rules that include many steps, or rules that contain shell loops to build sub-parts of your project, LSF Make runs the steps serially.
You can increase the parallelism in your makefile by breaking up complex rules into groups of simpler rules. Steps that must run in sequence can use make dependencies to enforce the order. LSF Make can then find more subtasks to run in parallel.
If your make job is divided into subdirectories, lsmake -M can process the subdirectories in parallel. The total number of parallel tasks is shared over all the subdirectories. Without the -M option, LSF Make processes subdirectories sequentially, although tasks within each subdirectory can be run in parallel.
To process subdirectories in parallel they must be built as separate targets in your makefile. You must specify the make command for each subdirectory with the built-in $(MAKE) macro so that LSF Make can substitute the correct lsmake command for the subdirectory.
Some makefiles may work correctly when run on a single machine, but may not work correctly when run in parallel through LSF Make.
Below is a makefile rule that uses a shell loop to process subdirectories.
DIRS = lib misc main
prog:
for subdir in $(DIRS) ; do \
cd $subdir ; $(MAKE) ; cd .
When this makefile is run on a single machine, the directories are processed sequentially, i.e. lib is built before misc and main. However, when run using lsmake with the -M option, all directories can be built in parallel. Therefore, it is possible for the misc and main directories to be built before lib, which is not correct.
Below is a set of makefile rules to perform the same tasks and allows the subdirectories to be built in parallel in the correct order.An extra rule is added so that the lib and misc subdirectories are built before the main directory:
DIRS = lib misc main
prog: $(DIRS)
$(DIRS):
cd $@ ; $(MAKE)
make jobs often require a lot of resources, but no user interaction. Such jobs can be submitted to the LSF Batch system so that they are processed when the needed resources are available. lsmake includes extensions to run as a parallel batch job under LSF Batch:
% bsub -n 10 lsmake
This command queues an LSF Make job that needs 10 hosts. When all 10 hosts are available, LSF Batch starts LSF Make on the first host, and passes the names of all hosts in an environment variable. LSF Make gets the host names from the environment variable and uses the RES to run tasks.
You can also specify a minimum and maximum number of processors to dedicate to your make job (see `Minimum and Maximum Number of Processors' on page 104):
% bsub -n 6,18 lsmake
Because LSF Make passes the suspend signal (SIGTSTP) to all its remote processes, the entire parallel make job can be suspended and resumed by the user or the LSF Batch system.
LSF Make is fully compatible with GNU make. There are some incompatibilities between GNU make and some other versions of make; these are beyond the scope of this document.
When LSF Make is running processes on more than one host, it does not send standard input to the remote processes. Most makefiles do not require any user interaction through standard I/O. If you have makefile steps that require user interaction, you can put the commands that require interaction into your local task list. Commands in the local task list always run on the local host, where they can read from standard input and write to standard output.