C. Using LSF with Alias Renderer

LSF is integrated with Alias|Wavefront's product Alias Renderer. This allows frame array jobs to take advantage of the checkpoint and migration features provided by LSF Batch, and eliminates the need to reprocess completed frames when an interrupted job is migrated or restarted.

In this appendix, we assume that you are already familiar with frame arrays and job starters in LSF.

Overview

When processing a set of frames, it is not convenient to submit each frame as a separate job. In LSF, a frame array is divided into a specified number of related jobs, and they can all be controlled by a single frame array command. Each job may contain several frames.

This is still not the most efficient way to process the data. For example, if a job consisting of five frames is interrupted while processing the fourth frame, the entire job is restarted elsewhere. Ideally, LSF would be aware that three frames have already been completed, and only the remaining fourth and fifth frames would be processed when a job is restarted. This is possible using the Alias|Wavefront software and special LSF commands.

LSF provides wrappers used to append frame parameters (start, end, and step) to the normal Alias Renderer commands. The Alias|Wavefront frame rendering tools call a specified callback function to export data when each frame is done. LSF provides that function, and updates a file that keeps track of the status of each frame in a job. LSF also provides a queue-level job starter that reads information in this status file before starting a job, so if a job is restarted, only the incomplete frames are submitted for processing.

This feature is called checkpointing. You can use a special LSF command to view the status of frames in checkpointed frame arrays.

Distribution

The distributed rendering control software that you can use with Alias|Wavefront's software is available from Platform. Installation instructions are included.

Installing the queue-level job starter

For frame array checkpointing to work, the frame array must be submitted to a queue that uses the special job starter provided by LSF. Include the following line in the queue definition (in the lsb.queues file):

JOB_STARTER = gbstarter

Submitting Checkpointed Frame Arrays

Submit the frame array as usual, but include the parameters required for checkpointing. The syntax for the gbsub command is:

gbsub [-k chkpntDir]
      [-q gbstarter_queue]
      -J "frameArrayName [IndexList,...]"
      [any other regular options to the bsub command]
      gbrenderer | gbraytracer | gbpowercaster | gbpowertracer
      [-f gbcallback]
      [any other regular options to the Renderer command]

The frame array checkpointing feature requires all the following parameters:

-k chkpntDir
Regular option to gbsub command, specifies the name of the checkpoint directory that will contain the status files which keep track of completed frames in each job. This directory must already exist; LSF will not create it for you.

-q gbstarter_queue
Regular option to gbsub command, specifies the name of a queue that uses gbstarter as the job starter.

-J "frameArrayName [IndexList,...]"
Regular parameter of the gbsub command, specifies the name and structure of the frame array.

gb* Renderer commands
Special commands replace the regular parameter of the gbsub command. These wrappers must be used to replace the Alias Renderer commands renderer, raytracer, powercaster, and powertracer. The wrappers accept all the command options accepted by the native commands, and can be used in scripts just like the native commands.

-f gbcallback
Special option to the gb* Renderer command, specifies the LSF function gbcallback that is called by the Alias Renderer application each time a frame is finished. This function automatically updates the status files in the specified checkpoint directory.

Example

You already have a directory called MyDir and the LSF job starter gbstarter is configured on a queue called MyQueue. If you would normally submit your frame array using the Alias Renderer powertracer command, type the following command to enable checkpointing:

gbsub -k MyDir -q MyQueue -J "MyArray [1-10x2:3]" gbpowertracer -f gbcallback

Tracking Checkpointed Frame Arrays

The regular gbjobs command is used with a special switch, -Jf, to show the status of frames in checkpointed frame arrays. If you have not submitted any checkpointed frame arrays, an error message will appear.

Example

The command

gbjobs -u all -Jf "myFrame"

produces output that looks like:

JOBID  USER     FRAMES     EXEC HOST  FRAMES DONE  FRAMES TO GO
  5    usr1   myjob1[1-10]    HostA       1-3          4-10
  6    usr1   myjob2[17-20]   HostB      17-20         none
  9    usr2   seq1[-1-12]     HostC       none        -1-12
  15   usr2   seq6[100-200]   HostD      100-138      139-200
  25   usr3   mov[500-1000]   HostE     500-813     814-1000