Learn more about Platform products at http://www.platform.com

[ Platform Documentation ] [ Title ] [ Contents ] [ Previous ] [ Next ] [ Index ]


Job Email, and Job File Spooling

Contents

[ Top ]

Mail Notification When a Job Starts

When a batch job completes or exits, LSF by default sends a job report by electronic mail to the submitting user account. The report includes the following information:

The output from stdout and stderr are merged together in the order printed, as if the job was run interactively. The default standard input (stdin) file is the null device. The null device on UNIX is /dev/null.

bsub mail options

-B

Sends email to the job submitter when the job is dispatched and begins running. The default destination for email is defined by LSB_MAILTO in lsf.conf.

-u user_name

If you want mail sent to another user, use the -u user_name option to the bsub command. Mail associated with the job will be sent to the named user instead of to the submitting user account.

-N

If you want to separate the job report information from the job output, use the -N option to specify that the job report information should be sent by email.

-o and -e Options

The output file created by the -o option to the bsub command normally contains job report information as well as the job output. This information includes the submitting user and host, the execution host, the CPU time (user plus system time) used by the job, and the exit status.

If you specify a -o output_file option and do not specify a -e error_file option, the standard output and standard error are merged and stored in output_file. You can also specify the standard input file if the job needs to read input from stdin.

The output files specified by the -o and -e options are created on the execution host.

See Remote File Access for an example of copying the output file back to the submission host if the job executes on a file system that is not shared between the submission and execution hosts.

Disabling job email

If you do not want job output to be sent by mail, specify stdout and stderr as the files for -o and -e. For example, the following command directs stderr and stdout to file named /tmp/job_out, and no email is sent. 

bsub -o /tmp/job_out sleep 5

On UNIX, If you want no job output or email at all, specify /dev/null as the output file: 

bsub -o /dev/null sleep 5

Example

The following example submits myjob to the night queue: 

% bsub -q night -i job_in -o job_out -e job_err myjob

The job reads its input from file job_in. Standard output is stored in file job_out, and standard error is stored in file job_err.

Size of job email

Some batch jobs can create large amounts of output. To prevent large job output files from interfering with your mail system, you can use the LSB_MAILSIZE_LIMIT parameter in lsf.conf to limit the size of the email containing the job output information.

By default, LSB_MAILSIZE_LIMIT is not enabled--no limit is set on size of batch job output email.

If the size of the job output email exceeds LSB_MAILSIZE_LIMIT, the output is saved to a file under JOB_SPOOL_DIR, or the default job output directory if JOB_SPOOL_DIR is undefined. The email informs users where the job output is located.

If the -o option of bsub is used, the size of the job output is not checked against LSB_MAILSIZE_LIMIT.

LSB_MAILSIZE environment variable

LSF sets LSB_MAILSIZE to the approximate size in KB of the email containing job output information, allowing a custom mail program to intercept output that is larger than desired. If you use the LSB_MAILPROG parameter to specify the custom mail program that can make use of the LSB_MAILSIZE environment variable, it is not necessary to configure LSB_MAILSIZE_LIMIT.

LSB_MAILSIZE is not recognized by the LSF default mail program. To prevent large job output files from interfering with your mail system, use LSB_MAILSIZE_LIMIT to explicitly set the maximum size in KB of the email containing the job information.

LSB_MAILSIZE values

The LSB_MAILSIZE environment variable can take the following values:

Directory for job output

The -o and -e options of the bsub and bmod commands can accept a file name or directory path. LSF creates the standard output and standard error files in this directory. If you specify only a directory path, job output and error files are created with unique names based on the job ID so that you can use a single directory for all job output, rather than having to create separate output directories for each job.

Specifying a directory for job output

Make the final character in the path a slash (/) on UNIX, or a double backslash (\\) on Windows. If you omit the trailing slash or backslash characters, LSF treats the specification as a file name.

If the specified directory does not exist, LSF creates it on the execution host when it creates the standard error and standard output files.

By default, the output files have the following format:

Standard output

output_directory/job_ID.out

Standard error

error_directory/job_ID.err

Example

The following command creates the directory /usr/share/lsf_out if it does not exist, and creates the standard output file job_ID.out in this directory when the job completes: 

% bsub -o /usr/share/lsf_out/ myjob

The following command creates the directory C:\lsf\work\lsf_err if it does not exist, and creates the standard error file job_ID.err in this directory when the job completes: 

% bsub -e C:\lsf\work\lsf_err\\ myjob

For more information

See the Platform LSF Reference for information about the LSB_MAILSIZE environment variable and the LSB_MAILTO, LSB_MAILSIZE_LIMIT parameters in lsf.conf, and JOB_SPOOL_DIR in lsb.params.

[ Top ]

File Spooling for Job Input, Output, and Command Files

About job file spooling

LSF enables spooling of job input, output, and command files by creating directories and files for buffering input and output for a job. LSF removes these files when the job completes.

You can make use of file spooling when submitting jobs with the -is and -Zs options to bsub. Use similar options in bmod to modify or cancel the spool file specification for the job. Use the file spooling options if you need to modify or remove the original job input or command files before the job completes. Removing or modifying the original input file does not affect the submitted job.

File spooling is not supported across MultiClusters.

Specifying job input files

Use the bsub -i input_file and bsub -is input_file commands to get the standard input for the job from the file path name specified by input_file. The path can be an absolute path or a relative path to the current working directory. The input file can be any type of file, though it is typically a shell script text file.

LSF first checks the execution host to see if the input file exists. If the file exists on the execution host, LSF uses this file as the input file for the job.

If the file does not exist on the execution host, LSF attempts to copy the file from the submission host to the execution host. For the file copy to be successful, you must allow remote copy (rcp) access, or you must submit the job from a server host where RES is running. The file is copied from the submission host to a temporary file in the directory specified by the JOB_SPOOL_DIR parameter in lsb.params, or your $HOME/.lsbatch directory on the execution host. LSF removes this file when the job completes.

The -is option of bsub spools the input file to the directory specified by the JOB_SPOOL_DIR parameter in lsb.params, and uses the spooled file as the input file for the job.

Use the bsub -is command if you need to change the original input file before the job completes. Removing or modifying the original input file does not affect the submitted job.

Unless you use -is, you can use the special characters %J and %I in the name of the input file. %J is replaced by the job ID. %I is replaced by the index of the job in the array, if the job is a member of an array, otherwise by 0 (zero). The special characters %J and %I are not valid with the -is option.

Specifying a job command file (bsub -Zs)

Use the bsub -Zs command to spool a job command file to the directory specified by the JOB_SPOOL_DIR parameter in lsb.params. LSF uses the spooled file as the command file for the job.

Use the bmod -Zs command if you need to change the command file after the job has been submitted. Changing the original input file does not affect the submitted job. Use bmod -Zsn to cancel the last spooled command file and use the original spooled file.

The bsub -Zs option is not supported for embedded job commands because LSF is unable to determine the first command to be spooled in an embedded job command.

About the job spooling directory (JOB_SPOOL_DIR)

If JOB_SPOOL_DIR is specified in lsb.params:

The JOB_SPOOL_DIR directory should be a shared directory accessible from the master host and the submission host. The directory must be readable and writable by the job submission users.

Except for bsub -is and bsub -Zs, if JOB_SPOOL_DIR is not accessible or does not exist, output is spooled to the default job output directory .lsbatch.

For bsub -is and bsub -Zs, JOB_SPOOL_DIR must be readable and writable by the job submission user. If the specified directory is not accessible or does not exist, bsub -is and bsub -Zs cannot write to the default directory and the job will fail.

If JOB_SPOOL_DIR is not specified in lsb.params:

If you want to use job file spooling, but do not specify JOB_SPOOL_DIR, the LSB_SHAREDIR/cluster_name directory must be readable and writable by all the job submission users. If your site does not permit this, you must manually create lsf_indir and lsf_cmddir directories under LSB_SHAREDIR/cluster_name that are readable and writable by all job submission users.

Modifying the job input file

Use the -i and -is options of bmod to specify a new job input file. The -in and -isn options cancel the last job input file modification made with either -i or -is.

Modifying the job command file

Use the -Z and -Zs options of bmod to modify the job command file specification. -Z modifies a command submitted without spooling, and Zs modifies a spooled command file. The -Zsn option of bmod cancels the last job command file modification made with -Zs and uses the original spooled command.

For more information

See the Platform LSF Reference for more information about the bsub and bmod commands, the JOB_SPOOL_DIR parameter in lsb.params, and the LSF_TMPDIR environment variable.

[ Top ]

[ Platform Documentation ] [ Title ] [ Contents ] [ Previous ] [ Next ] [ Index ]

      Date Modified: January 12, 2004
Platform Computing: www.platform.com

Platform Support: support@platform.com
Platform Information Development: doc@platform.com

Copyright © 1994-2004 Platform Computing Corporation. All rights reserved.