65855

How to use LSB_JOBINDEX in bsub array job arguments in Platform LSF?

Question:

I would like to pass LSB_JOBINDEX to as an argument to my script instead of using an environment variable.

This makes my script more LSF agnostic and avoids creating a helper script that uses the environment variable.

However, I was not able to use LSB_JOBINDEX in arguments: it only works as part of the initial command string.

For example, from a bash shell, I use the test command:

bsub -J 'myjobname[1-4]' -o bsub%I.log \ 'echo $LSB_JOBINDEX' \ '$LSB_JOBINDEX' \ \$LSB_JOBINDEX \ '$LSB_JOBINDEX' \ "\$LSB_JOBINDEX"

and the output of say bsub2.log is:

2 $LSB_JOBINDEX $LSB_JOBINDEX $LSB_JOBINDEX $LSB_JOBINDEX

So in this case, only the first $LSB_JOBINDEX got expanded, but not any of the following ones.

But I would rather not pass the entire command as a single huge string as the 'echo $LSB_JOBINDEX' in this example. I would prefer to just use separate arguments as in a regular bash command.

I've also tried to play around with %I but it only works for -o and related bsub options, not for the command itself.

Related: <a href="https://stackoverflow.com/questions/11212923/referencing-job-index-in-lsf-job-array" rel="nofollow">Referencing job index in LSF job array</a>

Tested in LSF 10.1.0. Related documentation: <a href="https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_admin/job_array_cl_args.html" rel="nofollow">https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_admin/job_array_cl_args.html</a>

Answer1:

bsub will add single quotes around the arguments if the argument starts with $. For example. If the bsub command line is

bsub command -a $ARG1 -b $ARG2

Then bsub will add quotes to the arguments to the 2nd and 4th parameters. The command is stored like this

command -a '$ARG1' -b '$ARG2'

One way to prevent this is to put the commands in a script. Like this:

$ cat cmd echo $LSB_JOBINDEX echo "line 2" echo $LSB_JOBINDEX

Then run your job like this:

$ bsub -I < cmd Job <2669> is submitted to default queue <normal>. <<Waiting for dispatch ...>> <<Starting on hostA>> 0 line 2 0

Note that the -I is not needed. Its just so you can see the job output on the bsub's stdout.

EDIT

OK. Looks like this works. But its not really a serious answer since it's so ugly. The thing is that bsub will surround the argument with single quotes if the argument starts with $. So the strategy is to find some way to make sure that the first character in the argument isn't a $. One way is to put any character other than $ as the first character of the argument. Follow it by a backspace literal, followed by the $. Note that it needs to be the actual backspace character, not ^ followed by H. Use ctrl-v followed by a ctrl-h to get the literal appended to the command line.

$ bsub -I echo "x^H\$LSB_JOBINDEX" "x^H\$LSB_JOBINDEX" Job <2686> is submitted to default queue <normal>. <<Waiting for dispatch ...>> <<Starting on hostA>> 0 0

EDIT2

A tab literal also works. Not that its much better.

$ bsub -I echo " \$LSB_JOBINDEX" " \$LSB_JOBINDEX" Job <2687> is submitted to default queue <normal>. <<Waiting for dispatch ...>> <<Starting on hostA>> 0 0

Recommend

  • Missing “Run as JUnit Test”
  • Why does typing a variable (or expression) print the value to stdout?
  • Why do I get IPC delays on 20% busy machine
  • Duplicate ID error with updated Mojarra 2.1.27
  • Carousel with thumbnail images at the bottom
  • why this left join returns only one record?
  • kube-apiserver high CPU and requests
  • Textarea toolbar?
  • Spring Security sec:authorize tag using Java Config
  • Sockets: What is causing read() to return EINVAL?
  • Problem with output_buffering and php.ini
  • Using simplemodal with wordpress
  • How can I know the service name?
  • .Net how to set IsReadOnly for a property for a usercontrol
  • Why do I need an infinite loop in STM32 programming?
  • Xamarin.Forms: How To Populate A Pie Chart From Web API Data?
  • Javapackager tool from command-line on OSX?
  • Display standard razor/mvc 3 validation messages displayed in another language
  • Parsing string using the Scanner class
  • How to order the ties in data so that the previously observed value appears first
  • CSS style for container with transparent border images
  • IE doesn't display png images
  • $this->a->b->c->d calling methods from a superclass in php
  • Is possible having two COM STA instances of the same component?
  • Unsupported ciphersuite TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  • Unable to run testNG tests from maven
  • playing mp3 from nsbundle
  • Swift manually rotate view controller
  • Java Collections.shuffle() weird behaviour [closed]
  • readmore button or show/hide for php
  • How to resolve this in PHPUnit where it is asking me to set KERNEL_DIR in my phpunit.xml?
  • Disable account chooser FirebaseUI React
  • Swift: UIView.animate works unexpectedly
  • Tensorflow Dataset API restore Iterator after completing one epoch
  • Python 3x- Compression Makes File Bigger :(
  • Can a PHP script be scheduled to run at a specific time or after a specific amount of time has expir
  • How to warp text around image in iOS?
  • Bind selectedDates Aggregation for Calendar
  • PHP/MySQL Special Characters aren't displayed properly [duplicate]
  • Call Microservice from another Microservice within Docker