Monitoring in Terminal

Monitoring Jobs

Note: If using the NoMachine NX client, you will need to open a terminal window to execute these commands. From the menu bar, select Applications > Accessories > gnome-terminal.

For a complete reference, please see our page on LSF Commands

bjobs, bpeek, and bhist are different commands that allow you to monitor job activity in LSF. bjobs is the primary and most accurate monitoring tool. bpeek allows you to look at the output and error files of currently running jobs, giving you a glimpse of how the job is progressing. And bhist gives you similar information for running jobs, and can also report on previously finished jobs.

Current & Pending Jobs

Running bjobs without arguments will list all your currently running, pending, and completing jobs:

	bjobs

or for a particular job

	bjobs JOBID

Where JOBID is one of the job IDs listed from the generic bjobs command or from your log of jobs you've run. If you include the -l option (for "long" output) you can get useful data, including the running state of the job.

Running bpeek with the jobID will display the last several lines of the output and error files from your job, as if you had issued the tail command.

	bpeek JOBID

If you wish to monitor them continuously, use the -f option, and type ctrl-C to cancel this.

Past and Current Jobs

Finally, the bhist command will allow you to see current and past information on jobs. For current information:

	bhist

For information on a specific job, include the jobID and -l (long) option (see note):

	bhist -l JOBID

To see past job information, use the -a parameter (all jobs; and see note):

	bhist -a

Note: For older jobs or on a busy cluster (as the HBS Grid has been of late), you may need to include the -n # option, where # is an integer from 1 to 100. This parameters instructs LSF to look thru prior job logs (1 = last log, 2 = 2nd last log, etc). To search all logs, use -n 0:

	bhist -a -n 0 -l 123456

but note this may take several tens of seconds.

Killing Jobs

If for any reason, you need to kill a job that you've submitted, just use the bkill command with the job ID:

	bkill JOBID

If things have really gone haywire and you need to kill all your jobs, give the jobID 0:

	bkill 0

To kill all jobs only in a particular queue, use the -q option:

	bkill -q normal 0

 

Updated 9/17/2019