A batchtools TORQUE backend resolves futures in parallel via a TORQUE/PBS job scheduler

Usage

batchtools_torque(
  ...,
  template = "torque",
  scheduler.latency = 1,
  fs.latency = 65
)

Arguments

template: (optional) Name of job-script template to be searched for by batchtools::findTemplateFile(). If not found, it defaults to the templates/torque.tmpl part of this package (see below).
scheduler.latency: [numeric(1)]
Time to sleep after important interactions with the scheduler to ensure a sane state. Currently only triggered after calling submitJobs.
fs.latency: [numeric(1)]
Expected maximum latency of the file system, in seconds. Set to a positive number for network file systems like NFS which enables more robust (but also more expensive) mechanisms to access files and directories. Usually safe to set to 0 to disable the heuristic, e.g. if you are working on a local file system.
...: Not used.

Details

Batchtools torque futures use batchtools cluster functions created by batchtools::makeClusterFunctionsTORQUE(), which requires that TORQUE commands qsub, qselect, and qdel are installed on the current machine.

The default template script templates/torque.tmpl can be found in:

system.file("templates", "torque.tmpl", package = "future.batchtools")

and comprise:

#!/bin/bash
######################################################################
# A batchtools launch script template for TORQUE/PBS
#
# Author: Henrik Bengtsson
######################################################################

## Job name:
#PBS -N <%= job.name %>

## Direct streams to logfile:
#PBS -o <%= log.file %>

## Merge standard error and output:
#PBS -j oe

## Email on abort (a) and termination (e), but not when starting (b)
#PBS -m ae

## Resources needed:
<% if (length(resources) > 0) {
  opts <- unlist(resources, use.names = TRUE)
  opts <- sprintf("%s=%s", names(opts), opts)
  cat(sprintf("#PBS -l %s\n", opts))
} %>

echo "Batchtools job name: '<%= job.name %>'"

echo "Session information:"
date
hostname
which Rscript
Rscript --version
Rscript -e ".libPaths()"

## Launch R and evaluate the batchtools R job
echo "Command: Rscript -e 'batchtools::doJobCollection("<%= uri %>")' ..."
Rscript -e 'batchtools::doJobCollection("<%= uri %>")'
res=$?
echo " - exit code: ${res}"
echo "Command: Rscript -e 'batchtools::doJobCollection("<%= uri %>")' ... done"

## End-of-job summary
qstat -f "${PBS_JOBID}"

## Relay the exit code from Rscript
exit "${res}"

References

https://en.wikipedia.org/wiki/TORQUE

Examples

if (FALSE) { # interactive()
# Limit runtime to 3 minutes and memory to 200 MiB per future
plan(batchtools_torque, resources = list(walltime = "00:03:00", mem = "200mb"))

message("Main process ID: ", Sys.getpid())

f <- future(Sys.getpid())
pid <- value(f)
message("Worker process ID: ", pid)
}