Differences

This shows you the differences between two versions of the page.

--- using-slurm [2013/08/14 10:55] – aorth
+++ using-slurm [2013/12/02 14:21] – aorth
@@ Line 3: / Line 3: @@
 Our SLURM is configured with the following job queues (also called "partitions" in SLURM):
   * debug
   * batch
   * highmem
-"debug" is the default queue, which is useful for testing job parameters, program paths, etc. The runtime limit of the "debug" partition is 5 minutes, after which jobs are killed.
+"debug" is the default queue, which is useful for testing job parameters, program paths, etc. The run-time limit of the "debug" partition is 5 minutes, after which jobs are killed.
 To see more information about the queue configuration, use ''sinfo -lNe''.
@@ Line 25: / Line 26: @@
 #SBATCH -n 4
-export BLASTDB=/export/data/bio/ncbi/blast/db
+# load the blast module
 module load blast/2.2.28+
+# run the blast with 4 CPU threads (cores)
 blastn -query ~/data/sequences/drosoph_14_sequences.seq -db nt -num_threads 4</code>
@@ Line 34: / Line 35: @@
 <code>$ sbatch blastn_test.sh
 Submitted batch job 1082</code>
+==== Batch job using local storage ====
+Users' home folders are mounted over the network (on "wingu"), so when you're on mammoth or taurus any time you write to the disk (ie job output) has to go round trip over the network.
+Instead, you can use a local "scratch" folder on the compute nodes to alleviate this burden, for example:
+<code>#!/bin/env bash
+#SBATCH -p batch
+#SBATCH -n 4
+#SBATCH -J blastn
+# load the blast module
+module load blast/2.2.28+
+WORKDIR=/var/scratch/$SLURM_JOBID
+mkdir -p $WORKDIR
+echo "Using $WORKDIR on $SLURMD_NODENAME"
+echo
+# run the blast with 4 CPU threads (cores)
+blastn -query ~/data/sequences/drosoph_14_sequences.seq -db nt -num_threads 4 -out $WORKDIR/output</code>
+All output is directed to ''$WORKDIR/'', which is the temporary folder on the compute node.  See these slides from [[http://alanorth.github.io/hpc-users-group3/#/2|HPC Users Group #3]] for more info.
 ==== Check queue status ====
 <code>squeue</code>