mpiblast
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
mpiblast [2010/01/19 07:42] – 172.26.0.166 | mpiblast [2010/01/29 09:42] – 172.26.0.166 | ||
---|---|---|---|
Line 2: | Line 2: | ||
Parallel implementation of NCBI's BLAST algorithm. | Parallel implementation of NCBI's BLAST algorithm. | ||
- | http:// | + | * http:// |
- | OpenMPI FAQ: http:// | + | |
< | < | ||
Line 9: | Line 8: | ||
* **nfrags** specifies how many database fragments you want to split the original database into. This should be equal to how many different nodes you want to run mpiblast on. | * **nfrags** specifies how many database fragments you want to split the original database into. This should be equal to how many different nodes you want to run mpiblast on. | ||
- | http:// | ||
- | ===== Updating BLAST Databases ===== | ||
- | http:// | ||
===== Notes on .ncbirc ==== | ===== Notes on .ncbirc ==== | ||
Notes on setting up the '' | Notes on setting up the '' | ||
Line 30: | Line 26: | ||
The Data variable gives the location of the NCBI Data directory containing BLOSUM and PAM scoring matrices, among other things. The scoring matrix files are necessary for any type of protein BLAST search and should be accessible by all cluster nodes. The BLASTMAT variable also specifies the path to the scoring matrices, and will usually be identical to the Data variable. The BLASTDB variable tells standard NCBI blastall (not mpiBLAST) where to find BLAST databases. As previously mentioned, the Shared and Local variables give the shared and local database paths, respectively. By setting BLASTDB to the same path as Shared, it is possible for NCBI blastall to share the same databases that mpiBLAST uses. In such a configuration, | The Data variable gives the location of the NCBI Data directory containing BLOSUM and PAM scoring matrices, among other things. The scoring matrix files are necessary for any type of protein BLAST search and should be accessible by all cluster nodes. The BLASTMAT variable also specifies the path to the scoring matrices, and will usually be identical to the Data variable. The BLASTDB variable tells standard NCBI blastall (not mpiBLAST) where to find BLAST databases. As previously mentioned, the Shared and Local variables give the shared and local database paths, respectively. By setting BLASTDB to the same path as Shared, it is possible for NCBI blastall to share the same databases that mpiBLAST uses. In such a configuration, | ||
- | |||
- | ===== wwwblast ===== | ||
- | http:// | ||
Line 48: | Line 41: | ||
MCA ras: gridengine (MCA v2.0, API v2.0, Component v1.3.2)</ | MCA ras: gridengine (MCA v2.0, API v2.0, Component v1.3.2)</ | ||
===== Benchmarks ===== | ===== Benchmarks ===== | ||
+ | |||
+ | ==== Standard BLAST ==== | ||
< | < | ||
| | ||
Line 53: | Line 48: | ||
user 7m40.775s | user 7m40.775s | ||
sys | sys | ||
+ | ==== MPI Blast with 4 jobs, 1 node ==== | ||
< | < | ||
Total Execution Time: 395.754 | Total Execution Time: 395.754 | ||
Line 60: | Line 55: | ||
user 12m13.891s | user 12m13.891s | ||
sys | sys | ||
+ | |||
+ | ==== MPI Blast with 12 jobs, 6 nodes ==== | ||
+ | < | ||
+ | Total Execution Time: 98.3068</ | ||
+ | |||
+ | ==== Paracel Blast ==== | ||
+ | < | ||
+ | |||
+ | real 3m6.163s | ||
+ | user 0m0.046s | ||
+ | sys | ||
+ | |||
The number of processes for an MPI job should be +1 of the number of CPUs because one process is used as the master to control the other jobs. | The number of processes for an MPI job should be +1 of the number of CPUs because one process is used as the master to control the other jobs. | ||
+ | |||
+ | ===== Random Notes ===== | ||
+ | |||
+ | ==== Number of Jobs ==== | ||
+ | |||
+ | https:// | ||
+ | |||
+ | %%With formatdb, the number of nodes refers to compute node number. | ||
+ | With mpiBLAST, np refers to number of processes, which isn' | ||
+ | necessarily linked to compute node or processor number. You can run | ||
+ | 10 processes on 4 processors. But it's recommended to run a single | ||
+ | process per processor. But the minimum number of processes for | ||
+ | mpiBLAST is 3, no matter what your compute node number is.%% | ||
+ | ==== Number of Fragments ==== | ||
+ | Rule of thumb for large databases, one segment for every gigabyte (144 GB, 144 segments). | ||
+ | |||
+ | * http:// | ||
+ | * https:// | ||
+ | ==== Incorrect mpiBLAST Version ==== | ||
+ | < | ||
+ | mpiblast-1.5.0-pio | ||
+ | $ mpiblast --version | ||
+ | mpiblast version 1.4.0 | ||
+ | </ | ||
+ | You're not crazy, it's a known issue. 1.5.0 reports as 1.4.0: http:// | ||
===== Links ===== | ===== Links ===== | ||
* Submitting MPI jobs using SGE: http:// | * Submitting MPI jobs using SGE: http:// | ||
- | + | * mpiBLAST Guide: http:// | |
- | $ sudo mkdir -p /state/ | + | * Updating the BLAST databases: http://www.ncbi.nlm.nih.gov/ |
- | $ sudo ln -s /scratch/ /state/partition1 | + | * Rocks documentation on mpiBLAST: http:// |
+ | * wwwblast: http:// | ||
+ | * OpenMPI FAQ: http:// |