ILRI Research Computing

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== QIIME2 ======
QIIME™ (canonically pronounced chime) stands for Quantitative Insights Into Microbial Ecology. QIIME is an open-source bioinformatics pipeline for performing microbiome analysis from raw DNA sequencing data. QIIME2 is a completely new and different version than QIIME1.

===== Information =====

  * Version: 2021.8
  * Added: May, 2018
  * Updated: October, 2021
  * Link: https://qiime2.org/

===== Usage =====

See versions of qiime2 that are available:
<code>$ module avail qiime2</code>

Load a particular version into your environment and run it:

<code>$ module load qiime2/2021.8
$ qiime --help</code>

===== Installation ======
Notes from the sysadmin during installation:

<code>$ mkdir -p /var/tmp/chroot/qiime2
$ rpm --rebuilddb --root=/var/tmp/chroot/qiime2
$ wget https://hpc.ilri.cgiar.org/mirror/centos/7/os/x86_64/Packages/centos-release-7-9.2009.0.el7.centos.x86_64.rpm
$ sudo rpm --root=/var/tmp/chroot/qiime2 -i centos-release-7-9.2009.0.el7.centos.x86_64.rpm
$ sudo yum --installroot=/var/tmp/chroot/qiime2 install -y rpm-build yum wget vim
$ sudo cp /etc/resolv.conf /var/tmp/chroot/qiime2/etc
$ sudo mount --bind /dev/ /var/tmp/chroot/qiime2/dev
$ sudo mount -t proc procfs /var/tmp/chroot/qiime2/proc
$ sudo mount -t sysfs sysfs /var/tmp/chroot/qiime2/sys
$ sudo chroot /var/tmp/chroot/qiime2
# wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
# bash Miniconda3-latest-Linux-x86_64.sh -u -p /export/apps/qiime2/conda
# eval "$(/export/apps/qiime2/conda/bin/conda shell.bash hook)"
# conda update -n base conda
# wget https://data.qiime2.org/distro/core/qiime2-2021.8-py38-linux-conda.yml
# conda env create -p /export/apps/qiime2/2021.8 --file qiime2-2021.8-py38-linux-conda.yml
# conda activate /export/apps/qiime2/2021.8
# env > conda-qiime2-2021.8
# exit
$ sudo mkdir -p /export/apps/qiime2/2021.8
$ sudo chown aorth /export/apps/qiime2/2021.8
$ rsync -av /var/tmp/chroot/qiime2/export/apps/qiime2/2021.8/ /export/apps/qiime2/2021.8
$ sudo chown -R root:root /export/apps/qiime2/2021.8
$ sudo umount /var/tmp/chroot/qiime2/dev /var/tmp/chroot/qiime2/proc /var/tmp/chroot/qiime2/sys</code>

QIIME2 has over 200 dependencies and is essentially impossible to install without the use of [[https://conda.io/miniconda.html|Miniconda]] (it is not even published on PyPy anymore, so installing it with pip is not an option). My strategy is to install Conda somewhere globally and then use it to install QIIME2. After the installation is done I think we don't technically need Conda itself anymore, as all binaries seem to be linked against libraries in the qiime2/2021.8 directory. Furthermore, I first installed qiime2/2021.8 in a chroot on the local file system and then rsynced it over to the network applications directory. This is MUCH faster, but more importantly, it avoids random "permission denied" errors I was getting when installing on the network file system.

To create the [[https://github.com/ilri/hpc-environment-modules/tree/master/qiime2|modulefile]] I compared the output ''env'' before and after loading the Qiime2 environment with Conda.