Differences

This shows you the differences between two versions of the page.

--- qiime2-software [2018/05/21 07:01] – aorth
+++ qiime2-software [2021/10/05 15:34] (current) – aorth
@@ Line 4: / Line 4: @@
 ===== Information =====
-  * Version: 2018.4
+  * Version: 2021.8
   * Added: May, 2018
+  * Updated: October, 2021
   * Link: https://qiime2.org/
 ===== Usage =====
-See versions of qiime2 which are available:
+See versions of qiime2 that are available:
 <code>$ module avail qiime2</code>
-Load a particular version:
+Load a particular version into your environment and run it:
-<code>$ module load qiime2/2018.4</code>
+<code>$ module load qiime2/2021.8
+$ qiime --help</code>
 ===== Installation ======
 Notes from the sysadmin during installation:
-<code>$ cd /tmp
+<code>$ mkdir -p /var/tmp/chroot/qiime2
-$ sudo mkdir /export/apps/qiime2
+$ rpm --rebuilddb --root=/var/tmp/chroot/qiime2
-$ sudo chown aorth:aorth /export/apps/qiime2
+$ wget https://hpc.ilri.cgiar.org/mirror/centos/7/os/x86_64/Packages/centos-release-7-9.2009.0.el7.centos.x86_64.rpm
-$ wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
+$ sudo rpm --root=/var/tmp/chroot/qiime2 -i centos-release-7-9.2009.0.el7.centos.x86_64.rpm
-$ bash Miniconda3-latest-Linux-x86_64.sh -p /export/apps/qiime2/conda
+$ sudo yum --installroot=/var/tmp/chroot/qiime2 install -y rpm-build yum wget vim
-$ source /export/apps/qiime2/conda/bin/activate
+$ sudo cp /etc/resolv.conf /var/tmp/chroot/qiime2/etc
-$ wget https://data.qiime2.org/distro/core/qiime2-2018.4-py35-linux-conda.yml
+$ sudo mount --bind /dev/ /var/tmp/chroot/qiime2/dev
-$ conda env create -p /export/apps/qiime2/2018.4 --file qiime2-2018.4-py35-linux-conda.yml
+$ sudo mount -t proc procfs /var/tmp/chroot/qiime2/proc
-$ conda activate /export/apps/qiime2/2018.4
+$ sudo mount -t sysfs sysfs /var/tmp/chroot/qiime2/sys
-$ sudo chown -R root:root /export/apps/qiime2
+$ sudo chroot /var/tmp/chroot/qiime2
-$ qiime --help</code>
+# wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
+# bash Miniconda3-latest-Linux-x86_64.sh -u -p /export/apps/qiime2/conda
+# eval "$(/export/apps/qiime2/conda/bin/conda shell.bash hook)"
+# conda update -n base conda
+# wget https://data.qiime2.org/distro/core/qiime2-2021.8-py38-linux-conda.yml
+# conda env create -p /export/apps/qiime2/2021.8 --file qiime2-2021.8-py38-linux-conda.yml
+# conda activate /export/apps/qiime2/2021.8
+# env > conda-qiime2-2021.8
+# exit
+$ sudo mkdir -p /export/apps/qiime2/2021.8
+$ sudo chown aorth /export/apps/qiime2/2021.8
+$ rsync -av /var/tmp/chroot/qiime2/export/apps/qiime2/2021.8/ /export/apps/qiime2/2021.8
+$ sudo chown -R root:root /export/apps/qiime2/2021.8
+$ sudo umount /var/tmp/chroot/qiime2/dev /var/tmp/chroot/qiime2/proc /var/tmp/chroot/qiime2/sys</code>
+QIIME2 has over 200 dependencies and is essentially impossible to install without the use of [[https://conda.io/miniconda.html|Miniconda]] (it is not even published on PyPy anymore, so installing it with pip is not an option). My strategy is to install Conda somewhere globally and then use it to install QIIME2. After the installation is done I think we don't technically need Conda itself anymore, as all binaries seem to be linked against libraries in the qiime2/2021.8 directory. Furthermore, I first installed qiime2/2021.8 in a chroot on the local file system and then rsynced it over to the network applications directory. This is MUCH faster, but more importantly, it avoids random "permission denied" errors I was getting when installing on the network file system.
-Qiime2 has 240 dependencies and is extremely tricky to install without the use of [[https://conda.io/miniconda.html|Miniconda]]—it is not even published on PyPy anymore, so installing it with pip is not an option. I install Conda somewhere globally so that I can use it to install Qiime2. After the installation is done you don't technically need Conda itself, but Qiime2 will need Conda's Python so we must keep it around. I compared the output ''env'' before and after loading the Qiime2 environment with Conda, then used that to create the module file.
+To create the [[https://github.com/ilri/hpc-environment-modules/tree/master/qiime2|modulefile]] I compared the output ''env'' before and after loading the Qiime2 environment with Conda.