Differences

This shows you the differences between two versions of the page.

--- qiime2-software [2018/07/22 16:21] – aorth
+++ qiime2-software [2021/10/05 15:34] (current) – aorth
@@ Line 4: / Line 4: @@
 ===== Information =====
-  * Version: 2018.6
+  * Version: 2021.8
   * Added: May, 2018
-  * Updated: July, 2018
+  * Updated: October, 2021
   * Link: https://qiime2.org/
@@ Line 16: / Line 16: @@
 Load a particular version into your environment and run it:
-<code>$ module load qiime2/2018.6
+<code>$ module load qiime2/2021.8
 $ qiime --help</code>
@@ Line 22: / Line 22: @@
 Notes from the sysadmin during installation:
-<code>$ cd /tmp
+<code>$ mkdir -p /var/tmp/chroot/qiime2
-$ sudo mkdir /export/apps/qiime2
+$ rpm --rebuilddb --root=/var/tmp/chroot/qiime2
-$ sudo chown aorth:aorth /export/apps/qiime2
+$ wget https://hpc.ilri.cgiar.org/mirror/centos/7/os/x86_64/Packages/centos-release-7-9.2009.0.el7.centos.x86_64.rpm
-$ wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
+$ sudo rpm --root=/var/tmp/chroot/qiime2 -i centos-release-7-9.2009.0.el7.centos.x86_64.rpm
-$ bash Miniconda3-latest-Linux-x86_64.sh -p /export/apps/qiime2/conda
+$ sudo yum --installroot=/var/tmp/chroot/qiime2 install -y rpm-build yum wget vim
-$ source /export/apps/qiime2/conda/bin/activate
+$ sudo cp /etc/resolv.conf /var/tmp/chroot/qiime2/etc
-$ conda update -n base conda
+$ sudo mount --bind /dev/ /var/tmp/chroot/qiime2/dev
-$ wget https://data.qiime2.org/distro/core/qiime2-2018.6-py35-linux-conda.yml
+$ sudo mount -t proc procfs /var/tmp/chroot/qiime2/proc
-$ conda env create -p /export/apps/qiime2/2018.6 --file qiime2-2018.6-py35-linux-conda.yml
+$ sudo mount -t sysfs sysfs /var/tmp/chroot/qiime2/sys
-$ conda activate /export/apps/qiime2/2018.6
+$ sudo chroot /var/tmp/chroot/qiime2
-$ sudo chown -R root:root /export/apps/qiime2
+# wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
-$ qiime --help</code>
+# bash Miniconda3-latest-Linux-x86_64.sh -u -p /export/apps/qiime2/conda
+# eval "$(/export/apps/qiime2/conda/bin/conda shell.bash hook)"
+# conda update -n base conda
+# wget https://data.qiime2.org/distro/core/qiime2-2021.8-py38-linux-conda.yml
+# conda env create -p /export/apps/qiime2/2021.8 --file qiime2-2021.8-py38-linux-conda.yml
+# conda activate /export/apps/qiime2/2021.8
+# env > conda-qiime2-2021.8
+# exit
+$ sudo mkdir -p /export/apps/qiime2/2021.8
+$ sudo chown aorth /export/apps/qiime2/2021.8
+$ rsync -av /var/tmp/chroot/qiime2/export/apps/qiime2/2021.8/ /export/apps/qiime2/2021.8
+$ sudo chown -R root:root /export/apps/qiime2/2021.8
+$ sudo umount /var/tmp/chroot/qiime2/dev /var/tmp/chroot/qiime2/proc /var/tmp/chroot/qiime2/sys</code>
-Qiime2 has 240 dependencies and is extremely tricky to install without the use of [[https://conda.io/miniconda.html|Miniconda]] (it is not even published on PyPy anymore, so installing it with pip is not an option). My strategy is to install Conda somewhere globally and then use it to install Qiime2. After the installation is done I think we don't technically need Conda itself anymore, but but I will keep it around for ease of upgrading and installing future versions.
+QIIME2 has over 200 dependencies and is essentially impossible to install without the use of [[https://conda.io/miniconda.html|Miniconda]] (it is not even published on PyPy anymore, so installing it with pip is not an option). My strategy is to install Conda somewhere globally and then use it to install QIIME2. After the installation is done I think we don't technically need Conda itself anymore, as all binaries seem to be linked against libraries in the qiime2/2021.8 directory. Furthermore, I first installed qiime2/2021.8 in a chroot on the local file system and then rsynced it over to the network applications directory. This is MUCH faster, but more importantly, it avoids random "permission denied" errors I was getting when installing on the network file system.
 To create the [[https://github.com/ilri/hpc-environment-modules/tree/master/qiime2|modulefile]] I compared the output ''env'' before and after loading the Qiime2 environment with Conda.