User Tools

Site Tools


qiime2-software

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
qiime2-software [2018/07/22 19:21]
aorth
qiime2-software [2019/12/15 13:11] (current)
aorth
Line 4: Line 4:
 ===== Information ===== ===== Information =====
  
-  * Version: ​2018.6+  * Version: ​2019.10
   * Added: May, 2018   * Added: May, 2018
-  * Updated: ​July2018+  * Updated: ​December2019
   * Link: https://​qiime2.org/​   * Link: https://​qiime2.org/​
  
Line 16: Line 16:
 Load a particular version into your environment and run it: Load a particular version into your environment and run it:
  
-<​code>​$ module load qiime2/2018.6+<​code>​$ module load qiime2/2019.10
 $ qiime --help</​code>​ $ qiime --help</​code>​
  
Line 22: Line 22:
 Notes from the sysadmin during installation:​ Notes from the sysadmin during installation:​
  
-<​code>​$ ​cd /tmp +<​code>​$ ​mkdir -p /var/tmp/​chroot/​qiime2 
-$ sudo mkdir /export/apps/qiime2 +$ rpm --rebuilddb --root=/​var/​tmp/​chroot/​qiime2 
-$ sudo chown aorth:​aorth ​/export/apps/qiime2 +$ wget https://​hpc.ilri.cgiar.org/​mirror/​centos/​7/​os/​x86_64/​Packages/​centos-release-7-7.1908.0.el7.centos.x86_64.rpm 
-$ wget https://​repo.continuum.io/​miniconda/​Miniconda3-latest-Linux-x86_64.sh +$ sudo rpm --root=/var/tmp/chroot/​qiime2 ​-i centos-release-7-7.1908.0.el7.centos.x86_64.rpm 
-bash Miniconda3-latest-Linux-x86_64.sh -p /​export/​apps/​qiime2/​conda +$ sudo yum --installroot=/var/tmp/chroot/​qiime2 ​install -y rpm-build yum wget vim 
-source ​/​export/​apps/​qiime2/​conda/​bin/​activate +sudo cp /​etc/​resolv.conf /​var/​tmp/​chroot/​qiime2/​etc 
-conda update -n base conda +$ sudo mount --bind /dev/ /​var/​tmp/​chroot/​qiime2/​dev 
-wget https://​data.qiime2.org/​distro/​core/​qiime2-2018.6-py35-linux-conda.yml +$ sudo mount -t proc procfs /​var/​tmp/​chroot/​qiime2/​proc 
-conda env create -p /​export/​apps/​qiime2/​2018.--file qiime2-2018.6-py35-linux-conda.yml +$ sudo mount -t sysfs sysfs /​var/​tmp/​chroot/​qiime2/​sys 
-conda activate /​export/​apps/​qiime2/​2018.6 +$ sudo chroot /​var/​tmp/​chroot/​qiime2 
-$ sudo chown -R root:root /​export/​apps/​qiime2 +wget https://​repo.anaconda.com/​miniconda/​Miniconda3-latest-Linux-x86_64.sh 
-qiime --help</​code>​+bash Miniconda3-latest-Linux-x86_64.sh ​-u -p /​export/​apps/​qiime2/​conda 
 +# eval "$(/​export/​apps/​qiime2/​conda/​bin/​conda shell.bash hook)" 
 +conda update -n base conda 
 +wget https://​data.qiime2.org/​distro/​core/​qiime2-2019.10-py36-linux-conda.yml 
 +conda env create -p /​export/​apps/​qiime2/​2019.10 --file qiime2-2019.10-py36-linux-conda.yml 
 +conda activate /​export/​apps/​qiime2/​2019.10 
 +# env > conda-qiime2-2019.10 
 +# exit 
 +$ sudo mkdir -p /​export/​apps/​qiime2/​2019.10 
 +$ sudo chown aorth /​export/​apps/​qiime2/​2019.10 
 +$ rsync -av /​var/​tmp/​chroot/​qiime2/​export/​apps/​qiime2/​2019.10/​ /​export/​apps/​qiime2/​2019.10 
 +$ sudo chown -R root:root /​export/​apps/​qiime2/2019.10 
 +sudo umount /​var/​tmp/​chroot/​qiime2/​dev /​var/​tmp/​chroot/​qiime2/​proc /​var/​tmp/​chroot/​qiime2/​sys</​code>​
  
-Qiime2 ​has 240 dependencies and is extremely tricky ​to install without the use of [[https://​conda.io/​miniconda.html|Miniconda]] (it is not even published on PyPy anymore, so installing it with pip is not an option). My strategy is to install Conda somewhere globally and then use it to install ​Qiime2. After the installation is done I think we don't technically need Conda itself anymore, ​but but will keep it around for ease of upgrading ​and installing ​future versions.+QIIME2 ​has over 200 dependencies and is essentially impossible ​to install without the use of [[https://​conda.io/​miniconda.html|Miniconda]] (it is not even published on PyPy anymore, so installing it with pip is not an option). My strategy is to install Conda somewhere globally and then use it to install ​QIIME2. After the installation is done I think we don't technically need Conda itself anymore, ​as all binaries seem to be linked against libraries in the qiime2/​2019.10 directory. Furthermore, ​first installed qiime2/​2019.10 in a chroot on the local file system ​and then rsynced it over to the network applications directory. This is MUCH faster, but more importantly,​ it avoids random "​permission denied"​ errors I was getting when installing ​on the network file system.
  
 To create the [[https://​github.com/​ilri/​hpc-environment-modules/​tree/​master/​qiime2|modulefile]] I compared the output ''​env''​ before and after loading the Qiime2 environment with Conda. To create the [[https://​github.com/​ilri/​hpc-environment-modules/​tree/​master/​qiime2|modulefile]] I compared the output ''​env''​ before and after loading the Qiime2 environment with Conda.
qiime2-software.1532276475.txt.gz · Last modified: 2018/07/22 19:21 by aorth