Differences

This shows you the differences between two versions of the page.

--- mkatari-bioinformatics-august-2013-introlinuxnotes [2015/06/11 09:22] – [Permissions] mkatari
+++ mkatari-bioinformatics-august-2013-introlinuxnotes [2015/06/11 11:50] (current) – [Controlling Jobs] mkatari
@@ Line 223: / Line 223: @@
     * 7 = read, write and execute
+<code>
+[mkatari@hpc ~]$ tail allusers.txt > bottomusers.txt
+[mkatari@hpc ~]$ ls -al bottomusers.txt
+-rw-rw-r--. 1 mkatari mkatari 772 Jun 11 13:54 bottomusers.txt
-        wget ftp://ftp.jgi-psf.org/pub/compgen/phytozome/v9.0/Mesculenta/v5.0assembly/cassavaV5_0.chromsomesRomanNumerals.fa.gz
+[mkatari@hpc ~]$ chmod u=+r-wx,g=+r-wx,o=-rwx bottomusers.txt
+[mkatari@hpc ~]$ ls -al bottomusers.txt
+-r--r-----. 1 mkatari mkatari 772 Jun 11 13:54 bottomusers.txt
-[[ftp://ftp.jgi-psf.org/pub/compgen/phytozome/v9.0/Mesculenta/annotation/Mesculenta_147_gene.gff3.gz
+[mkatari@hpc ~]$ rm bottomusers.txt
-]]
+rm: remove write-protected regular file `bottomusers.txt'? y
+[mkatari@hpc ~]$ ls -al bottomusers.txt
+ls: cannot access bottomusers.txt: No such file or directory
+</code>
+Notice that since we have taken away our own write permission to the file we can not automatically delete it. The shell is asking to change the permission so it can delete. If we have write permissions that it would have delete right away.
+====== History ======
+Your shell saves all your commands and you can access them using the up and down keys.
+Typing the command “history” returns all the commands you have entered and a number assigned to it.
+You can run a specific one again by typing ! (also called “bang”) Followed by the job number.
+!! Will perform the most recent command.
+In the example code below, ''tail'' is used to get the last lines of the file allusers.txt. The ''>'' sign redirects the output to a new file called bottomusers.txt.
+<code>
+[mkatari@hpc ~]$ history | grep bottom
+  tail allusers.txt > bottomusers.txt
+  ls -al bottomusers.txt
+  chmod u=+r-wx,g=+r-wx,o=-rwx bottomusers.txt
+  ls -al bottomusers.txt
+  rm bottomusers.txt
+  ls -al bottomusers.txt
+  rm bottomusers.txt
+  ls -al bottomusers.txt
+  history | grep bottom
+[mkatari@hpc ~]$ !1000
+tail allusers.txt > bottomusers.txt
+[mkatari@hpc ~]$ ls -al bottomusers.txt
+-rw-rw-r--. 1 mkatari mkatari 772 Jun 11 14:01 bottomusers.txt
+[mkatari@hpc ~]$ !1002
+chmod u=+r-wx,g=+r-wx,o=-rwx bottomusers.txt
+[mkatari@hpc ~]$ ls -al bottomusers.txt
+-r--r-----. 1 mkatari mkatari 772 Jun 11 14:01 bottomusers.txt
+</code>
+====== Transferring Files ======
+There are several ways to transfer files to a server. The most reliable and consistent way is to use ''scp''. scp is a combination of the cp copy command and ssh command for connecting securely. An important thing to note is that you can only use scp if the remote server accepts ssh connections. For example, if you are using a windows machine and have not setup a service to accept ssh connections, you can copy to your windows machines, but you can copy from your windows machine.
+One major advantage of using mobaxterm on windows machines is that it comes with some basic linux commands, including scp. On a mac, scp comes with the operating system. So if you want to copy a file that is on your Desktop to hpc from your laptop you can use the following command:
+<code>
+[2015-06-11 07:22.19]  ~
+[Manpreet.WIN-OAOVO3NM02E] ➤ cp allusers.txt Desktop/allusers_local.txt
+                                                                           ✔
+────────────────────────────────────────────────────────────────────────────
+[2015-06-11 07:22.37]  ~
+[Manpreet.WIN-OAOVO3NM02E] ➤ scp Desktop/allusers_local.txt mkatari@hpc.ilri.cgiar.org:
+Unauthorized access is prohibited.
+allusers_local.txt                        100%   18KB  18.2KB/s   00:00
+</code>
+the : at the end is very important because it tells the shell that is a server and not a file name. The : by itself puts the file in your home directory, but you can specify a specific path if you wanted to.
+Now to copy the file back to my computer I will use the following command. Note that this time I am also renaming the file on my computer.
+<code>
+[Manpreet.WIN-OAOVO3NM02E] ➤ scp mkatari@hpc.ilri.cgiar.org:allusers_local.txt ./return_allusers.txt
+Unauthorized access is prohibited.
+allusers_local.txt                        100%   18KB  18.2KB/s   00:00
+</code>
+====== Controlling Jobs ======
+The following commands and keyboard short-cuts can come in handing when you need to cancel, suspend, or start a job.
+^Command^Action^
+|ctrl-C|Terminate current running job|
+|ctrl-Z|Suspend Jobs|
+| bg | Once a job has been suspended ''bg'' can be put in the background |
+| fg | In order to put a background job in the foreground type ''fg''|
+| & | When executing a command and you want to put it in the background immediately put the ''&'' symbol at the end of the command.|
+| jobs | This gives a list of jobs (suspended, running, and terminating)|
+| top | This starts an interface where you can see all jobs. To see all jobs under your account type ''u'' and then when prompted, your account id.
+====== Commands for manipulating and querying files ======
+Some more cool commands
+^Command^Action^
+|less/more | read through the file without loading the entire file. Press spacebar to continue or q to quit. |
+|touch | create an empty file |
+|head | show the first few lines of the file|
+|tail | show the last few lines of the file|
+|cat | read through the file(s)|
+|grep | search for patterns in a file or files|
+|cut | separate file based on columns|
+|comm/diff | compare and see the difference between the files. The files have to be sorted before using either of these commands.|
+|split | splits file into smaller files based on the options|
+|sort | sort the file base on the options selected.|
+|wc | wordcount|
+====== Exercise ======
+  - Download the file to your computer from the following url [[ftp://ftp.jgi-psf.org/pub/compgen/phytozome/v9.0/Mesculenta/annotation/Mesculenta_147_gene.gff3.gz]]
+  - Transfer it to your LinuxJune2015 directory on HPC.
+  - Gunzip the file and rename it to : Cassavagenes.txt
+  - Determine the number of different genes are present in the file. Hint – check the third column
+  - Save the rows that contain the word “gene” in the third column as a separate file. Call it “GeneTypes.txt”
+  - Create a directory called “Share”
+  - Move GeneTypes.txt into Share and change permissions on Share so others can read the file.
 ====== Some useful information about linux ======