Skip to content

Latest commit

 

History

History
268 lines (157 loc) · 6.73 KB

AWS_v2_2021.md

File metadata and controls

268 lines (157 loc) · 6.73 KB
layout permalink title header1 header2 image startDate endDate
tutorial_page
/AWS_v2_2021
AWS 2021
Workshop Pages for Students
AWS and Unix intro
/site_images/CBW_bigdata_icon.jpg
2021-06-03
2021-06-03

Connecting and properly using a cloud computing cluster at the CBW

by Jose Hector Galvez, Zhibin Lu & Rob Syme

Schedule:

Today's schedule can be found here.

Contents:

  1. Logging into AWS

  2. Introduction to the command line

    2.1 Exercise: Exploring the filesystem

  3. File manipulation

    3.1 Exercise: Reading text files

    3.3 Exercise: Editing text files

  4. Searching and sorting files

  5. Putting it all together

6.AWS Machine Image (AMI)

1. Logging into AWS

Description of the lab:

This section will show students how to login to AWS and create an instance.

Once you have confirmed your account as per the email (subject line "You have been invited to join an AWS Educate Classroom"), you can log in here:

2. Introduction to the command line

Description of the lab:

This section will show students the basics of the command line, with a focus on navigation.

Exercise: Exploring the filesystem

  1. Connect to your AWS instance
  2. Type the ls command, what do you see?
Solution (click here) 
$ ls
CourseData  R  cvmfs_cache  workspace

The ls command lists the contents of a working directory.

  1. After following the tutorial, can you answer what these commands (cd, pwd) do?
Solution (click here)

The cd command is used to change directories. Without arguments, it will move to the home directory (~)

The pwd command shows the absolute path to the working directory.

3. File manipulation

Description of the lab:

This section will show students how to manipulate files, including reading, editing, and renaming text files.

Additional material:

Here are two cheat-sheets that can be useful to have as a reference for common UNIX/Linux commands:

Exercise: Reading text files

  1. What do the commands cat, head, and tail do? What do they have in common?
Solution (click here)

All three of these commands ouptut the contents of a text file to standard out:

  • cat outputs the full contents of the file
  • head outputs the first 10 lines of a file
  • tail outputs the last 10 lines of a file
  1. What does the command less do? How is it different from cat?
Solution (click here)

less opens a text file for viewing. Unlike cat, it will display it in a separate file viewer.

  1. How can you know the number of lines in a file?
Solution (click here)

The command wc -l will display the number of lines in a file.

wc (word count) displays the number of words, lines, and bytes in a file. The -l option, limits the output to lines.

Exercise: Editing text files

  1. Write "Hello world" into a file called helloworld.txt using nano. Save and then exit.
Solution (click here)

First, use the nano command to open a file called helloworld.txt

$ nano helloworld.txt

Inside the nano editor, write "Hello world" and then use the ^O option to write the changes and then ^X to exit.

  1. Create a subdirectory called test. Then, move the helloworld.txt file into the directory.
Solution (click here)

First, use the command mkdir to create this new directory. Then, use mv to move helloworld.txt into this directory.

$ mkdir test
$ mv helloworld.txt test/
  1. Create a copy of the helloworld.txt file called helloworld2.txt, inside the test directory.
Solution (click here)

First, change the working directory using cd, then use the cp command to create the copy.

$ cd test
$ cp helloworld.txt helloworld2.txt

4. Searching and sorting files

Description of the lab:

This section will show students how to search for files and in files.

Workshop notes and quiz questions here.

5. Putting it all together

Description of the lab:

This section will show students how the basic concepts fit together and in the context of bioinformatics.

Workshop notes and quiz questions here.

6. AWS Machine Image (AMI)

Starting the AWS AMI after the workshop is over:

An updated AWS Machine Image (AMI) will be be prepred by Zhibin at the end of this CBW workshop: It will have the same software used in the workshop, but it will not contain any of the data. To get such an image started, you will need to set up your own AWS account with your own credit card, and then, the same as you did in this workshop, look for the CBW AMI, and start that with your own project.

Launching the AMI

Let us know on the slack channel or at [email protected] if you are experiencing any problems with this.