Reference no: EM133627360
Cloud Services and Big Data
Learning Outcome 1: Describe and discuss Cloud Computing concepts, service models, delivery models, security issues, representative applications, and business drivers influencing cloud adoption.
Learning Outcome 2: Describe and discuss the practical and theoretical applications of big data and how it can be used to solve business problems, inform business decisions and/or create competitive advantage.
Learning Outcome 3: Identify different big data solution environments and compare their different features and benefits, and advantages and disadvantages.
Learning Outcome 4: Apply Map-Reduce techniques to a business problem involving big-data analytics.
Assessment Description
Students will be given practical work in class to critically assess them on the topics studied in order to examine and demonstrate their working knowledge of contemporary IS applications.
You are encouraged to attend a workshop on Referencing and Research Practice organised by the Academic Success Team (AST). You may also schedule a one-on-one workshop with the AST.
Session 1 Portfolio
Register an AWS account
Free tier will be used for Portfolio Session 1-3
Launch an instance
From the EC2 console dashboard, in the Launch instance box, choose Launch instance, and then choose Launch instance from the options that appear.
Under Name and tags, for Name, enter a descriptive name for your instance.
Under Application and OS Images (Amazon Machine Image), do the following:
Choose Quick Start, and then choose Amazon Ubuntu. This is the operating system (OS) for your instance.
From Amazon Machine Image (AMI), select a free-tier eligible version for Amazon Ubuntu. Notice that these AMIs are marked Free tier eligible. An Amazon Machine Image (AMI) is a basic configuration that serves as a template for your instance.
Under Instance type, from the Instance type list, you can select the hardware configuration for your instance. Choose the t2.micro instance type, which is selected by default. The t2.micro instance type is eligible for the free tier. In Regions where t2.micro is unavailable, you can use a t3.micro instance under the free tier. For more information, see AWS Free Tier.
Under Key pair (login), for Key pair name, choose the key pair that you created when getting
Next to Network settings, choose Edit. For Security group name, you'll see that the wizard created and selected a security group for you. You can use this security group, or alternatively you can select the security group that you created when getting set up using the following steps:
Choose Select existing security group.
From Common security groups, choose your security group from the list of existing security groups.
Keep the default selections for the other configuration settings for your instance.
Review a summary of your instance configuration in the Summary panel, and when you're ready, choose Launch instance.
A confirmation page lets you know that your instance is launching. Choose View all instances to close the confirmation page and return to the console.
On the Instances screen, you can view the status of the launch. It takes a short time for an instance to launch. When you launch an instance, its initial state is pending. After the instance starts, its state changes to running and it receives a public DNS name. If the Public IPv4 DNS column is hidden, choose the settings icon ( Settings icon. ) in the top-right corner, toggle on Public IPv4 DNS, and choose Confirm.
It can take a few minutes for the instance to be ready for you to connect to it. Check that your instance has passed its status checks; you can view this information in the Status check column.
Check information of Your Instance
To prepare to connect to an instance, get the following information from the Amazon EC2 console or by using the AWS CLI.
Get the public DNS name of the instance.
You can get the public DNS for your instance from the Amazon EC2 console. Check the Public IPv4 DNS column of the Instances pane. If this column is hidden, choose the settings icon ( ) in the top-right corner of the screen, and select Public IPv4 DNS. You can also find the public DNS in the instance information section of the Instances pane. When you select the instance in the Instances pane of the Amazon EC2 console, information about that instance will appear on the lower half of the page. Under the Details tab, look for Public IPv4 DNS.
If you prefer, you can use the describe-instances (AWS CLI) or Get-EC2Instance (AWS Tools for Windows PowerShell) commands.
If no Public IPv4 DNS is displayed, verify that the Instance state is Running, and that you have not launched the instance in a private subnet. If you launched your instance using the launch instance wizard, you may have edited the Auto-assign public IP field under Network settings and changed the value to Disable. If you disable the Auto-assign public IP option, the instance is not assigned a public IP address when it is launched.
(IPv6 only) Get the IPv6 address of the instance.
If you assigned an IPv6 address to your instance, you can optionally connect to the instance using its IPv6 address instead of a public IPv4 address or public IPv4 DNS hostname. Your local computer must have an IPv6 address and must be configured to use IPv6. You can get the IPv6 address of your instance from the Amazon EC2 console. Check the IPv6 IPs column of the Instances pane. Or, you can find the IPv6 address in the instance information section. When you select the instance in the Instances pane of the Amazon EC2 console, information about that instance will appear on the lower half of the page. Under the Details tab, look for IPv6 address.
If you prefer, you can use the describe-instances (AWS CLI) or Get-EC2Instance (AWS Tools for Windows PowerShell) commands.
Get the user name for your instance.
You can connect to your instance using the username for your user account or the default username for the AMI that you used to launch your instance.
Get the username for your user account.
Get the default username for the AMI that you used to launch your instance:
Connect to your Linux(Ubuntu) instance from Windows with OpenSSH
How we access the virtual machine on AWS- using SSH
The first information that you need is Public IPv4 DNS
(domain name to identify which computer /devices that you are accessing)
Public IP and security group- find the port number
To access the virtual machine, you need the PEM (the pin that you have downloaded prior)
To find the PEM, you need to use command line
To open the command line -Mac- search box- terminal
To open the command line -Window-search box (left bottom)- command prompt
Code
Go to Windows Command
Dir (Windows) or LS(Mac)-list all documents under the directory
CD-enter the specific directory-e.g.CD desktop-documents
ssh ubuntu-user@publicIP
Yes to continue
Ls find your PEM (use the key to open the port)
Ssh-I YOURKEY.pem ubuntu-user@publicIP
Chmod0400 YOURKEY.pem
Run again:Ssh-I YOURKEY.pem ubuntu-user@publicIP
Exit-controlz
Session 2 Portfolio
Create a MySQL on AWS EC2
check the section Installing MySQL on Ubuntu: Update Package Repository & Install MySQL Choose either install the database using
Terminal
You have already created the connection between the terminal and instance in our portfolio 1
AWS Ubuntu (Install directly over the cloud server) Launch an instance (Ubuntu)
Select Ubuntu, create a keypair or use existing keypair
Connect to the instance:
You may as Ubuntu ‘whoami' to check the status.
Commands used to host MySql Server on AWS EC2 Instance
Step 1: Update the system
sudo apt update
Step 2: Install MySql
sudo apt install mysql-server
Step 3: Check the Status of MySql (Active or Inactive)
sudo systemctl status mysql
Step 4: Login to MySql as a root
sudo mysql
Step 5: Update the password for the MySql Server
ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'place- your-password-here';
FLUSH PRIVILEGES;
Step 6: Test the MySql server if it is working by running sample sql queries
Create a simple relational database with at least 4 tables and relationships with dummy data. Take screenshots of the database you created.
Session 3
In this session we will launch an Amazon RDS database instance covered by the AWS free tier.
Please follow the below steps:
Amazon RDS free tier is available to you only in the first 12 months of your AWS account creation. Each calendar month, the free tier allows you to use the Amazon RDS resources listed below for free :
750 hrs of Amazon RDS in a Single-AZ db.t2.micro, db.t3.micro, and db.t4g.micro Instance.
20 GB of General Purpose Storage (SSD).
20 GB for automated backup storage and any user-initiated DB Snapshots.
Note: When your free usage expires or if your application use exceeds the free usage tier you pay standard pay-as-you-go service rates. Users need to know that not all RDS database engines are available under the AWS free tier.
RDS database engines available under the AWS free tier are as follows:
MySQL
MariaDB
PostgreSQL
Microsoft SQL Server
RDS database engines available under the AWS free tier do not include the following:
Amazon Aurora
Oracle database engines
Step 1: To create an RDS database that's available under the AWS free tier choose services and then enter RDS to navigate to the RDS console. On the RDS console make sure that you have selected the desired AWS region where you want to create the RDS instances then choose Create database.
Step 2: For Choose a database creation method, selects the method that you want. For this demonstration, we'll use the Standard create method.
Step 3: For Engine Type choose the engine type you want. For this demonstration, we will use MySQL, but you can choose between MySQL, MariaDB, PostgreSQL, or Microsoft SQL Server.
Step 4: For the version choose the version of the database engine that you want.
Step 5: For Templates choose Free Tier.
Step 6:For Settings provide the DB instance identifier master username and
password.
Step 7: For DB instance class, storage type and allocated storage leave it at default AWS Free tier values which are
db.t2.micro
General purpose(SSD)
Allocated storage 20 GB
Step 8: For connectivity, database authentication and additional configuration select the option that you want or leave the default configuration.
Step 9: For backup under Additional configuration uncheck enable automatic backups to avoid incurring storage fees for retaining backups of your Amazon RDS instance.
Step 10: Review the estimated monthly cost to be aware of the AWS free tier limits. Then click on Create database.
Now you have successfully created an Amazon RDS database instance that's covered by the AWS free tier.
Download a SQL Client
Once the database instance creation is complete and the status changes to available, you can connect to a database on the DB instance using any standard SQL client. In this step, we will download MySQL Workbench, which is a popular SQL client.
Go to the Download MySQL Workbench page to download and install MySQL Workbench. For more information on using MySQL, see the MySQL Documentation.
Note: Remember to run MySQL Workbench from the same device from which you created the DB instance. The security group your database is placed in is configured to allow connection only from the device from which you created the DB instance.
Go to the Download MySQL Workbench page to download and install MySQL Workbench.
You will be prompted to log in, sign up, or begin your download. You can choose No thanks, just start my download for a quick download.
You will be prompted to login, sign up, or begin your download.
Connect to the MySQL DATABASE
In this step, we will connect to the database you created using MySQL Workbench.
Launch the MySQL Workbench application and go to Database > Connect to Database (Ctrl+U) from the menu bar.
Launch the MySQL Workbench application and go to Database > Connect to Database (Ctrl+U) from the menu bar.
A dialog box appears. Enter the following:
Hostname: You can find your hostname on the Amazon RDS console as shown in the screenshot.
Port: The default value should be 3306.
Username: Type in the username you created for the Amazon RDS database. In this tutorial, it is 'masterUsername.'
Password: Choose Store in Vault (or Store in Keychain on MacOS) and enter the password that you used when creating the Amazon RDS database. Choose OK.
You are now connected to the database! On the MySQL Workbench, you will see various schema objects available in the database. Now you can create tables, insert data, and run queries.
Create a simple relational database with at least 4 tables and relationships with dummy data. Take screenshots of the database you created.
Briefly compare RDS service with EC2-based MySQL you created in session 1 and 2.
Submit your screenshots in a PDF file containing your database created in both EC2-based MySQL and RDS.
Use Harvard referencing.