AWS EC2

From Indie IT Wiki
Jump to: navigation, search

INFO:

EC2 from Amazon stands for "Elastic Compute Cloud", and is a core component of their cloud based computing services collectively know as Amazon Web Services (AWS).

NOTE: In the following tutorial replace <yourdomain.com> with your own domain name.

Volumes

Volumes are hard disk drives. They can be attached to a running virtual machine (just like plugging the cables in), and then have to be partitioned, formatted and mounted.

EBS

EBS is Elastic Block Storage and basically means that Amazon use a whole bunch of drives stored over several servers and then put those together to make your disk or volume. After a volume is attached to an instance, you can use it like any other physical hard drive. EBS volumes persist independently from the running life of an EC2 instance. This is just like having a physical hard drive, unplugging it and keeping the data on it. When you create an EBS volume in an Availability Zone, it is automatically replicated within that zone to prevent data loss due to failure of any single hardware component. After you create a volume, you can attach it to any EC2 instance in the same Availability Zone. After you attach a volume, it appears as a native block device similar to a hard drive or other physical device. At that point, the instance can interact with the volume just as it would with a local drive.

EFS

EFS is Elastic File Storage and is basically NFS or a network drive. To use it, you have to mount it in the operating system.

Elastic Block Storage (EBS) Pricing

Amazon EBS General Purpose SSD (gp2) volumes
$0.11 per GB-month of provisioned storage

Amazon EBS Provisioned IOPS SSD (io1) volumes
$0.138 per GB-month of provisioned storage
$0.072 per provisioned IOPS-month

Amazon EBS Throughput Optimized HDD (st1) volumes
$0.05 per GB-month of provisioned storage

Amazon EBS Cold HDD (sc1) volumes
$0.028 per GB-month of provisioned storage

Amazon EBS Snapshots to Amazon S3
$0.05 per GB-month of data stored

https://aws.amazon.com/ebs/pricing

Elastic File Storage (EFS) Pricing

US East (N. Virginia)
$0.30/GB-month

US East (Ohio)
$0.30/GB-month

US West (Oregon)
$0.30/GB-month

EU (Ireland)
$0.33/GB-month

https://aws.amazon.com/efs/pricing

Virtual Private Cloud (VPC) Dashboard

https://eu-west-1.console.aws.amazon.com/vpc/home?region=eu-west-1

Helpful YouTube Video

https://www.youtube.com/watch?v=TjVWpNZfTPE

Default Username and Password

ssh -i your-ssh-key.pem ubuntu@external.dns.name

Thanks AskUbuntu.

HOWTO: COMMAND LINE

Increase EBS Volume Size

aws ec2 modify-volume --volume-id vol-XXXXXXXXXXXXX --size <gbsize>

Thanks - https://www.youtube.com/watch?v=9znrrZNW_9o

Instance Stop Start

aws ec2 stop-instances --dry-run --region eu-central-1 --instance-ids i-XXXXXXXX

aws ec2 stop-instances --region eu-central-1 --instance-ids i-XXXXXXXX

aws ec2 describe-instance-status --region eu-central-1 --instance-ids i-XXXXXXXX --output json

aws ec2 start-instances --region eu-central-1 --instance-ids i-XXXXXXXX

aws ec2 describe-instance-status --region eu-central-1 --instance-ids i-XXXXXXXX --output text | grep 'STATE' | awk '{ print $3 }'

Volume Snapshots

aws ec2 create-snapshot --region eu-central-1 --volume-id vol-XXXXXXXX --description "aws cli test"

aws ec2 describe-snapshots --region eu-central-1 --owner-ids XXXXXXXXXX

aws ec2 copy-snapshot --region eu-central-1 --source-region eu-central-1 --source-snapshot-id snap-XXXXXXXXXXXXXXX --destination-region eu-west-2

Security Groups / Firewall

Add Rule

aws ec2 authorize-security-group-ingress --group-name launch-wizard-1 --protocol tcp --port 993 --cidr 123.456.789.10/32

Delete Rule

aws ec2 revoke-security-group-ingress --group-name launch-wizard-1 --protocol tcp --port 993 --cidr 123.456.789.10/32

HOWTO: TAKE SCREENSHOT

How to Take a Screenshot of an Unreachable Instance

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.
  2. In the left navigation pane, choose Instances.
  3. Select the instance to capture.
  4. Choose Actions, Instance Settings.
  5. Choose Get Instance Screenshot.
  6. Right-click on the image to download and save it.

Thanks - http://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/screenshot-service.html#how-to-ics

HOWTO: USAGE - SCRIPT

#!/bin/bash
S3BUCKET=s3://bucket.name;
S3DU=$(s3cmd du ${S3BUCKET} |awk '{print $1}');
S3OBJECTS=$(s3cmd du ${S3BUCKET} |awk '{print $2}');
S3KB=$(echo "scale=2; ${S3DU} / 1024" | bc -l);
S3MB=$(echo "scale=2; ${S3DU} / 1024 / 1024" | bc -l);
S3GB=$(echo "scale=2; ${S3DU} / 1024 / 1024 / 1024" | bc -l);
S3COST=$(echo "scale=2; ${S3GB} * 0.02" | bc -l);
echo "${S3BUCKET}";
echo "${S3OBJECTS} Objects";
echo "${S3KB} KB";
echo "${S3MB} MB";
echo "${S3GB} GB";
echo "${S3COST} GBP";
exit 0;

HOWTO: USERS

Create User Restricted To EC2

Identity and Access Management > Create Individual Users > Manage Users > Create New Users > Friendly Name > Create > Download Credentials

Identity and Access Management > Select User > User Actions > Manage Password > Create > Download Credentials

Identity and Access Management > Policies > Policy Type > Filter > EC2 > AmazonEC2FullAccess > Select > Policy Actions > Attach > Select User > Attach Policy

HOWTO: FIX:

SSH Connection

SSH might have stop running due to an error in the config file.

To fix it...

  1. Stop the Instance
  2. Detach the EBS Volume (making a note of the root device name e.g. /dev/sda1)
  3. Attach the EBS Volume as a secondary Volume to another Instance
  4. Mount the secondary Volume
  5. Fix the errors
  6. Unmount the secondary Volume
  7. Detach the secondary Volume
  8. Reattach to your original Instance making sure you specify /dev/sda1 as the root device

Thanks - https://stackoverflow.com/questions/14026148/running-ec2-instance-suddenly-refuses-ssh-connection/21563478#21563478

Shutting Down Instance

http://aws.amazon.com/instance-help/#ebs-stuck-stopping

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/TroubleshootingInstancesStopping.html

Locale Error

If you receive this error on SSH login...

bash: warning: setlocale: LC_ALL: cannot change locale (en_GB.utf8)

...then fix it with these commands...

sudo aptitude install language-pack-en-base
sudo update-locale
LC_ALL="en_GB.UTF-8" sudo locale-gen
sudo dpkg-reconfigure locales
locale
sudo nano /etc/environment
     LANG="en_GB.UTF-8" # Add this line to the file
sudo reboot

NOTE: Even if you do not receive the error message there is no harm in undertaking the instructions above.

Thanks AskUbuntu.

HOWTO: WEB SERVER

Installation

sudo apt-get install lighttpd
sudo mkdir /var/www/default
sudo touch /var/www/default/favicon.ico
sudo touch /var/www/default/index.html
sudo nano /etc/lighttpd/lighttpd.conf
  server.document-root        = "/var/www/default"
sudo service lighttpd restart

Log File Setup

sudo nano /etc/lighttpd/lighttpd.conf

Enable the access log module...

 server.modules = (
   "mod_accesslog", # Add this line in the server.modules section
)

Create and add the access log line...

accesslog.filename = "/var/log/lighttpd/access.log"

You can add an optional log file format line...

accesslog.format = "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\""

Reload the configuarion...

sudo service lighttpd reload

Thanks - http://www.error418.org/2013/02/lighttpd-access-log.html

Log File Rotation

  • NOTE In the following tutorial replace yourdomain.com with your own domain name.
sudo touch /etc/logrotate.d/lighttpd.yourdomainname.com
sudo nano /etc/logrotate.d/lighttpd.yourdomainname.com

/var/www/yourdomainname.com/logs/*.log {
   monthly
   missingok
   rotate 12
   compress
   delaycompress
   notifempty
   sharedscripts
   postrotate
        if [ -x /usr/sbin/invoke-rc.d ]; then \
           invoke-rc.d lighttpd reopen-logs > /dev/null 2>&1; \
        else \
           /etc/init.d/lighttpd reopen-logs > /dev/null 2>&1; \
        fi; \
   endscript
 }

Save (CTRL+o) and exit (CTRL+x).

Thanks to Cyberciti.

Virtual Hosts

  • NOTE 1: Make sure you move the default document root (shown below)
  • NOTE 2: In the following tutorial replace yourdomain.com with your own domain name.
  • NOTE 3: The following tutorial uses ftpuser1 replace this with your choice of username, i.e. jbloggs.
  • NOTE 4: Items shown like this # This is a comment are just that comments do not run in the terminal or paste them in to files.
Do this first...

Lighttpd#HOWTO:_Move_Default_Document_Root

sudo mkdir -p /var/www/yourdomainname.com/{html,logs}
sudo useradd -d /var/www/yourdomainname.com/ -s /usr/sbin/nologin -g www-data ftpuser1
sudo passwd ftpuser1
sudo id ftpuser1
sudo touch /var/www/yourdomainname.com/logs/{access,error}.log
sudo touch /var/www/yourdomainname.com/html/{favicon.ico,index.html}
sudo nano /var/www/yourdomainname.com/html/index.html

     yourdomainname.com # Add your domain name to the file then save and exit

sudo chown -R www-data:www-data /var/www/
sudo chown -R ftpuser1:www-data /var/www/yourdomainname.com/
sudo chmod g+w /var/www/yourdomainname.com/logs/*
sudo nano /etc/lighttpd/lighttpd.conf

     include "yourdomainname.com.conf" # Add this line to the end of the file then save and exit

sudo nano /etc/lighttpd/yourdomainname.com.conf

     $HTTP["host"] =~ "(^|\.)yourdomainname.com$" {
     server.document-root = "/var/www/yourdomainname.com/html"
     server.errorlog = "/var/www/yourdomainname.com/logs/error.log"
     accesslog.filename = "/var/www/yourdomainname.com/logs/access.log"
     # uncomment below for wordpress
     # url.rewrite-if-not-file = ( "^/(wp-.+).*/?" => "$0", "^/keyword/([A-Za-z_0-9\-]+)/?$" => "/index.php?keyword=$1", "^/.*?(\?.*)?$" => "/index.php$1" )
     }

sudo service lighttpd reload

Thanks to Cyberciti.

HOWTO: ADD TO SECURITY GROUPS

Allow PING

By default EC2 will add a ping rule based on the IP address you are using when you create an machine instance, so to ping from a different IP address you will need to create another rule.

EC2 Dashboard > Network & Security > Security Groups > Select the desired Group ID > Inbound (tab) > / 
                Edit > Add Rule > (Type) Custom ICMP Rule > (Protocol) Echo Request > (Source) Custom IP > Save

Allow HTTP & HTTPS Ports

Separate rules are required for both protocols.

EC2 Dashboard > Network & Security > Security Groups > Select the desired Group ID > Inbound (tab) > /
                Edit > Add Rule > (Type) TCP > Source (Anywhere, for the entire world) > Save

Additional SSH Locations

By default EC2 will add a SSH login rule based on the IP address you are using when you create an machine instance, so to log in via a different IP address you will need to create another rule.

EC2 Dashboard > Network & Security > Security Groups > Select the desired Group ID > Inbound (tab) > / 
                Edit > Add Rule > (Type) SSH > (Source) Either "My IP" or "Custom IP" (best not to use the "Anywhere" option) > Save

HOWTO: ADD SWAP FILE

You can add a 512MB swap to your instance with these commands:

sudo dd if=/dev/zero of=/swapfile bs=1M count=512
sudo mkswap /swapfile
sudo swapon /swapfile
sudo chmod 0600 /swapfile

To enable it by default after reboot, add this line to /etc/fstab:

/swapfile swap swap defaults 0 0

Thanks - http://stackoverflow.com/questions/17173972/how-do-you-add-swap-to-an-ec2-instance#17173973

HOWTO: INCREASE RAM

http://aws.amazon.com/ec2/instance-types

NEW METHOD

  1. Instances > Select Instance > Actions > Instance State > Stop
  2. Instances > Select Instance > Actions > Instance Settings > Change Instance Type > Configure > Review > Change

OLD METHOD

Unfortunately, there is no easy way to increase the RAM in a running instance. You have to create a copy of the instance, launch a new instance based on the copy but with more RAM, associate the Elastic IP address with your new instance, then stop your old instance.

  1. Instances > Select Instance > Actions > Instance State > Stop
  2. Instances > Select Instance > Actions > Image > Bundle Image (AMI) > Name > Save Image
  3. Instances > Launch Instance > My AMIs > Select > Configure > Review > Start
  4. Instances > Select NEW Instance > Note ID
  5. Elastic IPs > Select Elastic IP > Actions > Associate Address > NEW Instance ID > Associate
  6. Instances > Select NEW Instance > Actions > Connect > SSH > Check RAM
  7. Instances > Select OLD Instance > Actions > Instance State > Stop

Thanks - http://imperialwicket.com/aws-upgrade-ec2-instance-memory

HOWTO: BACKUP, COPY, MOVE, DELETE:

Backup Of EC2 Instance

Thanks - http://www.techrepublic.com/blog/the-enterprise-cloud/backing-up-and-restoring-snapshots-on-amazon-ec2-machines/

Copy Instance To A Different Amazon Geographic Region

This is an example of copying an Amazon EC2 Instance from the USA to Ireland.

  1. Log in to AWS Console
  2. Select AWS Region (top right of screen).
  3. Stop Instance.
  4. Create Snapshot.
  5. Copy Snapshot (to EU West Region - Ireland).
  6. Switch AWS Region.
  7. Snapshots > right click on Snapshot and choose Create Image.
  8. AMI > right click on AMI and choose Launch.
  9. Configure Instance as needed.

Official Page - https://aws.amazon.com/blogs/aws/ec2-ami-copy-between-regions/

Nice Tutorial - https://ranjan01.wordpress.com/2014/01/03/how-to-migrate-copy-an-ec2-instance-between-amazon-aws-regions/

Official Doc - http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#migrating-instance-availability-zone

Kernel Fix - http://serverfault.com/questions/447980/correct-process-for-starting-new-ec2-instance-from-an-ebs-snapshot

First Doc - http://stackoverflow.com/questions/5402013/move-amazon-ec2-amis-between-regions-via-web-interface

Ubuntu EC2 Images - http://cloud-images.ubuntu.com/locator/ec2/

HOWTO: DELETE:

AWS calls deletion termination.

  • Select the instance to be deleted
  • From the Actions menu select 'Instance Settings' then 'Change Termination Protection'