AWS EC2 Chapter-1

1. EC2 instances are virtual machines offered by Amazon to run your workloads. These instances can be spin up or down on demand and they come with a variety of different configurations in terms of hardware and software.
2. EC2 runs multiple threads on a single CPU and each thread is also known as virtual CPU. Basically, vCPUs are Core counts multiplied by Threads per core.
3. When an EC2 instance is stopped and started, the EBS root volume data in the EC2 instance remains intact. However, all EBS root volume data is lost on the termination of an instance.
4. When an EC2 instance starts, we can perform startup operations such as initializations and software installs using EC2 User Data.
5. EC2 instances can not only be started, stopped, and terminated but also put into hibernate mode.
6. Hibernate allows even RAM data to be preserved. There are some limitations in instance types, etc.

Different options to purchase an EC2 instance

1. On-Demand instances have predictable pricing and can be bought when necessary.
2. Reserved Instances are the best cost option if you already know you will be using the instance in a steady-state for a very long duration(a minimum of 1 year). Convertible reserved instances provide flexible instances — for example, you can change instance types with more memory and/or CPU. Scheduled Reserved Instances — example every Monday from 9–5 only.
3. Capacity reservations are to ensure that a certain capacity is available to use at a given point in time. There is no time commitment required(1/3 years) as is with reserved instances.
4. Spot instances are the cheapest option and should be used for workloads that are resilient to failure since spot instances can be lost at any point in time. If the price you are paying for the spot instance is lower than the current spot price, you can lose your instance after a grace period of 2 minutes. Batch or Data Analytics jobs that can be stopped and restarted at any point are perfect candidates. They are also used for scaling up when stateless applications meet high demand temporarily.
5. Spot Block allows blocking off an instance for a time block (1–6 hours) without interruptions.
6. Spot Fleets allow to launch of multiple instances, spot or on-demand to meet specified strategies. The strategy could be the lowest cost, high availability, or diversified.
7. We can also set up spot persistent requests that will remain active and will be reactivated if the spot instance was terminated. All spot instances should be canceled before terminating the instances, else new spot instances may be provisioned automatically.
8. Dedicated Hosts are used for compliance requirements and server-bound software licenses. They are committed to only the buyer and are not shared.
9. EC2 Dedicated Instances are only slightly different from Dedicated Hosts. I found this explanation that makes the most sense.
10. When an EC2 instance is stopped and started its public IP changes. An Elastic IP ensures the public IP does not change. By default, you can only have 5 Elastic IPs per AWS account. The best practice would be to use registered DNS names through Route 53 or an Application Load Balancer that point to any IP address.

EC2 Placement Groups

These are used when we want control over how EC2 instances should be placed relative to each other.

1. Cluster — These are for low latency(10Gbps) communication amongst the Ec2 instances, They are placed in a group in the same rack and same Availability Zone. The risk here is that the rack fails we lose all the servers. example — Big Data jobs.
2. Spread — These are for critical applications that can't afford to lose all their servers at once. The servers can be spread out across availability zones to minimize outages but there is a limit of 7 servers per AZ with this option
3. Partition- Similar to spread but across partitions or racks across availability zones. Here we can have up to 7 partitions per AZ but 100s of EC2 instances spread out. HDFS, Cassandra, Kafka is good to use cases.

Elastic Network Interface

1. Elastic Network Interfaces can be attached to EC2 instances and can be reused and moved from one EC2 instance to another. You can create a network interface, attach it to an instance, detach it from an instance, and attach it to another instance.ENI’s can be used to implement quick and easy failover strategies.
2. ENIs am bounded to a particular AZ. An ENI one AZ cannot be attached to an EC2 instance in a different AZ.
3. EC2 Nitro is a new virtualization technology with increased IOPS for higher speed EBS(64K IOPS as opposed to a max 32000 IOPS).

Instance Userdata and Metadata

1. http://169.254.169.254/latest/meta-data is an internal URL that will give you details about your instance.
2. http://169.254.169.254/latest/user-data is an internal URL that will give you the launch script of your instance.

EC2 Storage Options

EC2 instances on AWS can have different types of storage attached to them.

1. Root Volume: The root device volume contains the image used to boot the instance. EC2 instances can be launched as instance store-backed AMI or an Amazon EBS-backed AMI. Any data on the instance store volumes persists as long as the instance is running, but this data is deleted when the instance is terminated
2. Instance Stores: If you need very high performance(high IOPS) hardware attached disks, EC2 instance stores may be used. The instance stores are ephemeral, that is their life span is temporary and they live as long as the EC2 instance lives. So if you stop or terminate the EC2 instance, the instance store is lost as well.

An instance store is great for temporary content, cache buffer type data.

3. EBS Volume: Elastic Block Store is a network drive that can be attached to the instance.

They can be attached at any time to an EC2 instance, even while they are running. Even if the EC2 instance has been terminated, the EBS volume still retains the persisted data. One EC2 instance can have multiple EBS Volumes attached to it.

EBS volumes are locked to an availability zone. They can be attached to any EC2 instances within an availability zone.

However, an EBS volume in one AZ cannot be attached to an EC2 instance in another AZ. To move a volume from one AZ to another, you have to find the following process.

1. Take a snapshot of the EBS Volume.
2. Copy the snapshot to a different region. This can be done from the snapshot menu options on the console.
3. Restore the snapshot to a new EBS Volume that you can create in a different AZ.

Delete on Termination — When EC2 is terminated, if the delete on termination flag is on, the: EBS Volume will also be deleted.

Since these are network drives, there could be some latency in communication between an EC2 instance and the EBS volume.

4. SnapshotsWhen you are taking a snapshot of an EBS Volume, it is recommended that you detach the volume to stop traffic. However, this is not absolutely required.

5. EBS Volume Types:

Different EBS Volumes may be selected based on size and throughputs or IOPS needed.

gp2/gp3 SSD — general purpose volumes that balance price and performance.Can be used as a boot volume. They can go from 3000–16000 IOPS

io1/io2 SSD — highest performance SSD volumes for low latency and high throughput applications. Can be used as a boot volume.If you need >1600 IOPS. Great for database workloads. EC2 nitro with io1/io2 can give you 32000–64000 IOPS. Supports EBS Multi-Attach.

Multi-Attach allows same EBS volume to multiple EC2 instances in the same AZ.

st1 HDD — these are used for high throughput frequent access and low cost. Big Data and Data Warehousing — max throughput 500MiB/s-max IOPS 500

sc1 HDD- Cold HDD lowest cost for less frequently accessed applications. Best for archive data.

6. Encryption: When an encrypted EBS volume is created, all data at rest, in transit, and snapshots are encrypted by default. All the encryption happens automatically. Very low impact on latency so it's recommended to be used. Encryption uses KMS keys with AES-256.

Method for encrypting an unencrypted EBS volume

1, Make an EBS snapshot of the EBS Volume

2. Create a new EBS volume from the snapshot and check “Encrypt this volume”.