Courses taken :
-
Andrew Brown from Exam Pro - https://www.youtube.com/watch?v=Ia-UEYYR44s
-
Stephan Maarek's AWS Solution Architect on Udemy
* 65 Questions - MCQ (1 out of 4) and Multiple Response (any out of 4)
* 130 Mins (2h 10 Mins)
* ~72% Passing Score (Max points : 1000, Min. Points : 720) equivalent to a C- grade.
* 3 year Valididity
* 150 USD
* Scoring : Unanswered questions are scored as incorrect; there is no penalty for guessing.
Content Outline of the Exam Syllabu with weightage and services hinted at. Link to PDF
- Elastic Block Storage, S3 and other storage options.
- Application integration like SQS and SNS
- Generally, solutions have 3 layers :
- Data Layer
- Web Layer
- Load Balancing Layer
- Route53, Loadbalancing, AutoScaling Groups.
- What happens when an AZ goes out?
- What happens when a Region goes out?
- DynamoDB vs RedShift vs RDS
- ElasticCache
- CloudFront
- AutoScaling Groups
- How and when to secure the 3 layers : Data, Web and Network(Load Balancing)
- Data at Rest or
- Data at Transit
- Knowing VPCs inside and out.
- Mainly S3 service.
- S3 has a bunch of classes that we can use with different pricing. Knowing when to use which.
- When to use different type of EC2 instances?
- When to use AutoScaling groups?
- Choose design features that enable operational excellence
- AWS Best Practises
- AWS Well-Architected Framework
--
- Users : Mapped to a physical user. Has a password.
- Groups : Logical entities that contain User. One user can be in more than one group.
- Policies : Set of rules (like a JSON Document) describing the access privileges of a Group/User/Role.
- A statement in an IAM Policy consists of
Sid, Effect, Principal, Action, Resource, and Condition
. - Version is part of the IAM Policy itself, not the statement.
โ ๏ธ
- A statement in an IAM Policy consists of
-
Roles : An IAM Entity that defines a set of permissions for making requests to AWS Services, and will be used by an AWS Service.
- Provide access privileges to AWS services to perform actions on behalf of Users.
-
Security : MFA + Password policy
-
Access Keys : Need these keys to access AWS via CLI or SDK.
- Access Key is like your username
- Secret access key is like your password
-
Auditing :
- IAM Credentials Report (account level info)
- IAM Access Advisor (user level info)
-
Other Important Stuff : โ๏ธ
- Principle of Least Privileges.
Sample Codes used :
#!/bin/bash
# Use this script for user data
# install httpd
yum update -y
yum install -y httpd
systemctl start httpd
systemctl enable httpd
echo "<h1> Hello World from $(hostname -f)</h1>" > /var/www/html/index.html
- Solving the 'compute' problem.
- EC2 = Elastic Compute Cloud = Infrastructure as a Service
- Not just 1 service, it's a collection of services :
- Renting virtual machines (EC2)
- Storing data ono virtual drives (EBS)
- Distributing load across machines (ELB)
- Scaling the services (ASG)
- EC2 Sizing and Config Options
- OS - Linux, Windows, Mac OS
- CPU
- RAM
- Storage Space:
- Network Attached (EBS and EFS)
- Hardware (EC2 instance store)
- Network : Speed, Public IP Address
- Firewall Rules :
Security Groups
- Bootstrap Script (configure at first launch) :
EC2 User data
- Bootstrapping means launching commands when a machine starts
- The boostrap script is only run once at the instance first start.
- Commands like : Installing Updates, Softwares, Downloading files from internet, etc.
root user
used too execute bootstraping script.
Questions : "Which type of instance is the best given the nature of workloads?"
-
On-Demand Instance (OD)
- Pay for what you use. Linux/Windows - pay per second, after 1st min. Other OSs - billing per hour.
- Highest Cost, but no upfront payment.
Short-term, irregular and un-interrupted workloads
- No long term commitment.
-
Reserved Instance (RI)
- 72% cheaper vs On-Demand. Reserve specific instance attributes (Instance Types, Region, Tenancy, OS)
- Reservation Features:
- Period - 1 year or 3 years ('Standard' Reserved)
- Payment Options - No Upfront(+), Partial Upfront(++), All Upfront(+++)
- Scope - Regional or Zonal (reserved capacity in an AZ)
Recommended for steady-state usage applications (think database)
- Can buy and sell in the reserved instance market place when not in use anymore.
- Post the Reserved instance usage, pricing returns to On-Demand rates.
- 'Convertible' Reserved : Flexible on EC2 instance type, family OS, scope and tenancy. Less Discount - 66%.
-
Saving Plan Instance (SP)
- Discount on Long-term usage. (upto 72% - same as RIs)
- Commitment required : Consistent usage $10/Hr for 1 or 3 years
- Usage beyond commitment is priced @ On-Demand Rates.
- Locked to instance family & AWS Region. (Eg. M5 in us-east-1). Flexible on Instance Size, OS and Tenancy (Hst Dedicated, Default)
-
Spot Instance
- Discount upto 90% vs On-Demand
- Most cost efficient. Can lose the instance anytime.
Workloads with flexible start and end times that can withstand interruptions.
- Eg : Batch Jobs, Img Processing. Not recommended for Databases.
- Interesting :
- User defines max spot price that they are willing to pay, then get the instance while
current price < user-defined max price
. - If
current price > user-defined max price
, then stop or terminate your instance with a 2 min grace period.
- User defines max spot price that they are willing to pay, then get the instance while
- Spot Fleets: Ultimate way to save money.
- Fleet = set of Spot Instances _ optional On-Demand Instance.
- Fleet tries to meet the target capacity with price constraints:
- Define possible launch pools : instance type, OS, AZ
- Can have multiple launch pools, so the Fleet can choose.
- Fleet stops launching new instances when reaching capacity or max cost.
- Fleet Strategies:
- lowestPrice - pick from pool with the lowest price. (cost optimize, short workload)
- diversified - distributed across all pools (great for availability, long workloads)
- capacityOptimized: pool with optimal capacity
-
Dedicated Host and Instance (DH)
- HOST
- A physical servers with EC2 instance capacity fully dedicated to your use.
- Useful to address compliance requirements and use your existing server-bound software licences (per-socket, per-core, per VM S/W Licenses)
- Purchasing Options : On-Demand pay per second for active dedicated host. Reserved 1 or 3 years (All upfront only)
- Most Expensive.
- Useful for software that have complicated licensing model. (BYOL - Bring Your Own License). _Eg: Oracle Netsuite, Adobe Photoshop, etc.
- INSTANCE
- Instances run on h/w that's dedicated to you.
- May share h/w with other instances in same account.
- No control over instance placement.
- HOST
-
Capacity Reservation Instances
- Reserved On-Demand instances capacity in a specific AZ for any duration.
- No time commitment. No billing discounts.
- Combine w/ Regional Reserved
- Charged @ On-Demand rates whether use or not.
Which room is right for me?
CheatSheet
Use to control where and how my EC2 instances get deployed(or placed).
- Cluster : Clusters instances into a low-latency network. Same Rack. Same AZ
- Great Network. But, if the rack fails, then all the instances go down.
- Use Case : Big data jobs that need to complete fast. App that needs extremely low latency and high network throughput. high performance
- Spread : Instances spread across different hardware (in a server rack).
- Reduced risk of simultaneous failure. If H/W in AZ1 fails, then less likely h/w will fail in AZ2.
- Limited to only 7 instances per AZ per placement group.
- Use Case : Apps that need high availability. Critical apps where each instance must be isolated from failures of each other.
critical
- Partition : Instances spread across different partitions (i.e. server racks) within an AZ.
- Reduced risk of 'rack failure'. If rack in AZ1 fails, then less likely h/w will fail in AZ2.Upto 100s of EC2 instances.
- A partition failure will affect EC2 instances on that rack only.
- Use Case : HDFS, HBase, Cassandra, etc.
distributed
- Amazon Machine Image : Customization of an EC2 instance.
- Add your own OS, Software, config, etc.
- Faster boot times since all the software is pre-packaged.
- Built for a specific region. Can be copied though.
- EC2 instances can be launched from :
- Public AMI
- Your own AMI
- An AWS Mktplace AMI : Other people create and sell these AMIs.
- 'Store' as in storage.
- EC2 instance is a virtual machine, but it is attached to real hardware in a server. Sometimes, these servers have a hard disk attached to them.
- Use EC2 Instance Store if you need a high performance hardware disk.
- Better I/O performance
- EC2 Instance Store lose their storage, when the instance is stopped. ephemeral storage
- Good for buffer/cache/scratch data/temp content.
- Risk of data loss, if hardware fails.
- Backup and Replication of data on Instance Store is our responsibility.
- EC2 Nitro - Underlying platform for the next-gen EC2 instances. New virtualisation tech. Better performance (64k IOPS on EBS) and security.
- vCPU - EC2 instances come with a combination of RAM and vCPU. 1 thread on a CPU = 1 vCPU. If a CPU has 2 cores with 2 threads per core, then it means 4 vCPU. If you are being charged by a licensing s/w on the basis of #of vCPUs, then you can reduce the number of threads or cores that run on a EC2 instance. This configuration happens only at the instance launch.
- Capacity Reservations - Ensure you have capacity when you need it. Reservations have a manual or planned ending date. No need oof 1 or 3 year commitment. You are billed as soon as the capacity is reserved. AZ, Instance Type, OS must be specified.
- Kinda like 'Firewalls' on EC2 instances.
- Important for network security in AWS
- Security groups only contain ALLOW rules.
- Rules : Control how traffic is allowed into or out of our EC2 instances.
- Access to ports.
- Authorised IP ranges - IPv4 and IPv6
- Control of inbound network (outside to instance).
- Control of outbound network (instance to outside).
- Default Rules : All inboound traffic is blocked. All outbound traffic is allowed.
- Important Points
- Can be attached to multiple instance. An instance can have multiple security groups attached.
- Locked to a region/VPC comobination. If we change regions, we will have to create different SG.
- SGs do not run on EC2. They lie outside and if they block some data, then the underlying EC2 won't see it.
- Example :
There is a EC2 machine called
X1
with 2 security groups attached to it calledSG1
andSG2
. Denoted byX1[SG1, SG2]
. There are other EC2 machines calledX2
andX3
with security groups SG1 and SG2 attached, respectively. Denoted byX2[SG1]
,X3[SG2]
.
The EC2 machines with same SG can intereact with each other. i.e. X2
and X3
can signal X1
since they have same SGs.
X2[SG1] -> X1[SG1, SG2]
and X3[SG2] -> X1[SG1, SG2]
- Ports :
- 22 = SSH into Linux
- 21 = FTP
- 22 = SFTP
- 80 = HTTP
- 443 = HTTPS
- 3389 = RDP into Windows instance
-
Two sorts of IP formats :
- IPv4-
1.160.212.240
, most common form. Format : [0-255].[0-255].[0-255].[0-255] - 3.7 Bn addresses. - IPv6 -
1900:4231:3:204:g8he:ir59:40ge
, usually meant for IoT.
- IPv4-
-
Types : | Public IP | Private IP | |--------------------------------------|----------------------------------------------------------------------------| | Machines can be ID'd on the internet | Can be ID'd on a private n/w only. | | Must be unique across the whole web | IP must be uniques across a private network (2 pvt n/w can have some IPs) | | Can be geo-located | Machines connect to Internet via an internet gateway. |
-
Elastic IPs
- The IP of an EC2 instance changes whenever we restart it. Elastic IP provides a fixed IP for your instance.
- Can be attached to only 1 instance at a time. If that EC2 instance fails, we can rapidly remap it to another functional instance.
- Only 5 Elastic IP in your account.
Best Practices Try to avoid using Elastic IP
- Instead, use a random public IP and register a DNS name to it.
- Better, use a Load Balancer and don't use a public IP.
- Logical component in a VPC that represents a virtual network card (A NIC (Network Interface Card) provides the hardware interface between a computer and a network).
- ENI have 4 attributes :
- Primary private IPv4, one or more secondary IPv4.
- One Elastic IP(Public) per private IPv4.
- One or more security groups.
- A MAC Address
- ENI can be created independently and attach and detach them to instances on the fly.
- ENIs are AZ-bound so an ENI in AZ1 can't be attached to a EC2 in AZ2.
-
It's a network drive (kinda like "a network pendrive") you attach to your instances.
-
Allows your instance to persist data, even after termination.
-
Can be mounted/attached to only one EC2 instance at a time.
- Can be detached and attached to other EC2 instance.
-
AZ-Bound.
-
EBS volumes can exist in the unattached to any EC2 instance.
- Connected via network so there will be some latency.
- Storage must be provisioned. Define the capacity at the time of initialisation.
- You will be billed for the provisioned capacity.
-
Delete on termination attribute:
- By default, the root EBS volume is deleted when an EC2 gets terminated.
- But, the attached EBS volume doesn't get deleted.
-
__EBS Snapshots - __
- Make a backup of EBS volume at a point in time. Recommended to detach before snapshotting.
- Can copy across AZ or Region.
- Snapshot Archive Snapshots can be archived to "archived tier". 75% cheaper. Restoring takes 24 - 72 hours.
- Recycle Bin to retain the deleted EBS volumes for 1 day till 1 year to recover any accidental deletions.
-
EBS Volume Types
- EBS classification basis - Size | Throughput | IOPS
- gp2/gp3 (SSD) : General purpose SSD volume. Balances price and performance.
Boot Volume
. - io1/io2 (SSD) : Highest-performance SSD volume for mission-critical low-latency or high-throughput workloads.
Boot Volume
.Provisioned IOPS
Multi-Attach
- st1 (HDD) : Low cost HDD. Frequently accessed, throughput-intensive workloads.
- sc1 (HDD) : Lowest cost HDD. Less frequently accessed workloads.
-
EBS Encryption
- What is encrypted? Data at rest, Data inflight between instance and volume, snapshots, all volumes created from the snapshot.
- Encryption is recommended due to minimal impact on latency.
- Uses KMS keys (AES-256).
- Copying an unencrypted snapshot can be encrypted.
- How to encrypt an un-encrypted EBS volume? Create a snapshot -> Encrypt the snapshot -> Create a volume using the snapshot. -> The volume will be encrypted.
- Managed NFS(Network File System) that can be mounted on multiple EC2.
- Works with multi-AZ. An EFS can be connected to EC2 instances in multiple AZs.
- Highly available, scalable, expensive, pay-per-use.
- Use cases: Content Mgmt, Web serving, data sharing, wordpress.
- Security : Uses NFSv4.1 protocol. Uses Security Group to control access. Encryption at rest using KMS.
- Compatible with only Linux based AMIs.
- Uses POSIX file system (same as Linux).
- File system scales automatically, so no need to provision and plan capacity!
Scale:
- 1000s of concurrent NFS clients, 10GB+/s throughput
- Grows to petabyte-scale file network file system, automatically. Performance Mode: (configured at creation time)
- General purpose(default): latency-sensitive use cases (web server, CMS, etc.)
- Max I/O - higher latency, throughput, highly parallel (big data, media processing..) Throughput Mode:
- Bursting (1TB = 50MBps + burts oof uto 100MBps) Scales with the size of file system.
- Provision: set a throughput regardless of storage size. Storage Tiers:
- Standard : Frequently accessed files.
- Infrequest Access (EFS-IA) : Cheap storage, retrieval. Enable EFS-IA with a Lifecycle Policy.
- Lifecycle Policy is a set oof rules to define the scope of storage of data based on it's usage pattern. Availability:
- Regional : muti-AZ, great for production.
- One Zone : great for dev, backup enabled by default. Compatible with IA. 90% cheaper.
EFS Exam Questions: When should you use EFS? How to configure it to maintain compliance?
Scalability and Availability Scalability means "Application can handle greater load by adapting." There are two kinds of scalability:
- Vertical Scalability- Increasing the size of the instance eg - add more RAM, etc.
- Instance sizes can vary from
t2.nano
(0.5GB ram/1vCPU) to12tb1.metal
โ 12.3 TB of RAM, 448 vCPUs.- Horizontal Scalability (= elasticity) - Increasing the number of instances / systems. Implies distributed systems.
- AutoScaling Group
- Load Balancer
Availability means "Application can survive a data center loss". Achieved by running instances for the same application across multiple AZs. Availability and Horizontal Scalability go hand in hand.
- Auto Scaling Group multi AZ
- Load Balancer multi AZ
What is Load Balancer? Load Balancers are servers that forward and distribute traffic to multiple servers (e.g. EC2 instances) downstream. In other words, as the web traffic increases, the ELB will redirect the traffic to the different instances to distribute the traffic(load) evenly across the instances.
Why use ELB?
- Spread load across multiple downstream instances.
- Expose single point of access (DNS) to your application.
- Seamlessly handle failures of downstream to your instances.
- Regular health checks on instances.
- Provide SSL termination (HTTPS) for your websites.
- Enforce stickiness with cookies.
- High availability across zones.
- Separate public traffic from private traffic.
Features
- ELB is a managed service. i.e.
- Managed Services mean AWS handles availability, upgrade, maintenance, etc.
- Costs less to setup your own Load Balancer, but too much hassle to manage it.
- AWS ELB comes integrated with many AWS offerings / services :
- EC2, EC2 Auto Scaling Groups, Amazon ECS
- AWS Certificate Manager, Cloud Watch.
- Route 53, AWS WAF, AWS Global Accelerator.
Health Checks ๐
- Crucial for ELBs.
- Notify Load Balancers if the instances are available to reply to requests.
- Health Check is done on a port and a route (/health is common)
- If the response is not 200 (OK), then the instance is unhealthy and the load balancer won't send traffic to that instance.
If the EC2 instances and the Load Balancers communicate toogether, how do they secure the traffic?
Since the ELB fronts the EC2 instances, it faces all the traffic from outside. So, it's security group rules will look like below:
SG-ELB-1 : Security Group Rules for ELB
Allow HTTP from 0.0.0.0, Port 80.
Allow HTTPS from 0.0.0.0 , Port 443.
Whereas, the instance talks to only the ELB as it receives the outside traffic from it. So, it security group rules will look like below :
SG-Ec2-A : Security Group Rules for an EC2 instance connected to ELB.
Allow HTTP from SG-ELB-1. Port 80
AWS has 4 kinds of Load Balancers:
Load Balancer | Year | Protocol |
---|---|---|
Classic Load Balancer | 2009 | HTTP, HTTPS, TCP, SSL |
Application Load Balancer | 2016 | HTTP, HTTPS, WebSocket |
Network Load Balancer | 2017 | TCP, TLS(Secure TCP), UDP |
Gateway Load Balancer | 2020 | Operates at layer 3 (Network Layer) - IP Protocol |
Recommended to use the newer generation Load Balancers. Some LBs can also be setup as internal or private ELBs.
- Supports TCP (Layer 4), HTTP & HTTPS (Layer 7)
- Health Checks - TCP or HTTP based.
- Fixed hostname XXX.region.elb.amazonaws.com