Elastic Load Balancing (ELB)

1. Overview

Load Balancers are servers that forward internet traffic to multiple servers (EC2 Instances) downstream. LBs can scale but not instantaneously.

1.1 Why use a load balancer

• Spread load across multiple downstream instances
• Expose a single point of access (DNS) to your application
• Seamlessly handle failures of downstream instances
• Do regular health checks to your instances
• Provide SSL termination (HTTPS) for your websites
• Enforce stickiness with cookies
• High availability across zones
• Separate public traffic from private traffic

1.2 Why use an EC2 Load Balancer

• An ELB(EC2 Load Balancer) is a managed load balancer
  • AWS guarantees that it will be working
  • AWS takes care of upgrades, maintenance, high availability
  • AWS provides only a few configurations knobs
• It costs less to set up your own load balancer but it will be a lot more effort on your end
• It is integrated with many AWS offering/services

1.3 Health Checks

• Health Checks are crucial for Load Balancers
• They enable the load balancer to know if instances it forwards traffic to are available to reply to requests
• The health check is down on a port and a route (/health is common)
• If the response is not 200(OK), then the instance is unhealthy

1.4 Types of load balancer on AWS

• Classic Load Balancer (v1 - old generation, 2009)
  • HTTP, HTTPS, TCP
• Application Load Balancer (v2 - new generation, 2016)
• Network Load Balancer (v2 - new generation, 2017)
• Overall, it is recommended to use the newer / v2 generation load balancers as they provide more features
• You can set up internal(private) or external(public) ELBs

1.5 Load Balancer Security Groups

1.6 Troubleshooting and Monitoring

1.6.1 Troubleshooting

• 4xx errors are client induced errors
• 5xx errors are application induced errors
• Load Balancer Errors 503 means at capacity or no registered target
• If the LB can't connect to your application, check your security groups

1.6.2 Monitoring

• ELB access logs will log all access requests (so you can debug per request)
• CloudWatch Metrics will give you aggregate statistics

2. Classic Load Balancer (v1)

• Supports TCP (Layer 4), HTTP & HTTPS (Layer 7)
• Health checks are TCP or HTTP based
• Fixed hostname (xxx.region.elb.amazonaws.com)

3. Application Load Balancer (v2)

• Application load balancers are Layer 7 (HTTP)
• Load balancing to multiple HTTP applications across machines (target groups)
• Load balancing to multiple applications on the same machine(ex: containers)
• Support for HTTP/2 and WebSocket
• Support redirects (from HTTP to HTTPS for example)
• Routing tables to different target groups:
  • Routing based on path in URL (example.com/users & example.com/posts)
  • Routing based on hostname in URL (one.example.com & other.example.com)
  • Routing based on Query String, Headers (exmaple.com/users?id=123&order=false)
• ALB is a great fit for microservices & container-based application(example: Docker & Amazon ECS)
• Has a port mapping feature to redirect to a dynamic port in ECS
• In comparison, we'd need multiple Classic Load Balancer per application
• Fixed hostname (XXX.region.elb.amazonaws.com)
• The application servers don't see the IP of the client directly
  • The true IP of the client is inserted in the header X-Forwarded-For
  • We can also get Port (X-Forwarded-Port) and proto (X-Forwarded-Proto)

3.1 Target Groups

• EC2 instances (can be managed by an Auto Scaling Group) - HTTP
• ECS tasks (managed by ECS itself) - HTTP
• Lambda functions - HTTP request is translated into a JSON event
• IP Addresses - must be private IPs
• ALB can route to multiple target groups
• Health checks are at the target group level

4. Network Load Balancer (v2)

• Network load balancers (Layer 4) allow to:
  • Forward TCP & UDP traffic to your instances
  • Handle millions of request per seconds
  • Less latency ~ 100ms (vs 400 ms for ALB)
• NLB has one static IP per AZ and supports assigning Elastic IP (helpful for whitelisting specific IP)
• NLB is used for extreme performance, TCP or UDP traffic
• Not included in the AWS free tier

5. Load Balancer Stickiness

• It is possible to implement stickiness so that the same client is always redirected to the same instance behind a load balancer.
• This works for Classic Load Balancers and Application Load Balancers.
• The "cookie" used for stickiness has an expiration date you control.
• Use case: Make sure the user doesn't lose his session data
• Enabling stickiness may bring imbalance to the load over the backend EC2 instances

6. Cross-Zone Load Balancing

• With Cross Zone Load Balancing: each load balancer instance distributes evenly across all registered instances in all AZ
• Otherwise, each load balancer node distributes requests evenly across the registered instances in its Availability Zone only

6.1 Classic Load Balancer

• Disabled by default
• No charges for inter AZ data if enabled

6.2 Application Load Balancer

• Always on (can't be disabled)
• No charges for inter AZ data

6.3 Network Load Balancer

• Disabled by default
• You pay charges ($) for inter AZ data if enabled

7. SSL Certificates

• The load balancer uses an X.509 certificate (SSL/TLS server certificate)
• You can manage certificates using AWS Certificate Manager (ACM)
• You can create upload your own certificate alternatively
• HTTPS listener:
  • You must specify a default certificate
  • You can add an optional list of certs to support multiple domains
  • Clients can use Server Name Indication(SNI) to specify the hostname they reach
  • Ability to specify a security policy to support older versions of SSL/TLS (legacy clients)

7.1 Server Name Indication (SNI)

• SNI solves the problem of loading multiple SSL certificates onto one web server (to serve multiple websites)
• It's a "newer" protocol, and require the client to indicate the hostname of the target server in the initial SSL handshake
• The server will then find the correct certificate, or return the default one
• Only works for ALB, NLB, and Cloudfront. But not work for CLB

7.2 Classic Load Balancer (v1)

• Support only one SSL certificate
• Must use multiple CLB for multiple hostnames with multiple SSL certificates

7.3 Application Load Balancer (v2)

• Supports multiple listeners with multiple SSL certificates
• Uses Server Name Indication (SNI) to make it work

7.4 Network Load Balancer (v2)

• Supports multiple listeners with multiple SSL certificates
• Uses Server Name Indication (SNI) to make it work

8. Connection Draining

• CLB: Connection Draining
• Target Group: Deregistration Delay (for ALB & NLB)

• Time to complete "in-flight requests" while the instance is de-registering or unhealthy.
• Stops sending new requests to the instance which is de-registering
• Between 1 to 3600 seconds, default is 300 seconds
• Can be disabled (set value to 0)
• Set to a low value if your request is short

9. Auto Scaling Group

The goal of an Auto Scaling Group (ASG) is to:

• Scale-out (add EC2 instances) to match an increased load
• Scale-in (remove EC2 instances) to match a decreased load
• Ensure we have a minimum and a maximum number of machines running
• Automatically Register new instances to a load balancer

• Scaling policies can be on CPU, Network, and so on. These can even be on custom metrics or based on a schedule (if you know your visitor's patterns)
• ASGs use Launch configurations or Launch Templates (newer)
• To update an ASG, you must provide a new launch configuration/launch template
• IAM roles attached to an ASG will get assigned to EC2 instances
• ASGs are free. You pay for the underlying resources being launched
• Having instances under an ASG means that if they get terminated for whatever reason, the ASG will automatically create new one as a replacement. Extra safety.
• ASG can terminate instances marked as unhealthy by an LB (and hence replace them)

9.1 Attributes

• A launch configuration
  • AMI + Instance Type
  • EC2 User Data
  • EBS Volumes
  • Security Groups
  • SSH Key Pair
• Min Size / Max Size / Initial Capacity
• Network + Subnets Information
• Load Balancer Information
• Scaling Policies

9.2 Auto Scaling Alarms

• It is possible to scale an ASG based on CloudWatch alarms
• An Alarm monitors a metric such as Average CPU
• Metrics are computed for the overall ASG instances
• Based on the alarm:
  • We can create scale-out policies which increase the number of instances
  • We can create scale-in policies which decrease the number of instances

9.2.1 Auto Scaling New Rules

It is now possible to define better auto-scaling rules that are directly managed by EC2. These rules are easier to set up and can make more sense

• Target Average CPU Usage
• Number of Requests on the ELB  per instance
• Average Network In
• Average Network Out

9.2.2 Auto Scaling Custom Metric

• We can auto-scale based on a custom metric (ex: number of connected users)
  • Send custom metric from an application on EC2 to CloudWatch
  • Create CloudWatch alarm to react to low/high values
  • Use the CloudWatch alarm as the scaling policy for ASG

9.3 Scaling Policies

• Target Tracking Scaling
  • Most simple and easy to set-up
  • Example: I want the ASG CPU to stay at around 40%
• Simple/Step Scaling
  • When a CloudWatch alarm is triggered (example CPU > 70%), then add 2 units
  • When a CloudWatch alarm is triggered( example CPU < 30%), then remove 1
• Scheduled Actions
  • Anticipate a scaling based on known usage patterns
  • Example: increase the min capacity 10 at 5 pm on Fridays

9.4 Scaling Cooldown

• The cooldown period helps to ensure that your Auto Scaling group doesn't launch or terminate additional instances before the previous scaling activity takes effect.
• In addition to default cooldown for Auto Scaling group, we can create cooldowns that apply to a specific simple scaling policy
• A scaling-specific cooldown period override the default cooldown period
• One common use for scaling-specific cooldowns is with a scale-in policy. (a policy that terminates instances based on a specific criteria or metric.) Because this policy terminates instances, Amazon EC2 Auto Scaling needs less time to determine whether to terminate additional instances.
• If the default cooldown period of 300 seconds is too long, you can reduce costs by applying a scaling-specific cooldown period of 180 seconds to the scale-in policy.
• If your application is scaling up and down multiple times each hour, modify the Auto Scaling Group cool-down timers and the CloudWatch Alarm Period that triggers the scale in.

9.5 Rules of Terminating

9.5.1 Default Termination Policy

• Find the AZ which has the most number of instances
• If there are multiple instances in the AZ to choose from, delete the one with the oldest launch configuration
• ASG tries the balance the number of instances across AZ by default

9.6 Lifecycle Hooks

• By default as soon as an instance is launched in an ASG it's in service
• You have the ability to perform extra steps before the instance goes in service(Pending state)
• You have the ability to perform some actions before the instance is terminated(Terminated state)

9.7 Launch Template vs Launch Configuration

9.7.1 Both

ID of the Amazon Machine Image(AMI), the instance type, a key pair, security groups, and the other parameters that you use to launch EC2 instances(tags, EC2 user-data)

9.7.2 Launch Configuration(legacy)

Must be re-created every time

9.7.3 Launch Template(newer)

• Can have multiple versions
• Create parameters subsets (partial configuration for re-use and inheritance)
• Provision using both On-Demand and Spot instances(or a mix)
• Can use T2 unlimited burst feature
• Recommended by AWS goring forward

10. Reference

https://medium.com/@stephane.maarek

https://docs.aws.amazon.com/