Scale an application running on AWS EC2 for a big day event

Wondering how I did it. Let’s go over the experience.

Task : Being an Architect and a member of Cloud Center of Excellence (CCOE) i have been reached out by the Application team to review the Architecture to support the event.

While reviewing the architecture I found the current infrastructure was not capable of supporting bursty workload for the big day.

How did i know it is not capable of supporting bursty traffic

We did extensive testing with a Load testing tool and sent traffic to current infrastructure and as expected the system crumbled and crashed with a lot of requests.

Here is the architecture diagram of Existing Architecture.

Let’s break it down.

We followed a 3 Tier Architecture model.

Web Tier — Accepting all user requests.

Logic Tier — Applying the business logic and interacting with third party systems and AWS Services.

Database Tier — Storing information related to Products, User info, Order, billing etc.

Challenges with existing Infrastructure

1.Not capable of handling traffic for a bursty workload.

2.Load balancer was not optimized for handling heavy traffic.

3.Autoscaling strategies used were Target Tracking which automatically adjusted capacity to maintain a target metric (CPU utilization at 50%).If CPU usage exceeds 50%, auto Scaling increases EC2 instances. Not ideal to support big day traffic events.

4.AMI used to provision new instances used to take more than 60 seconds due to the large number of packages and libraries.

5.Logic tier was hitting the database directly with requests and causing the database to be overwhelmed.

Architecture Diagram to support Big day event

Let’s break it down and see how we used the above architecture to overcome the existing challenges.

1.Load Balancer was Pre-warmed to support the bursty traffic.

2.Autoscaling strategies were updated from Target Tracking to Scheduled Scaling which allowed to pre-warm ec2 instances before the event.

3.Worked with the App team and removed unwanted packages and libraries from code and made the AMI lightweight that allowed Ec2 instances to provision quickly.

4.RDS proxy service was used as a layer between Logic tier and database to stop databases from being overwhelmed.

Would like to highlight Couple of AWS Services that were Game Changers to support this event and it deserves a mention here.

Autoscaling Strategies : Scheduled Scaling

Scheduled Scaling allows you to proactively adjust capacity by setting up scaling actions at specific times, based on predictable demand patterns. Instead of reacting to real-time changes, AWS Auto Scaling increases or decreases resources at predefined times

How Scheduled Scaling Works

1. Define a Schedule: Specify a start time and, optionally, an end time for scaling.
2. Set Desired Capacity: Define the minimum and maximum number of instances or resources.
3. AWS Auto Scaling Adjusts Resources: Instances or resources are automatically adjusted at the scheduled time.

Amazon RDS Proxy

Amazon RDS Proxy is a fully managed database proxy service for Amazon RDS (Relational Database Service) and Amazon Aurora. It helps improve the scalability, security, and performance of applications that use MySQL, PostgreSQL, or SQL Server databases by pooling and sharing database connections efficiently.

Key Benefits of RDS Proxy

1. Connection Pooling

• Reduces the overhead of frequently opening and closing database connections.
• Helps handle large numbers of client connections efficiently.

2. Improved Availability & Failover

• Speeds up failover times in Amazon Aurora and Amazon RDS (reducing from minutes to seconds).
• Ensures that applications remain highly available during database failovers.

3. Better Scalability

• Protects databases from being overwhelmed by too many connections.
• Efficiently reuses existing connections instead of constantly creating new ones.

4. Enhanced Security

• Uses AWS IAM authentication instead of storing database credentials in the application.
• Supports AWS Secrets Manager to securely store and manage credentials.

5. Cost Efficiency

• Reduces database instance costs by optimizing connection usage.
• Minimizes the need to scale up database instances for handling connection limits.

How RDS Proxy Works

• It sits between your application and the database.
• The proxy maintains a pool of established connections to the database.
• When the application requests a connection, it is assigned an existing pooled connection instead of creating a new one.
• It automatically detects and reroutes traffic in case of database failover.

Use Cases

• High-traffic web applications that need efficient database connection management.
• Serverless applications using AWS Lambda that require persistent database connections.

By implementing auto-scaling, load balancing, database optimizations, and real-time monitoring, our infrastructure and application were well-prepared to handle sudden traffic spikes during big events and helped us make our event successful.

Have you ever been asked to support a big day event. How did you do it? Feel free to share your experience.