8 steps to build a scalable and distributed system design for web applications

A highly scalable, available distributed application with low cost and downtime is the major goal for B2C and in some cases, B2B businesses on AWS, Azure, Alibaba or any other cloud platform. Let's discuss the steps to achieve these goals :-

1. Application and Database Server (Closely knit together)

Implement the application and database server on a single node.

2. Vertical Scaling (Stronger and faster)

As the traffic grows, add more CPU and RAM to your system to handle load. This works well but there are following disadvantages:

• There is a limit to how much processing power and memory can be added to a single node.
• The law of diminishing returns kicks in and the value added reduces on every increment.
• Increases impact due to downtime every time node is upgraded.

3. Vertical Partitioning (Divide work)

Now, separate the application server and database server into separate nodes. This increases availability, reduces context switching and enhances flexibility. This partitioning can be done at multiple layers: application, server, database and hardware. As the users increase, you can use step 2 for each on the nodes. The disadvantages are:

• Sub-optimal utilization of hardware resources.
• Has an upper limit in terms of processing.

4. Horizontal Scaling - Application Server (Delegate and Balance)

Add multiple application server nodes and add a load balancer in front of them. The nodes must be identical and perform the same exact tasks. Hardware load balancers are fast and software balancers are customizable. In order to avoid load balancer as a single point of failure introduce master-master mode or master-slave mode load balancer clusters. This is easy to setup and increases availability without any application changes.

For applications need to access the same application server sticky sessions are used. It is better to maintain central session storage or clustered session storage accessible to all application servers. This helps in scaling as the application servers remain stateless.

5. Caching (Respond Fast)

Caching helps is reducing the latency in writing and reading data as the frequently used data is stored in small-sized high speed RAM memory. Caching can be done at multiple layers : web browser, application, database, database query level, object level and content delivery network. Caching strategies used can be write around, write around and write back. As the cache size is small, an effective cache eviction strategy is a must. The eviction can be done using LRU, LFU, FIFO, RR or custom techniques.

6. Horizontal Scaling and Replication - Database Server (Create copies)

Increase the number of database nodes. Each database node has a complete copy of the database without sharing anything with other node. Use master-slave or multi-master configuration to create replicas of database.

The master-slave configuration with slaves as read-replicas is preferred as there are conflict / deadlock possibilities in multi-master configuration. The master-slave configuration read replication can be achieved asynchronous (out-of-band) replication or synchronous (in-band) replication. Former provides a faster response to the client but slaves lag behind the master with an eventual consistency once data is replicated to all the slaves.

As the slaves increase in master-slave configuration, the incremental benefit of introducing an additional slave diminishes. The master database has to issue writes to all the slave databases. The time needed by slaves to process the writes issues by the master reduces the time spent to service the reads issued by clients. Similarly, the single master would have to write to all the slaves to keep data consistent with the latest updates. Hence, as the number of slaves increase, the number of writes by master increases, reducing the processign power available to accept writes from users.

7. Sharding and Federation - Database Server (Divide and Conquer)

Implement sharding and/or federation to reduce the load on a single node.

Sharding a database consists of row-wise partitioning a database. Each shard has the same schema but different set of rows and reside on a different node. A cluster of nodes can be created for each shard as well for replication. Sharding can be performed using techniques based on range, time, hash based, random or round robin. This method needs a lookup map (expect for hash based technique) which may need a separate database with proper replication to avoid a single point of failure.

Federation is a method to create a group of tables in a database and put each group on a different node. A cluster of nodes with a single group of tables can be created for high availability and reliability. The application code logic is needed to access a table in correct location. This method has disadvantages of not able to create joins and referential integrity concerns.

8. Reverse Proxy (Preserve and Protect the core)

Add a reverse proxy as a single interface for users. This can be added at any step. A reverse proxy provides an uniform interface and receives requests from users. It redirects requests internally to different servers. It can redirect static content to different server, implement collapsed forwarding, cache responses, hide backend servers, provide protection from DoS attacks and implement session management.

At this point, we have a distributed, reliable and high available application with almost infinite scalability.

Looking forward for more techniques and conversations in the comments section....

#scalable #available #distributed #systemdesign