System Design — Part 2
Load Balancing.
Load Balancer (LB) is a critical component of any distributed system. It helps to spread the traffic across a cluster of servers to improve responsiveness and availability of applications, websites or databases. LB also keeps track of the status of all the resources while distributing requests. If a server is not available to take new requests or is not responding or has elevated error rate, LB will stop sending traffic to such a server.
Typically a load balancer sits between the client and the server accepting incoming network and application traffic and distributing the traffic across multiple backend servers using various algorithms. By balancing application requests across multiple servers, a load balancer reduces individual server load and prevents any one application server from becoming a single point of failure, thus improving overall application availability and responsiveness.
Types
- Layer 4 (L4) load balancers work at the transport level. That means they can make routing decisions based on the TCP or UDP ports that packets use along with their source and destination IP addresses. L4 load balancers perform Network Address Translation but do not inspect the actual contents of each packet.
- Layer 7 (L7) load balancers act at the application level, the highest in the OSI model. They can evaluate a wider range of data than L4 counterparts, including HTTP headers and SSL session IDs, when deciding how to distribute requests across the server farm.
- Global server load balancing (GSLB) can extend the capabilities of either type across multiple data centers so that large volumes of traffic can be efficiently distributed and so that there will be no degradation of service for the end user.
Where to put the load balancer?
Between user client and web server
Between web servers and application servers
Between web server and databases
Benefits
- Users experience faster and uninterrupted service
- Service providers experience less downtime and higher throughput
- System administrators can handle incoming requests while decreasing wait time
- Load balancer benefits from predictive analytics that determine traffic bottlenecks before they happen
Load Balancing algorithms
Round robin method
Round robin is a simple technique for making sure that a virtual server forwards each client request to a different server based on a rotating list. It is easy for load balancers to implement, but does don’t take into account the load already on a server. There is a danger that a server may receive a lot of processor-intensive requests and become overloaded. To overcome this problem there are two available options:t
- weighted round robin: a weight is assigned to each server based on a criteria (e.g. server’s traffic-handling capacity)
- dynamic round robin: a weight is assigned to each server dynamically, based on real-time data about the server’s currrent load and idle capacity.
Least connection method
Whereas round robin does not account for the current load on a server (only its place in the rotation), the least connection method does make this evaluation and, as a result, it usually delivers superior performance. Virtual servers following the least connection method will seek to send requests to the server with the least number of active connections.
Least response time method
More sophisticated than the least connection method, the least response time method relies on the time taken by a server to respond to a health monitoring request. The speed of the response is an indicator of how loaded the server is and the overall expected user experience. Some load balancers will take into account the number of active connections on each server as well.
Least bandwidth method
A relatively simple algorithm, the least bandwidth method looks for the server currently serving the least amount of traffic as measured in megabits per second (Mbps).
IP Hash
Methods in this category make decisions based on a hash of various data from the incoming packet. This includes connection or header information, such as source/destination IP address, port number, URL or domain name, from the incoming packet.
Available solutions
This and other components will be studied during the series, start now to follow me and don’t lose new upcoming parts.
