The Method to Manage Traffic

January 19, 2025 · 7 min read · Page View:

Load Balancer, Reverse Proxy, Forward Proxy, API Gateway

If you have any questions, feel free to comment below.

Do you know the basic method to manage the traffic? There are four methods: Load Balancers, Reverse Proxies, Forward Proxies, and API Gateways. And they have different features and usage scenarios.

Note: this article is reprint from the article : The System Design Cheat Sheet: Load Balancer, Reverse Proxy, Forward Proxy, API Gateway, the author is Aleksandr Gavrilenko who I really appreciate.

Load Balancers #

A load balancer is a specialized network device or software application designed to optimize the distribution of incoming network traffic across multiple servers or resources, which prevents any single server from becoming a bottleneck. Load balancers achieve this by employing various algorithms to route incoming requests to the most appropriate server intelligently.

Load balancers can also be used within a data center to balance traffic between different components of an application, such as microservices.

Types #

Based on specific needs, load balancing can be performed at the network/transport and application layer of the OSI layers:

Layer 4

Operate at the Transport layer of the OSI model, dealing primarily with TCP and UDP packets. This load balancers route traffic based on source and destination IP addresses and ports. They are relatively simple, fast, and effective for routing user requests to available servers without inspecting the content of the packets. Used when needed:
- High-speed data routing
- Simple load distribution based on IP and port

Layer 7

Operate at the Application layer and can inspect the data packets’ content. This allows them to make more intelligent routing decisions based on HTTP headers, cookies, or application-specific data. Used when needed:
- SSL termination
- Content-based routing
- Application-level decisions like directing users to a specific version of a web page

Layer 2/3

Though less common, some load balancers operate at the Data Link (Layer 2) and Network (Layer 3) levels. These load balancers are generally used in specialized scenarios requiring packet-level routing. Used when needed:

MAC address-based routing (Layer 2)
IP-based routing without port considerations (Layer 3)

GSLB(Global Server Load Balancer) #

Global Server Load Balancing, is designed to distribute user traffic across multiple geographically dispersed data centers. GSLB is primarily based on the Domain Name System (DNS). Based on many factors, the DNS server returns the IP address of the most suitable data center.

Use Cases #

DNS round-robin: Distributes traffic between all data centers in multiple locations.
Geolocation-based DNS: Detect users’ locations and route traffic to the nearest data center to lower latency.
Failover: Send all traffic to a primary data center, but redirect traffic to a secondary data center if the primary becomes inaccessible.

Popular Solutions: F5 BIG-IP DNS, Citrix ADC, AWS Route 53, Cloudflare Load Balancer

Local Load Balancers #

A Local Load Balancer operates within a single data center or cloud region, primarily focusing on distributing incoming traffic among local servers. Its main goal is to optimize resource utilization, maximize throughput, and minimize response time.

Popular Solutions: HAProxy, NGINX Load Balancer, AWS Elastic Load Balancer (ELB), F5 BIG-IP Local Traffic Manager (LTM)

Load-balancing algorithms #

Different algorithms offer various advantages and trade-offs, making them more or less suitable for particular scenarios.

Round Robin #

Distributes incoming requests sequentially and evenly across all available servers cyclically.

Sticky Round Robin #

A hybrid approach that combines Round Robin distribution with session persistence, ensuring that once a user session is established, it remains on the assigned server.

Weighted Round Robin #

Similar to Round Robin, each server is assigned a weighted score, affecting the distribution of requests. Servers with higher weights receive a larger share of the incoming requests.

IP/URL Hash #

This algorithm hashes the client’s IP address to determine the server for routing the request, ensuring session persistence by always directing a specific client’s requests to the same server.

Dynamic Algorithms #

Least Time #

Requests are redirected to the server with the fastest average response time, balancing server load and user experience.

Least Connections #

Requests are redirected to the server with the fewest active connections, requiring additional computation by the load balancer to identify less-busy servers.

Reverse Proxy #

The reverse proxy is a server that sits between clients and a web server, directing incoming requests to appropriate backend servers. The key difference between a reverse proxy and a load balancer is their primary focus. While both can distribute traffic across multiple servers, a load balancer is designed explicitly for this purpose and usually offers more advanced distribution algorithms. A reverse proxy, on the other hand, provides a broader range of functionalities, such as:

Backend Anonymity: Backend servers remain hidden from the external network, protecting against potential vulnerabilities.
DDoS Mitigation: Many reverse proxies have built-in features to shield backend servers from distributed denial-of-service attacks, such as IP deny listing and client connection rate limiting.
SSL Offloading: Handles the decryption of incoming requests and encryption of server responses, relieving backend servers from these computationally intensive tasks.
Data Compression: Reduces the size of server responses for faster data transfer.
Response Caching: Serves previously cached responses to identical requests, improving speed and reducing server load.
Direct Serving of Static Content: Manages the delivery of static files like HTML, CSS, JavaScript, images, and videos directly to the client.
URL/Content Rewriting: Modifies the URL or content before forwarding requests to the backend servers.

Reverse proxies can be helpful even with just one web server or application server.

Popular Solutions: Nginx, Apache HTTP Server (mod_proxy), HAProxy, Squid, Azure Application Gateway

Forward Proxy #

A Forward Proxy is a server that sits between client devices and the Internet, acting as an intermediary for outgoing requests from the client. A forward proxy accepts connections from computers on a private network and forwards those requests to the public internet. It is the single exit point for subnet users accessing resources outside their private network.

The key difference between a forward proxy and a reverse proxy lies in their primary roles and whom they serve.

A forward proxy primarily serves the client’s needs, helping it access blocked or restricted content and providing anonymity.
A reverse proxy, on the other hand, is installed on the server side and manages incoming requests to the server.

A forward proxy is client-focused and provides functions like：

Clients Anonymity: A forward proxy conceals the client’s original IP address, adding an extra layer of security during internet access.
Access Management: Organizations can employ forward proxies to limit access to specific resources, safeguarding sensitive information.
Caching: By caching commonly accessed resources, forward proxies can enhance client internet response times.
Traffic Control: Forward proxies can manage and control network traffic flow, optimizing bandwidth usage.
Logging: Forward proxies can record all outgoing requests and responses, aiding in monitoring and auditing.

Popular Solutions: Squid, Tinyproxy, CCProxy, WinGate

API Gateway #

An API Gateway is a centralized entry point that manages and routes API requests from client applications to appropriate backend services. It acts as a layer of abstraction between the client and multiple backend services, streamlining their interaction. It is a crucial component in modern architecture, especially in microservices-based systems. API gateways offer various functionalities like:

An API Gateway is a centralized entry point that manages and routes API requests from client applications to appropriate backend services. It acts as a layer of abstraction between the client and multiple backend services, streamlining their interaction. It is a crucial component in modern architecture, especially in microservices-based systems. API gateways offer various functionalities like:

Routing: Directs client-originating API requests to the suitable backend service or microservice, guided by established rules and settings.
Authentication and Authorization: Manages user credentials to ensure only approved clients can access services. This includes verification of API keys, tokens, or other forms of identification.
Rate Limiting and Throttling: Safeguards backend services by enforcing client request rate limits or throttling based on pre-configured policies.
Load Balancing
Caching
Request and Response Transformation: Alters incoming and outgoing data, such as data format conversions or header modifications, to maintain compatibility between clients and backend services.
Monitoring
Request and Response Validation
Circuit Breaking: Implements a circuit breaker pattern to prevent a single service failure from compromising the entire system. It monitors service health and can switch to a backup service if needed.
Service Discovery: Identifies available microservices and their locations, allowing clients to interact with them without knowing their specific addresses.
Enhanced Security: Enforces robust authentication and access control measures, bolstering the system’s overall security against unauthorized access.

Popular Solutions: Kong, Amazon API Gateway, Apigee, Azure API Gateway, MuleSoft Anypoint Platform

Conclusion #

Load Balancers primarily focus on distributing incoming traffic across multiple servers to ensure no single server is overwhelmed. They are essential for scalability and high availability but are generally agnostic to the type of content being served.

Reverse Proxies sit in front of web servers and direct client requests to the appropriate backend server. They are server-facing and are often used for caching, SSL termination, and load distribution within an internal network.

Forward Proxies act as intermediaries between clients and servers, often filtering requests, providing anonymity for users, or bypassing geo-restrictions. They are client-facing and are generally used to control outbound traffic.

API Gateways, on the other hand, are specialized types of reverse proxies tailored for API traffic. They offer advanced functionalities like request routing, API composition, rate limiting, and security features such as authentication and authorization.