Basics of Load Balancing with ECMP

By Leonir Hoxha on July 10, 2023 • ( 0 )

IP routing protocols are capable of providing data traffic load balancing with the assumption that multiple paths exist and that the cost of each path is the same. In a particular routing protocol, unequal cost of different paths can also be considered for load balancing.
Equal-cost multi-path routing (ECMP) is a routing approach that allows packets to be forwarded to a particular destination via several “best paths” that have the same routing metric calculation. This strategy can be applied with most routing protocols since it only affects a single routers decision-making process. By distributing traffic over multiple paths, ECMP has the potential to considerably ‘increase’ bandwidth availability if designed the right way. ECMP routing is supported by a range of routing protocols such as OSPF, ISIS, EIGRP, and BGP. When we talk about Load-balancing, I need to be specific here; I’m not referring to a Load-balancer device that does the same job but mostly focused on the application side (server side); I’m referring to load-balancing from a router perspective.

Depending on the vendor, a router might take a different load-balancing approach, but in most cases, the default model is based on ‘source IP / destination IP’ and in some cases, also includes Layer4 info such as protocol types (TCP/UDP).

The concept of per-destination load balancing is an intriguing one, where a router divides data packets based on their destination addresses. Think of it like this – if two roads lead to the same city, all mail for one address would be routed through the first road, while mail for another address would take the second road. This method maintains the sequence of the packets, although it could lead to uneven use of the available paths. For instance, if one address receives a substantial amount of traffic, it monopolizes one path, leaving other paths underutilized. However, a larger pool of destination addresses could balance out the link usage.
Another approach by optimizing resource allocation, routers can implement a different strategy, known as ‘per-packet’ load balancing. Unlike the per-destination approach, per-packet load balancing divides traffic equally over all available paths, regardless of the destination addresses. This approach ensures a more even distribution of traffic, but it does introduce potential issues with the sequence of packets. Since different paths may have varying latency, packets could arrive at the destination out of order, which could degrade the performance of certain applications, especially real-time applications like voice over IP (VoIP) or video conferencing, online gaming, etc.

To efficiently distribute traffic across multiple paths, routers leverage the concept of hashing. Hashing is a function that takes an input (or ‘message’) and returns a fixed-size string of bytes. In the context of load balancing, hashing algorithms are used to consistently distribute traffic across multiple paths based on certain attributes of each packet, such as source and destination IP addresses, port numbers, and more. This process, also known as ‘tuple’ hashing, ensures that all packets of a particular flow follow the same path, thus maintaining packet order while still spreading traffic evenly.

Load-balancing strategies are not one-size-fits-all, and selecting the right approach depends on the specific requirements and constraints of the network. Factors like the number of paths, their capacity and latency, the type and distribution of traffic, and the tolerance of different applications to latency and packet reordering can all influence the choice of strategy. In some cases, a combination of different approaches might be needed to achieve the best balance between performance, reliability, and resource utilization.
Understanding load-balancing mechanisms is a key in today’s interconnected world and one that will continue to be important as network complexity and traffic volumes continue to grow.

Usually, I tend to be very techie in terms of providing configs and outputs from a router perspective, but this time I thought about writing from an architectural high-level point of view regarding ECMP and the ‘most used’ implementation methods.