Strategic Re-routing with Equal-Cost Multi-Path (ECMP) – New in SonicOS 6.5 for Firewalls
As intranet networks grow and evolve over time, often duplicate, or even multiple, paths are created to reach a destination. As these paths evolve and get more complex, they can result in failed links. Interior Gateway Protocols provide fast re-routing around these failed links using link-state algorithms, such as Open Shortest Path First (OSPF) and Intermediate System to Intermediate System (IS-IS). Enterprise networks deploy OSPF much more often. However, I have seen carrier networks who prefer IS-IS, especially when acquiring other networks’ addresses.
Link-state algorithms do an excellent job of fast re-routing inside their areas due to their detection of link failure, and due to each Layer 3 device having a topology of their intra-area network. (Outside of that intra-area, the networks require more of a distance vector routing protocol. But that is for another blog). Link-state algorithms also give us the ability to take into consideration speed of links or costing when determining the best path. This comes in handy when doing prefix evaluation, but it also can give us the ability to have multiple equal-cost paths to a destination.
Equal-Cost Multi-Path (ECMP), which is supported in SonicOS 6.5 for SonicWall’s next-gen firewalls, is an egress routing method used when you have multiple interfaces pointing to a destination. Equal cost routes are added to the connection cache for session setup. As sessions are created, SonicWall hashes the packet 5-tuple in the TCP header to decide which path the session will egress to the next hop. A 5-tuple is comprised of a source IP address, source port number, destination IP address, destination port number and the TCP protocol. Do not confuse this with per-packet load-balancing. That was tried many years ago, and caused out-of-sequence packets. Large packets followed by smaller packets would egress faster, and would break applications, despite being part of the TCP specifications. This is why you want to have sessions stay on the interface, as opposed to multiplexing packets over the interfaces you have configured with ECMP.
So, what do you want to look out for when designing a network with ECMP?
First off, who is your downstream neighbors, and how are they configured? I mentioned how ECMP is an egress routing method. Typically, you would use ECMP when you are not connecting multiple interfaces to the same devices. The connections are not 1:1 from Device A to Device B, but rather Device A to Device B/C/D, etc. You would use some type of link aggregation for this design.
If your downstream device is a session-aware device, such as a firewall, it may see the source prefix and report that it has detected IP Spoofing. This is due to the arrival of a packet from a source that is not consistent with the routing table. For example, if the firewall expects 1.1.1.1 should come from X4, but instead sees it on X3, it would report IP Spoofing.
Two other scenarios could also trigger an IP Spoofing message in the firewall log that drops the session. One is if you have a router and are performing Reverse Path Forwarding checking to create a loop-free multi-cast network. Another is if are truly looking for malicious spoofed-source IP addresses.
Another possible scenario I’ve seen before is where, after the hashing of the 5-tuple has occurred, the balance of sessions puts the sessions on one interface. It’s the result of another ECMP hash that has been performed on the 5-tuple prior to receiving those sessions. Since the hash calculation has already been performed, and the device has been given one set of sessions that were derived from the hash value, when we hash again they have the same value, hence, they land on the same interface. A quick fix is to have the upstream device modify the 5-tuple down to four. This lets the downstream device have a different value on the TCP header.
Ultimately, if you account for these potential issues, ECMP offers a great way to utilize multiple paths in a dynamic network and maximize investment in your infrastructure.
This is just one of the 60 new features in SonicOS 6.5 for all of SonicWall next-gen firewalls. Want to learn more? Check out a new video on SonicOS 6.5.
