Routing and Performance tuning for Stingray Traffic Manager on Linux

[#1]

Before reading this document, please refer to the documents Basic performance tuning for Stingray Traffic Manager on Linux and Advanced performance tuning for Stingray Traffic Manager on Linux.

This document summarizes some routing-related kernel tunables you may wish to apply to a production Stingray Traffic Manager instance.  It only applies to Stingray Traffic Manager software installed on a customer-provided Linux instance; it does not apply to the Stingray Traffic Manager Virtual Appliance or Cloud instances.

Note: Tech tip: How to apply kernel tunings on Linux

Using Netfilter conntrack

Note: Only use Netfilter conntrack in a performance-critical environment when absolutely necessary as it adds significant load.

If you're getting the error message: "ip_conntrack: table full, dropping packet" in dmesg or your system logs, you can check the number of entries in the table by reading from /proc/sys/net/ipv4/ip_conntrack_count, and the size of the table using ip_conntrack_max.  On most kernels, you can dynamically raise the maximum number of entries:

# echo 131072 > /proc/sys/net/ipv4/ip_conntrack_max

If the ‘ip_conntrack_max’ file is missing, you most likely have not loaded the ip_conntrack module.

The best way to permanently set the conntrack table sizes is by adding the following options to /etc/modules.conf (or /etc/modprobe.d/<filename>):

options ip_conntrack hashsize=1310719

options nf_conntrack hashsize=1310719

Note that Netfilter conntrack (used by Stingray’s IP transparency and other NAT use cases) adds significant load to the kernel and should only be used if necessary.  When you enable NAT or other features that use conntrack, the conntrack kernel modules are loaded; they are not always unloaded once these features are disabled.  Search for and unload the unused modules ip_conntrack, iptable_filter, ip_tables and anything else with iptables in its name.

Packet forwarding and NAT

Stingray Traffic Manager is typically deployed in a two-armed fashion, spanning a front-end public network and a back-end private network.

In this case, IP forwarding should be disabled because the back-end private IP addresses are not routable from the front-end network:

# echo 0 > /proc/sys/net/ipv4/ip_forward

Stingray will not forward any IP packets.  Only traffic that is directed to a listening service on the traffic manager will be relayed between networks.  Although Stingray should not be regarded as a replacement for a network firewall, this configuration provides a strong security boundary which only allows known, identified traffic to reach the back-end servers from the front-end network, and all traffic is automatically ‘scrubbed’ at L2-L4.

In some cases, you may wish to allow the back-end servers to initiate connections to external addresses, for example, to call out to a public API or service.  The Stingray host can be configured to forward traffic and NAT outgoing connections to the external IP of the host so that return traffic is routable; the following example assumes that eth0 is the external interface:

# echo 1 > /proc/sys/net/ipv4/ip_forward

Flush existing iptables rules if required:

# iptables --flush

Masquerade traffic from the external interface

# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE

Optionally, drop all forwarded connections except those initiated through the external interface

# iptables -A FORWARD -m state --state NEW -i eth0 -j ACCEPT

# iptables -A FORWARD -m state --state ESTABLISHED,RELATED -j ACCEPT

# iptables -A FORWARD -j DROP

Unless you are using asymmetric routing, you should disable source routing as follows:

# echo 0 > /proc/sys/net/ipv4/conf/default/accept_source_route

# echo 1 > /proc/sys/net/ipv4/conf/default/rp_filter

Duplicate Address Detection

The Duplicate Address Detection (DAD) feature seeks to ensure that two machines don't raise the same address simultaneously. This feature can conflict with the traffic manager’s fault tolerance; when an IP is transferred from one traffic manager system to another, timing conditions may trigger DAD on the traffic manager that is raising the address.

# echo 0 > /proc/sys/net/ipv6/conf/default/dad_transmits

# echo 0 > /proc/sys/net/ipv6/conf/all/dad_transmits