Andrew Ayer

I recently lost an afternoon dealing with a most vexing routing problem on a server which turned out to be the result of an ICMP redirect attack.

ICMP redirects are a "feature" of IP which allows a router to inform a host that there's a more efficient route to a destination and that the host should adjust its routing table accordingly. This might be OK on a trusted LAN, but on the wild Internet, where malice abounds, it may not be such a good idea to alter your routing table at someone else's whim. Nevertheless, ICMP redirects are enabled by default on Linux.

The problem manifested itself with a customer unable to contact one of our servers. The customer provided a traceroute which starred out after the last hop before our server, leading him to conclude that our firewall was blocking him. We would probably have concluded the same thing, except we knew that no such firewall existed. From our end, traceroutes to him starred out immediately before even reaching our default gateway. Even more bizarrely, if we picked a certain one of our server's four IP addresses as the source address for the traceroute, the traceroute worked! Furthermore, we were able to traceroute to adjacent IP addresses on the customer's subnet from any source address without issue.

We looked again and again at our routing table and our iptables rules. There was nothing to explain this behavior. This server is virtualized, so we were starting to suspect the underlying host, when I decided to look at the kernel's route cache with the ip route show cache command. What I saw worried me greatly:

root@tommy:~# ip route show cache | grep 198.168.103.11

198.168.103.11 via 10.254.87.146 dev eth0 src 66.228.52.206

198.168.103.11 from 66.228.52.251 via 10.254.87.146 dev eth0

198.168.103.11 from 66.228.52.209 via 10.254.87.146 dev eth0

198.168.103.11 from 66.228.52.206 via 66.228.52.1 dev eth0

198.168.103.11 from 66.228.52.250 via 10.254.87.146 dev eth0

These entries say to route packets to 198.168.103.11 (the customer's IP address, changed to protect their identity) via 10.254.87.146. However, 10.254.87.146 is not our default gateway. In fact, we don't use any private IP addresses that look remotely like that. The fourth entry, which has a legitimate gateway and applies only to packets with a source address of 66.228.52.206, explains why we could successfully use that one IP address as a source address.

I had read about ICMP redirect attacks before and suspected that they may be at play here. To test this hypothesis I spun up a test server and used an extremely useful tool called scapy to send my own fake ICMP redirect packets. The results were strange. Sending the fake ICMP redirect did not immediately put a bogus entry in the route cache. However, if I attempted to contact the host targeted by the redirect within 10 minutes of sending the redirect, then an entry appeared in the route cache that prevented me from contacting the host. Furthermore, once the entry got in the cache, there was no getting rid of it. Even if I flushed the cache, the rogue entry came back the next time I tried to contact the targeted host. The kernel was clearly keeping some separate state of redirected route entries, and as far as I could tell, there was no way to inspect it. This meant that the only way to recover the affected server was to reboot it!

There are clear denial-of-service possibilities. If you want to prevent a host (the "victim") from contacting another host (the "target"), just send a fake redirect packet for the target to the victim! Since you don't need to forge a packet's source address to send a rogue ICMP redirect, it will make its way past RP filters. The only constraint is that your victim needs to contact the target within 10 minutes of receiving the redirect for it to stick. This is easy to overcome: you can send the redirect when the victim is likely to be contacting the target, or simply send a new redirect every 10 minutes (that's hardly high volume). Or, more diabolically, if you have the ability to spoof source addresses, you can follow the redirect packet with a TCP SYN packet with its source address spoofed as the target. The victim will reply to the target with a SYN-ACK, and in doing so make permanent the effect of the ICMP redirect.

Obviously you need to disable ICMP redirect packets on any public-facing host. Unfortunately, the most intuitive and widely-documented way of disabling ICMP redirects on Linux (by writing 0 to /proc/sys/net/ipv4/conf/all/accept_redirects) doesn't always work! From Documentation/networking/ip-sysctl.txt of the Linux source:

accept_redirects - BOOLEAN Accept ICMP redirect messages. accept_redirects for the interface will be enabled if: - both conf/{all,interface}/accept_redirects are TRUE in the case forwarding for the interface is enabled or - at least one of conf/{all,interface}/accept_redirects is TRUE in the case forwarding for the interface is disabled accept_redirects for the interface will be disabled otherwise default TRUE (host) FALSE (router)

This arcane logic means that for a non-router (i.e. most servers), not only must /proc/sys/net/ipv4/conf/all/accept_redirects be 0, but so must /proc/sys/net/ipv4/conf/interface/accept_redirects. So, to recap, the following will reliably disable ICMP redirects (assuming your interface is eth0):

echo 0 > /proc/sys/net/ipv4/conf/all/accept_redirects

echo 0 > /proc/sys/net/ipv4/conf/eth0/accept_redirects

If you put that in a system startup script, you should be safe.

A final note: the precise behavior for handling ICMP redirects and the route cache may differ between kernel versions. My tests were conducted on 2.6.39.