While installing the Cisco 887, I came across a bug in the IPv6 support of Cisco IOS and the way it handles DHCPv6 and null-routes vs static routes.
DISCLAIMER: I am by no means an expect is Cisco IOS. I have had no training in it and achieve mosts tasks thanks to Google.
Internode are, as far as I know, the first and only ISP in Australia to offer Native IPv6 to all their customers. All home users get a /64 IPv6 address range, and business customers get a /56 range. Note that at this stage, customers must opt-in to IPv6 connectivity by connecting to their service with a modified username.
The transit for all out connectivity (IPv4 and IPv6) is done over an ADSL2+ connection, terminating at the Cisco 887. The Cisco 887 is configured to request both a dynamic /64 for the ADSL PPP tunnel, and our static /56 assigned range using DHCPv6 with the Prefix Delegation flag.
When the router receives the DHCPv6 answer from the Internode server, it adds a null route for the received Prefix Delegation. This is to prevent routing loops where traffic comes inbound from the ISP, and the Cisco sends it back to the ISP (by virtue of it's default route).
This is all fine, sane and acceptable.
|Simplified overview of connectivity.|
However behind the Cisco is our Firewall (actually, a High Availability pair of firewalls) that connects, routes and firewalls our internal networks to the public internet.
This is where the majority of our IPv6 address space needs to be routed to.
The normal solution that anyone familiar with networking would say is to add static routes:
- Reserve a /64 for the link between the Cisco and the Firewalls
- Statically route the rest of the /56 to the firewall's address.
This was my solution too! However it turns out the bug in IOS (Version 12.4(22r)YB5, RELEASE SOFTWARE (fc1)) creates static and null-routes with equal metrics. The result of this is that packets are routed in a load-balancing pattern over the 2 routes; Every X packet will be null routed, while every other packet is routed successfully. This manifests at the client as very high packet loss, and in turn extremely slow IPv6 connectivity.
SOLUTIONCisco have an internal bug tracking request open on this issue; I am unsure on their resolution or timeline etc. My suggestion if you are bitten by this bug is to add multiple routes, each route 1 bit smaller than your PD. The more specific route will take precedence over the null-route even though they are of equal metrics.
In our case, this means adding 2 x /57 routes:
S 2001:db8:412f:C600::/56 [1/0] via Null0, directly connected S 2001:db8:412f:C600::/57 [1/0] via 2001:db8:412f:C600::F0 S 2001:db8:412f:C680::/57 [1/0] via 2001:db8:412f:C600::F0 C 2001:db8:412f:C600::/64 [0/0] via Vlan1, directly connected
The above routes, in order:
- The automatically created null route for the /56 PD.
- The first /57 route to send traffic to our firewall.
- The second /57 route to send traffic to our firewall.
- A /64 route for the address space used between the Cisco and the Firewall.