ISC F root server - IPv6 issue at NIXI Chennai

06-07-2017

Last week I noticed that F root was showing poor connectivity with Indian RIPE Atlas probes for F-root. The graph looked really terrible.

I traced to it from one of RIPE Atlas probes and saw this trace:

Probe #6107
  1 2401:7500:fff0:1::1                      0.838 ms     0.747 ms     0.632 ms
  2 2400:5200:1c00:d::1                      1.755 ms     1.745 ms     1.726 ms
  3 2403:0:100::2be                          2.089 ms     2.054 ms     2.049 ms
  4 2404:a800:2a00::13d                     45.589 ms    26.274 ms     33.64 ms
  5 2404:a800::178                          26.376 ms    25.406 ms    25.276 ms
  6 2001:de8:1:2::3                         25.363 ms    25.232 ms    25.223 ms
  7 *                                           *            *            *
  8 *                                           *            *            *
  9 *                                           *            *            *
 10 *                                           *            *            *
 11 *                                           *            *            *

Here the last hop before timeout i.e hop 6 is of NIXI Chennai peering subnet 2001:de8:1:2::/64. As soon as I saw it, it reminded me older issue which happened and broke IPv4 connectivity to root DNS servers. I blogged about it here, here and here. So the problem remains that NIXI is broken cost wise due to charge on in - out policy. This leads to people accepting routes at all NIXI’s but they do not announce their routes. Thus return path is broken and essentially traffic is being blackholed. Earlier this issue was fixed by adding IP transit support to these root DNS servers so that a default route stays in case of all other failures. It seems like same is missing in IPv4 world and routes are not being announced. During this time, I saw two BGP sessions at NIXI Chennai for F root:

2001:de8:1:2::3 24049 ESTAB 25d 3h10m 1 0 2263 0
2001:de8:1:2::4 24049 ESTAB 125d18h30m 1 0 2264 0

Both were announcing prefix covering F root server’s pool:

show ipv6 bgp neighbors 2001:de8:1:2::3 received-routes
       There are 1 received routes from neighbor 2001:de8:1:2::3
Searching for matching routes, use ^C to quit...
Status A:AGGREGATE B:BEST b:NOT-INSTALLED-BEST C:CONFED\_EBGP D:DAMPED
       E:EBGP H:HISTORY I:IBGP L:LOCAL M:MULTIPATH m:NOT-INSTALLED-MULTIPATH
       S:SUPPRESSED F:FILTERED s:STALE
       Prefix             Next Hop        MED        LocPrf     Weight Status
1      2001:500:2f::/48   2001:de8:1:2::3 10         100        0      BE
         AS_PATH: 24049 3557


show ipv6 bgp neighbors 2001:de8:1:2::4 received-routes
       There are 1 received routes from neighbor 2001:de8:1:2::4
Searching for matching routes, use ^C to quit...
Status A:AGGREGATE B:BEST b:NOT-INSTALLED-BEST C:CONFED\_EBGP D:DAMPED
       E:EBGP H:HISTORY I:IBGP L:LOCAL M:MULTIPATH m:NOT-INSTALLED-MULTIPATH
       S:SUPPRESSED F:FILTERED s:STALE
       Prefix             Next Hop        MED        LocPrf     Weight Status
1      2001:500:2f::/48   2001:de8:1:2::4 10         100        0      E
         AS_PATH: 24049 3557

I posted about it on SANOG and APNIC mailing list. Though there hasn’t been any reply by ISC, Sunny from APNIC passed info to them and I noticed that prefix announcement from NIXI has been withdrawn. Connectivity to F root now works to the instances outside India.

Waiting to hear from ISC as of now. Time to get back to work!

Telekom Germany