25 Oct

NIXI root DNS servers and updates

Has been a while since I checked the status of root servers which are hosted at NIXI. The list as per their official member list stays the same i.e i root in Mumbai, K root in Noida and F root in Chennai. 

i root seems to be up!

show ip bgp neighbors 218.100.48.75 received-routes
       There are 5 received routes from neighbor 218.100.48.75
Searching for matching routes, use ^C to quit...
Status A:AGGREGATE B:BEST b:NOT-INSTALLED-BEST C:CONFED_EBGP D:DAMPED
       E:EBGP H:HISTORY I:IBGP L:LOCAL M:MULTIPATH m:NOT-INSTALLED-MULTIPATH
       S:SUPPRESSED F:FILTERED s:STALE
       Prefix             Next Hop        MED        LocPrf     Weight Status
1      192.36.148.0/24    218.100.48.75   0          100        0      BE
         AS_PATH: 8674 29216
2      194.58.198.0/24    218.100.48.75   0          100        0      BE
         AS_PATH: 8674 56908
3      194.58.199.0/24    218.100.48.75   0          100        0      BE
         AS_PATH: 8674 56908
4      194.146.106.0/24   218.100.48.75   0          100        0      BE
         AS_PATH: 8674
5      194.146.107.0/24   218.100.48.75   0          100        0      BE
         AS_PATH: 8674

K root seems to be down!

Router: NIXI Delhi (Noida)
Command: show ip bgp neighbors 218.100.48.6 received-routes
show ip bgp neighbors 218.100.48.6 received-routes
Inbound soft reconfiguration not enabled for neighbor 218.100.48.6

F root seems to be up!

show ip bgp neighbors 218.100.48.135 received-routes
       There are 1 received routes from neighbor 218.100.48.135
Searching for matching routes, use ^C to quit...
Status A:AGGREGATE B:BEST b:NOT-INSTALLED-BEST C:CONFED_EBGP D:DAMPED
       E:EBGP H:HISTORY I:IBGP L:LOCAL M:MULTIPATH m:NOT-INSTALLED-MULTIPATH
       S:SUPPRESSED F:FILTERED s:STALE
       Prefix             Next Hop        MED        LocPrf     Weight Status
1      192.5.5.0/24       218.100.48.135  10         100        0      ME
         AS_PATH: 24049 3557 3557

Atleast while 2 out of 3 root servers seems to be up but for some reason my connection in Haryana isn’t  hitting i root. F root instance it is taking me there for sure. 

i root latency check from my home: 

ping -c 5 i.root-servers.net.
PING i.root-servers.net (192.36.148.17) 56(84) bytes of data.
64 bytes from i.root-servers.net (192.36.148.17): icmp_seq=1 ttl=52 time=156 ms
64 bytes from i.root-servers.net (192.36.148.17): icmp_seq=2 ttl=52 time=155 ms
64 bytes from i.root-servers.net (192.36.148.17): icmp_seq=3 ttl=52 time=155 ms
64 bytes from i.root-servers.net (192.36.148.17): icmp_seq=4 ttl=52 time=156 ms
64 bytes from i.root-servers.net (192.36.148.17): icmp_seq=5 ttl=52 time=155 ms
--- i.root-servers.net ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4004ms
rtt min/avg/max/mdev = 155.817/156.034/156.481/0.551 ms

That’s clearly too high latency. Latency from my location to Mumbai is typically 30-40ms. Let’s trace to the i root. 

traceroute i.root-servers.net.
traceroute to i.root-servers.net. (192.36.148.17), 30 hops max, 60 byte packets
 1  172.16.0.1 (172.16.0.1)  0.590 ms  0.664 ms  0.774 ms
 2  103.201.140.218 (103.201.140.218)  3.500 ms  3.413 ms  3.495 ms
 3  10.10.26.1 (10.10.26.1)  6.182 ms  6.304 ms  6.012 ms
 4  10.10.26.9 (10.10.26.9)  6.318 ms  6.047 ms  5.964 ms
 5  nsg-static-77.249.75.182-airtel.com (182.75.249.77)  45.636 ms  44.225 ms  44.144 ms
 6  182.79.191.89 (182.79.191.89)  56.411 ms 182.79.181.218 (182.79.181.218)  59.690 ms 182.79.153.86 (182.79.153.86)  66.090 ms
 7  182.79.149.95 (182.79.149.95)  207.319 ms 182.79.217.94 (182.79.217.94)  57.670 ms 182.79.149.95 (182.79.149.95)  207.285 ms
 8  182.79.177.101 (182.79.177.101)  187.845 ms 182.79.224.134 (182.79.224.134)  183.999 ms 182.79.224.124 (182.79.224.124)  180.657 ms
 9  182.79.146.218 (182.79.146.218)  211.258 ms 182.79.154.2 (182.79.154.2)  187.929 ms 182.79.154.10 (182.79.154.10)  192.907 ms
10  182.79.149.103 (182.79.149.103)  183.405 ms  181.645 ms  181.540 ms
11  peering.r1.lnx.dnsnode.net (195.66.225.151)  157.300 ms  157.214 ms  157.293 ms
12  i.root-servers.net (192.36.148.17)  157.364 ms  156.423 ms *

Thus Airtel is taking me to all the way to London (while LNX = Airport code for Smolensk Airport, Smolensk, Russia but route clearly shows it’s being exchanged at LINX. Someone in Netnod got into habit of writing LINX as LNX which is confussing). 

I see the same by querying id.server and hostname.bind in CHAOS class.

dig chaos @192.36.148.17 id.server txt  +short
"s1.lnx"
dig chaos @192.36.148.17 hostname.bind  txt  +short
"s1.lnx"

So, for now, Airtel is preferring route learnt via LINX peering over route learnt at NIXI. In a check by all Indian RIPE Atlas probes, I see that out of 50 RIPE Atlas probes, 23 are hitting s1.mum in Mumbai, 19 are hitting LINX London (s1.lnx) and 1 (which is hosted on NKN) is hitting s1.amx in Amsterdam (json data here). 

Why this happens? 

It’s often the lack of peering and/or case of prefered routes. For smaller networks, it’s simply missing peering. For larger networks, it’s about which route they prefer, which not. Here’s a view of networks with their ASNs sorted by latency (wherever RIPE Atlas Probe) was present (measurement link here). 

So what can be done about it? 

NIXI needs to be more attractive to various (smaller) networks which clearly it is not since it just does not has any content player connected to it due to policy issue. Furthermore customers of Airtel need to buzz it and request for a better route to i root’s local instance. 

Comments & thoughts expressed in the post are personal and have nothing to do with my employer. I am also volunteering for supporting tech platform for BharatIX to facilitate peering.

26 Oct

K root route leak by AS49505 – Selectel, Russia

There seems be an ongoing route leak by AS49505 (Selectel, Russia) for K root server.
K root server’s IP: 193.0.14.129
Origin Network: AS25152
 
Here’s trace from Airtel Looking Glass, Delhi PoP

Mon Oct 26 16:21:18 GMT+05:30 2015
traceroute 193.0.14.129
Mon Oct 26 16:21:22.053 IST
Type escape sequence to abort.
Tracing the route to 193.0.14.129
 1   *
    203.101.95.146 19 msec  4 msec
 2  182.79.224.73 14 msec  3 msec  1 msec
 3  14.141.116.89.static-Delhi.vsnl.net.in (14.141.116.89) 7 msec  3 msec  2 msec
 4  172.23.183.134 26 msec  45 msec  26 msec
 5  ix-0-100.tcore1.MLV-Mumbai.as6453.net (180.87.38.5) 151 msec  153 msec  152 msec
 6  if-9-5.tcore1.WYN-Marseille.as6453.net (80.231.217.17) [MPLS: Label 383489 Exp 0] 160 msec  163 msec  155 msec
 7  if-2-2.tcore2.WYN-Marseille.as6453.net (80.231.217.2) [MPLS: Label 595426 Exp 0] 161 msec  162 msec  162 msec
 8  if-7-2.tcore2.FNM-Frankfurt.as6453.net (80.231.200.78) [MPLS: Label 399436 Exp 0] 149 msec  151 msec  155 msec
 9  if-12-2.tcore1.FNM-Frankfurt.as6453.net (195.219.87.2) 164 msec  163 msec  159 msec
 10 195.219.156.146 153 msec  151 msec  160 msec
 11 spb03.transtelecom.net (188.43.1.226) 190 msec  192 msec  189 msec
 12 Selectel-gw.transtelecom.net (188.43.1.225) 185 msec  185 msec  185 msec
 13 k.root-servers.net (193.0.14.129) 183 msec  204 msec  196 msec
RP/0/8/CPU0:DEL-ISP-MPL-ACC-RTR-9#

 
The routing information (show route 193.0.14.129 output) from their looking glass doesn’t seems useful since it shows that it’s learning K root Noida route via NIXI. This is likely because routing information is different from actual forwarding information in that device.
So the trace looks extremely weird. It’s leading traffic to K root which does has anycast instance in Noida, landing into Russia!
 
Why is that happening?
Let’s look at what Tata Communications (AS6453) routing table has for K root’s prefix. I am looking at feed of AS6453 which it’s putting into RIPE RIS RRC 03 collector.

anurag@server7:~/temp$ awk -F ‘|’ ‘$5==6453’ rrc03-table-26-Oct-2015.txt|grep 193.0.14.0/24
TABLE_DUMP_V2|10/26/15 08:00:03|A|80.249.209.167|6453|193.0.14.0/24|6453 20485 49505 25152|IGP
anurag@server7:~/temp$

 
Let’s analyse this AS_PATH

  1. AS25152 is orignating prefix to AS49505 (Selectel Russia)
  2. AS49505 is “leaking” route to it’s upstream AS20485 (Trans Telecom, Russia)
  3. AS20485 is further propagating route to Tata Communications AS6453 making route visible globally via Tata Communications IP backbone

 
What impact of it?
Impact is much higher latency with K root from India. Here’s how RIPE Probe 170111 hosted at my home finds latency to K root:
K_root_Performance
 
As per graph change, leak started on 24th Oct at 9am UTC and this resulted in jump of latency of over 180ms.
 
Disclaimer: Post, comments, thoughts and analysis is in personal capacity and in no way linked to my employer.