NIXI

This post covers how routing is being manipulated across Indian IXPs resulting in free IP transit

Post to determine whether or not NIXI carries full Indian routing table?

Background

Lately, NIXI has been making a bit of news in the Indian peering ecosystem. NIXI for those who may not be aware is the National Internet Exchange of India. It was founded in 2003 with the idea to provide inter-connection layer 2 peering fabric for local Indian ISPs. They were supposed to ensure domestic Indian traffic is exchanged within India and not outside of India. In my previous post, I did cover how that is not true for now. They never picked up much interconnection due to a number of fundamental issues with their policies.

After my last post about home networking, I am jumping back into global routing. More specifically how Indian traffic is hitting the globe when it does not need to. This is an old discussion across senior management folks in telcos, policymakers, and more. It’s about “Does Indian internet traffic routes from outside of India?” and if the answer is yes then “Why?” and “How much?”

It became a hot topic, especially after the Snowden leaks. There was even an advisory back in 2018 from Deputy National Security Advisor to ensure Indian internet traffic stays local (news here). Over time this has come up a few dozen times in my discussion with senior members from the Indian ISP community, individuals, and even latency-sensitive gamers. So I am going to document some of that part here. I am going to put whatever can be verified publically and going to avoid putting any private discussions I had with friends in these respective networks. The data specially traceroutes will have measurement IDs from RIPE Atlas so they can be independently verified by other network engineers.

A friend of mine buzzed me yesterday about his missing route objects. Later multiple other ISPs told the same story which triggered me to put this as a question on INNOG Mailing list. Many folks replied of missing route objects there and it seems to be limited to IRINN members only. I also asked the same question on APNIC mailing list and it was again confirmed about the issue.
Before I proceed further, here’s what it is all about.

IRINN (Indian Registry for Internet Names and Numbers) is a NIR (National Internet Registry) for India operating under the APNIC RIR (Regional Internet Registry). IRINN is run and managed by NIXI. It’s a decent NIR and was set up in 2012. Indian organisations have the option to either maintain relation with APNIC or with IRINN.

A large number of small networks prefer IRINN because it’s annual charges are 25000 INR / $351 USD against APNIC’s membership fee which is over 2x of that.

I am in Chiang Mai, Thailand for APNIC 48 conference. Earlier today attended APIX meeting where many IX members from Asian community gave an update including NIXI i.e National Internet Exchange of India.

As per the update NIXI now allows content players to peer at the exchange. NIXI earlier had a strict requirement of telecom license for anyone to peer but as of now it allows anyone with IP address and AS number to be part of the exchange just like all other exchanges. This is a really good development coming this year after their announcement of the removal of x-y charge. One strange thing remains that their website is still not updated to reflect that which is probably just work in progress. As per representative from NIXI they now openly welcome all content players to peer at NIXI.

It’s kind of fun times in India with many IX’es showing up and now NIXI finally removing their traffic wise charge.

I came across this email which they sent to their member networks recently:

From: I X <[ix@nixi.in](mailto:ix@nixi.in)\>  
Sent: Thu, 21 Feb 2019 12:48:31 GMT+0530  
To: "members " <[members@nixi.in](mailto:members@nixi.in)\>  
Cc: "[ceo@nixi.in](mailto:ceo@nixi.in)" <[ceo@nixi.in](mailto:ceo@nixi.in)\>  
Subject: Change in Data Transfer Charges (X-Y)  

Dear Members,  
  
In reference to the Board's direction to reduce the current data transfer (X-Y) charges from Re. 1/GB to Rs. 0/GB. The said reduction in X-Y charges to Rs. 0/GB is being implemented with effect from 1st March 2019. 
This is for your kind information please.  
  
Thanks & Regards.  
National Internet Exchange of India (NIXI)  
[www.nixi.in](http://www.nixi.in/)

 

Has been a while since I checked the status of root servers which are hosted at NIXI. The list as per their official member list stays the same i.e i root in Mumbai, K root in Noida and F root in Chennai. 

 

i root seems to be up!

show ip bgp neighbors 218.100.48.75 received-routes
       There are 5 received routes from neighbor 218.100.48.75
Searching for matching routes, use ^C to quit...
Status A:AGGREGATE B:BEST b:NOT-INSTALLED-BEST C:CONFED_EBGP D:DAMPED
       E:EBGP H:HISTORY I:IBGP L:LOCAL M:MULTIPATH m:NOT-INSTALLED-MULTIPATH
       S:SUPPRESSED F:FILTERED s:STALE
       Prefix             Next Hop        MED        LocPrf     Weight Status
1      192.36.148.0/24    218.100.48.75   0          100        0      BE
         AS_PATH: 8674 29216
2      194.58.198.0/24    218.100.48.75   0          100        0      BE
         AS_PATH: 8674 56908
3      194.58.199.0/24    218.100.48.75   0          100        0      BE
         AS_PATH: 8674 56908
4      194.146.106.0/24   218.100.48.75   0          100        0      BE
         AS_PATH: 8674
5      194.146.107.0/24   218.100.48.75   0          100        0      BE
         AS_PATH: 8674

 

Last week I noticed that F root was showing poor connectivity with Indian RIPE Atlas probes for F-root. The graph looked really terrible.

 

I traced to it from one of RIPE Atlas probes and saw this trace:

Probe #6107
  1 2401:7500:fff0:1::1                      0.838 ms     0.747 ms     0.632 ms
  2 2400:5200:1c00:d::1                      1.755 ms     1.745 ms     1.726 ms
  3 2403:0:100::2be                          2.089 ms     2.054 ms     2.049 ms
  4 2404:a800:2a00::13d                     45.589 ms    26.274 ms     33.64 ms
  5 2404:a800::178                          26.376 ms    25.406 ms    25.276 ms
  6 2001:de8:1:2::3                         25.363 ms    25.232 ms    25.223 ms
  7 *                                           *            *            *
  8 *                                           *            *            *
  9 *                                           *            *            *
 10 *                                           *            *            *
 11 *                                           *            *            *

Here the last hop before timeout i.e hop 6 is of NIXI Chennai peering subnet 2001:de8:1:2::/64. As soon as I saw it, it reminded me older issue which happened and broke IPv4 connectivity to root DNS servers. I blogged about it here, here and here. So the problem remains that NIXI is broken cost wise due to charge on in - out policy. This leads to people accepting routes at all NIXI’s but they do not announce their routes. Thus return path is broken and essentially traffic is being blackholed. Earlier this issue was fixed by adding IP transit support to these root DNS servers so that a default route stays in case of all other failures. It seems like same is missing in IPv4 world and routes are not being announced. During this time, I saw two BGP sessions at NIXI Chennai for F root: