05 Oct

Understanding NIXI and it's policies

NIXI i.e National Internet Exchange of India is well known for it’s inefficiency and for its bad policies. I am posting this blog post to discuss some of them.

Bit of background:

NIXI is one (and only) Indian IXP i.e Internet Exchange Point established in 2003 so as to facilitate peering between Indian ISPs. Before this, there were lot of cases when Indian ISP’s were connecting to each other from outside India in Singapore and Europe. Thus NIXI established few exchanges in key cities where necessary infrastructure was provided to ISP’s to “peer”.  With peering, the strict technical meaning is that exchange of traffic between ISPs.

Understanding peering…

E.g if ISP A is peering with ISP B, ISP A will announce it’s and it’s downstream customer prefixes to B and so does B will do with A. If A is peering with C, and B is peering with D, C and D will NOT be able to reach to each other via A and B’s interconnection. While say if E is (downstream) customer of A and F is downstream customer of B, then there will be a route between E to F as E > A > B > F and vice versa (depending on prefixes announced). Thus, if big ISPs like Airtel and Tata peer at NIXI, it will help to keep traffic between them local + also their customers (not peers).

What is benefit of keeping traffic local?

A lot! 🙂
Firstly that’s way more efficient. Why to have packets to go from India > Europe > US > Singapore > India just because source and destination networks are not connected within India? Also, theoretically it will save us on bandwidth costs (in real it is not yet, which I will discuss in this post). Other then that there are huge technical advantages of keeping traffic i.e low latency which matters a lot when latency within India is usually less then 100ms, while with US it’s 300ms. And worst if we are hitting US twice in a round trip connection, we will get latency of over 500ms! Also, networks work lot faster on direct peering due to TCP window size factor apart from fact that having 4-5 networks between source and destination effects a lot.

So how much NIXI has done so far?

Well not much!
Let’s talk about some numbers. India has around 10 million broadband users and a little over 100 million users (so a lot still on slow non-broadband usage) as per this data. Aggregated traffic passing from all NIXI exchanges is around 20Gbps as per their official data. If we look at Europe, number of Internet users around 50 million as per this data, while aggregated traffic of Amsterdam Internet Exchange (AMX) is over 1.6Tbps alone! (Source). Other important exchanges like London Internet Exchange carry 1.4Tbps, while DE-CIX exchange in Germany carry around 2Tbps of aggrigated traffic. It’s very hard to count total data of all these exchanges but clearly AMX + LINX + DECIX alone carry 5Tbps i.e 5000Gbps of aggregated bandwidth which is 250 times more then that of Indian exchange NIXI. (I am still ignoring dozen of big exchanges here. You can find IXP directory at PCH’s website here)

At this point one can think of fact that is NIXI only exchange in India?

Well, yes. Indian “licence raj” as usual! Also at AMX, we can see around 514 connected networks, at LINX 435 members while at DECIX around 465 participants. At all NIXI we don’t see more then 30-40 connected networks! I can guess aggrigated number of unique networks to be say 50 or so.
Look at list of members in AMX and compare with NIXI. Some big names are missing at NIXI. Few of them are – Google, Akamai, Limelight, Microsoft, root servers networks, Verisign, Yahoo etc. One common thing among all these networks is the fact they are content providers and not really consumers. They send more traffic (way more) then what they receive. NIXI has a requirement that only ISPs can join NIXI and with ISP it means you must have license to operate in India. There’s no place for content provider! This is strange and an absurd because traffic between content consuming ISPs is very low as compared to content between content providers like CDN, datacenters, etc. Unless they join in the exchange, there won’t be really much traffic for ISP participants to consume.
At this point one can think of fact that if content provider like Google is connected to an NIXI participating ISP, won’t it be available at NIXI via that ISP?
OK! Let’s look at this. Google.co.in uses IPs from subnet. It is announcement by Google’s AS 15169.  Let’s look at this prefix at NIXI Delhi, Mumbai and Chennai using their looking glass.

NIXI Looking Glass – show ip bgp
Router: NIXI Delhi (Noida)
Command: show ip bgp
% Network not in table
NIXI Looking Glass – show ip bgp
Router: NIXI Mumbai
Command: show ip bgp
% Network not in table
NIXI Looking Glass – show ip bgp
Router: NIXI Chennai
Command: show ip bgp
show ip bgp
BGP4 : None of the BGP4 routes match the display condition

So – no where. One cannot reach Google AS15169 from NIXI and reason for that is the fact that Google just peers with Indian providers like Airtel, Tata & Reliance and doesn’t really has a transit relationship. Likely Google doesn’t pays to any of them and they only to Google’s Indian PoPs and caching servers.  Since Google has a peering relationship, ISPs do not share Google’s route with other peers. If we look at say Akamai India – I see they have one of DNS servers ns1-2.akam.net in India. It is using IP which is coming from BGP announcement of by AS 21342. Looking at prefix at NIXI Delhi:

NIXI Looking Glass – show ip bgp
Router: NIXI Delhi (Noida)
Command: show ip bgp
BGP routing table entry for, version 62228
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Advertised to update-groups:
1 2 3
9498 20940 20940 20940 21342, (received & used) from (
Origin IGP, localpref 100, valid, external, best

Clearly route available via Airtel (AS 9498) and so does at NIXI Mumbai and Chennai too. Here Akamai is visible because Akamai seems to be a downstream customer of Bharti Airtel AS9498.
This is a problem! So if you are content provider, NIXI damages you as:

  • You cannot connect to NIXI’s peering fabric directly unless you have a license to operate in India (NIXI’s requirement here)
  • You cannot reach to NIXI via other ISPs unless you become their direct customers. No transit routes, only peering routes. This is normal for an exchange though.

Also problem is just not limited to this. Another issue at NIXI is cost factor. It’s really terrible how economics work here. As per NIXI’s routing & tariff policy, there is really crazy pricing.

Counting on costs…

Joining charges – 1000Rs (just $20!), port charges for 100Mbps around $2000, while Gig port costs around $5000 yearly. Comparing it to AMX they don’t seem selling less then 500Mbps directly (well ….) while for 1Gbps it costs 500 Euro monthly which will around $7800 per year. So NIXI seems cheap? Well no! There’s a catch here that NIXI is likely world’s only exchange which has concept of charging on data transferred too!
It goes as for exchange of traffic between ISP A and ISP B, the “requester pays” logic goes. So if B’s customer are pulling off data from A then B will have to pay 12Rs/GB to ISP A. Now here catch is if you are a datacenter (well yeah there are datacenters too connected to NIXIs with ISP license) then requester pays concept doesn’t works but datacenter does has to pay for it’s own downstream.

How does cost is so damaging for a small ISP?

Let’s assume if a small ISP connects to NIXI. Since it is small and has consumers who are just pulling data, it would have to pay 12Rs/GB to big old incumbent operators while they would be taking relatively very less content from that network and would be paying negligible in return (apart from fact that small ISP would be surely using one of them for Internet transit to have global reach ability). So for say a 100Mbps connection in full use, it will transfer around 8640000 Mega bits in one day or 394200000 Mega bytes in one year which is around 394200 GB. At 12Rs/GB pull off costs, it will be around $94k yearly bill! Apart from  $2000 port charges! 🙂
So, we are talking about partial routing tables with almost no content provider and paying around $96k for 100Mbps peering capacity. That too when retail ISP would be likely selling connections at around $10/user on average. So we are talking about atleast 9600 users alone to cover cost of peering apart from layer 2 circuit cost to reach NIXI and surely a HUGE costs of Internet transit to cover rest of non-NIXI routes of global Internet.
Also, if we look at cost of $96k for 100Mbps, it might be little less due to fact that consumers will upload a bit during that time and ISP will also earn. So doing a rough assuming ISP would be paying atleast $80k  in real world case when burst port completely. So $80k for 100Mbps means $800 for each mega bit per second. One can easily get a STM 1 i.e OC3 circuit (for American readers!) for around 25 lakh which will be $50,000. I have ignored fact that ISP has not to pay if it is taking content from datacenters because there are simply not many datacenters in list. Effect will be negligible here!
So we are looking at STM 1 – 155Mbps Global transit bandwidth for $50,000 Vs NIXI peering partial routes with missing major content providers for around $80,000 an year. Amazing?
As you might have already guessed – one can get way more cheaper transit bandwidth from ISPs then reaching (only) them via NIXI. Also, not to forget that transit matters a lot because lot of content is still in US and not hosted in India. Also providers like Google who are building their own network have their own PoPs at Delhi, Mumbai and Chennai and one can connect there directly rather then reaching via NIXI (Google’s Peering info map here). So clearly NIXI is failing heavily here.
Also, if we talk about datacenters like Control S and Netmagic who hold ISP license also and are connected to NIXI.  If we look at Control S AS18229 – they are announcing hundreds of prefixes (check here) in global IPv4 table. They participate at NIXI Delhi, Mumbai and Chennai with name Pioneer E labs and use same ASN.
Looking at their prefixes at each of these exchanges via NIXI’s looking glass:

NIXI Looking Glass – show ip bgp neighbors routes
Router: NIXI Mumbai
Command: show ip bgp neighbors routes
BGP table version is 5482108, local router ID is
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
S Stale
Origin codes: i – IGP, e – EGP, ? – incomplete
Network Next Hop Metric LocPrf Weight Path
*> 0 0 18229 i
*> 0 0 18229 i
*> 0 0 18229 i
*> 0 0 18229 i
*> 0 0 18229 i
*> 0 0 18229 i
*> 0 0 18229 i
Total number of prefixes 7
NIXI Looking Glass – show ip bgp neighbors routes
Router: NIXI Delhi (Noida)
Command: show ip bgp neighbors routes
NIXI Looking Glass – show ip bgp neighbors received-routes
Router: NIXI Chennai
Command: show ip bgp neighbors received-routes
show ip bgp neighbors received-routes
There are 13 received routes from neighbor
Searching for matching routes, use ^C to quit…
Prefix Next Hop Metric LocPrf Weight Status
1 1 100 0 BE
AS_PATH: 18229
2 1 100 0 BE
AS_PATH: 18229
3 1 100 0 BE
AS_PATH: 18229
4 1 100 0 BE
AS_PATH: 18229
5 1 100 0 BE
AS_PATH: 18229
6 1 100 0 BE
AS_PATH: 18229
7 1 100 0 BE
AS_PATH: 18229
8 1 100 0 BE
AS_PATH: 18229
9 1 100 0 BE
AS_PATH: 18229
10 1 100 0 BE
AS_PATH: 18229
11 1 100 0 BE
AS_PATH: 18229
12 1 100 0 BE
AS_PATH: 18229
13 1 100 0 BE
AS_PATH: 18229

So 7 prefixes at Mumbai and 13 at Chennai. None at Delhi and rest all 100 (along with these) via their transit providers – Tata, Reliance, Airtel, Tulip etc.  This tells the state of peering in India.
Netmagic seems little better here. They are using AS17439. They are arouncing close to 200 prefixes in Global IPv4 routing table. They also participate at NIXI Mumbai, Delhi and Chennai. They are announcing 57 prefixes at Mumbai, 12 at Delhi and around 40 at Chennai.
NIXI isn’t really supporting datacenters in any sense. Their concept that “requester pays” sucks badly. As per NIXI’s policy if you are a datacenter i.e if you send out 5x times more then what you receive, then you will not be paid by requesting ISP. So e.g if a small ISP having 100Mbps shared peering at NIXI Mumbai starts sucking lot of traffic say 20Mbps from Control S datacenter alone. So datacenter is sending at 20Mbps to that ISP. In other way datacenter would be still getting request of say 1/5th of size so 4Mbps. Thus datacenter is pulling off content from that ISP at 4Mbps for which it has to pay even when it’s peering!
So for say 100Mbps connection, datacenter would be consuming say roughly 1/5th i.e 20Mbps, it will consume over  78TB of data annually. This turns out to be $18k per FE port in peering costs alone for sending domestic traffic to domestic users. This is an absurd!

Changes needed in NIXI and IXP part as whole:

  1. IXP sector should be open and there should be private players in IXP sector like Equinix, Coresite etc. Forget Walmart, please bring Equinix! I would love to see Airtel-Equinix rather then Bharti Walmart! 🙂
  2. NIXI should be open to content provider. Without content provider there’s just no one injecting the content into these networks.
  3. NIXI should end cost based on metering and big ISPs should be rather forced to peer with small ISPs. If destination traffic is for their or their downstream network, they should take it and route it across their network.
  4. More easy availability of layer 2 optical circuits to connect to NIXI fabric.

Local time check: 4:37am here. Time for me to get some sleep! 🙂

20 thoughts on “Understanding NIXI and it's policies

  1. ISPAI president was involved when they were starting nixi.they demanded this whole “requester pays” thing.big isps claimed to have invested a lot in infrastructure…what if a small isp just connects to nixi and starts sharing their infrastructure for free 🙂
    regarding connectivity to non-licensed parties, it violates telegraph act..as any telegraph in india must be operated by union govt. or a licensed operator.suppose i connect my small network which is not under any isp, then i would not be bound by any license to create log and what if something bad happens from my side..whose license government cancels then
    i am not a big admirer though but this is the way it is

    • Hi Pankaj
      Nice to hear from you after long time.
      1.Regarding “requester pays” I don’t agree to logic at all. Thing is small ISPs will pay to big ones anyways for transit anyways since lot of content is still outside. Apart from fact that ISPs are just sharing their regional routes. So why Airtel would have problem to take traffic from someone when it destinationed for its own users (or downstream customers) within Delhi? No one can handle them Delhi traffic at Mumbai unless they really announce those non-regional prefixes. So apart from “requester pays” cost, they still have lot of advantage from Submarine cable infrastructure to domestic layer 2 circuits which runs a good transit business.
      2. Licensing part is just crap. If it’s really a security concern, they can just have ISP license for 1000Rs. That money would be good enough for Govt. to verify the requester and have “power” to cancel that license if ISP screws up anything. Charging awfully high for license along with super complex rules and delayed process is nothing but a clear support for incumbents. We don’t need more fiber from Govt, we just need simple license and better policies at NIXI and that’s it. Rest people like you or me will do better. 🙂

  2. Only one Internet Exchange to serve all Indian ISPs, sounds really low.
    I thought the whole point of peering was to have settlement free exchange of traffic, (apart from port cost and IX maintenance fees)
    Why limit the participants to ISPs only? Makes no sense at all. You need the content providers to be able to exchange traffic in both directions.
    Looks like Google and other content providers simply peer directly with each ISP at this point due to the fact that there is no decent IXP presence in India.
    Good article Anurag!

  3. the objective with which Nixi was set up was to ensure Internet proliferation. One of the factors for proliferation is the low cost of bandwidth for the ISPs, big or small.
    The cost of bandwidth is proportional to the cost of setting up the “last mile” to the consumer, using wireless or wired means.
    One of the ways to achieve is to proliferate many off-take points for the IP traffic,like the wireless towers/ junction boxes, base stations and exchanges are used for telephony. Indeed, that infrastructure should be upgraded for supporting IP traffic. Such infrastructure is replicated by competing telcos or shared, but that is left entirely to the market. Where there is no infrastructure, the Govt. sets it up initially and leases it out, but never operates it!
    If you folks want to speak on this subject, please write to me on bharat200659@rediffmail.com

  4. btw, there is no license needed to set up an internet exchange..:) so one can set up a PCH in India. You don’t need Nixi’s or anyone’s permission for it. Nixi is not a Govt. body, only has the blessings

    • Hi Bharat
      Thanks for your comments. Well there are few more factors in the picture which keep messing up in the picture. For an IX you primarily need 3 or more ISPs interested in peering. Incase of India big players won’t take interest in such initiative while small ones are way to small to run BGP, and get into exchange. Also, layer 2 market isn’t that hot as I understand.
      I would be happy to discuss more about this with you.
      Disclaimer: Comments are personal & DO NOT relate to PCH policies.

  5. Hi, Nice blog…i agree on this one sided approach….but what does law of land say ? what stops IX’s from coming to India or any provider following AMX model ?
    Am sure there is space for more than one IX given the way content is growing but the catch has to specific law….did google, could not come across anythinh which stops or puts one leg-in-leg while taking this initiative…let me know if you have come across points which make this roadblock happen.

    • Hi Niraj
      Good to hear from you.
      >what stops IX’s from coming to India or any provider following AMX model ?
      Yes you are right, it’s not NIXI who is stopping but it’s lack of layer 2 market which is because of lack of competition in last mile retail ISPs. If Govt. of India let ISPs to operate without expensive licence costs, we could have seen lot more WISPs as well as small community based fiber network. All that would have triggered a business model for private IXPs.
      >did google, could not come across anythinh which stops or puts one leg-in-leg while taking this initiative…let me know if you have come across points which make this roadblock happen.
      Google is already doing a fair amount of private peering in India with almost all major broadband providers in Delhi, Mumbai and Chennai. Infact Google carries all traffic to it’s servers in US/Europe over it’s backbone and ISPs are needed just to hand it off to Google within India. All this is done in private peering sessions (check AS15169 record at peeringdb.net).
      I hope this answers to your questions.

  6. Hi Anurag,
    Thanks for update.
    What i am trying to research is : Any law that stops private corporations from setting up a IXP in India ?
    In same context have scanned content & policies available in public domain but did not come across any which stops one from doing so ?
    Do let me know if you come across anything i may have skipped.
    Thanks & keep up the good work…your blogs are interesting…do share your number on mail, will look forward to cacth-up with you over the wire.

    • Hi
      >Any law that stops private corporations from setting up a IXP in India ?
      As far as I understand – none. One can start an IXP. There can be an argument whether we need ISP license or not since technically one needs license when “selling bandwidth” as per terms. Now is traffic exchange at an IXP is “selling of traffic” or not is hard to say in legal terms.
      Thanks for all your kind words. Talk to you soon. 🙂

  7. Anurag, we are a B class license holder and we have setup our own IXP in Mumbai and there is no law for IXP, because IXP is most important for a country, and its only traffic flow, not selling internet. IXP can only sell port. So, its completely legal.

    • Hi…
      can you guide me for ISP related querries,
      i want to start own ISP in a small city ( tier 2/3 city) in HARYANA.
      ISP license Class “C”
      firstly for setup a ISP control room/server, which company provide me hardware/software consultancy.
      what is the bandwidth price i have to pay to big ISP like Airtel/BSNL and what is the pricing model after joining NIXI

    • HI..
      what is the bandwidth price i have to pay to big ISP like Airtel/BSNL and what is the pricing model after joining NIXI.
      if you can give me detailed with example.
      its helpful for me

  8. Hi…
    can anyone guide me for ISP related querries,
    i want to start own ISP in a small city ( tier 2/3 city) in HARYANA.
    ISP license Class “C”
    firstly for setup a ISP control room/server, which company provide me hardware/software consultancy.
    what is the bandwidth price i have to pay to big ISP like Airtel/BSNL and what is the pricing model after joining NIXI

  9. Pingback: National Internet Exchange of India starts allowing content providers to peer

Leave a Reply to Bharat Kumar Cancel reply

%d bloggers like this: