25 Nov

Peering with content networks in India

peering

One of frequent email and contact form message I get my blog is about available content networks in India and where one can peer. There are certain content networks in India and of course most of the content networks have open peering policy and are usually happy with direct inter-connection (we call as “peering“) with the ISP networks (often referred to as “eyeball networks”). Some of these networks have a backbone which connects back to their key datacenter locations on their own circuits via Singapore/Europe, some other have simply placed their caching server where cache fill happens over IP transit.

 

Based on publically known information across community and of course peeringdb, following content players are available in India and known to be open for peering:

  1. Google
  2. Microsoft
  3. Amazon
  4. Limelight

 

A quick list of these with datacenter names and locations as taken from Peeringdb record of these networks.

Organisation ASN City Datacenter Location
Amazon 16509 Mumbai GPX Mumbai Unit A-001, Boomerang Chandivali Farm Road, Near Chandivali Studio, Andheri East Mumbai, Mumbai, 400 051
Amazon 16509 Noida Sify Greenfort – Noida B7, Block A, Sector 132, Noida Expressway, Noida , UP 201304
Amazon 16509 Mumbai Tata Mumbai IDC LVSB, Opposite Kirti College
6th floor, Prabahdevi
Mumbai, MH, 400 028
Google 15169 Chennai Bharti Airtel Santhome Bharti Towers, 101 Santhome High Road, Chennai, 600 028
Google 15169 Mumbai GPX Mumbai Unit A-001, Boomerang Chandivali Farm Road, Near Chandivali Studio, Andheri East Mumbai, Mumbai, 400 051
Google 15169 Noida Sify Greenfort – Noida B7, Block A, Sector 132, Noida Expressway, Noida , UP 201304
Google 15169 Chennai TATA Communications Ltd 14th floor, 2nd block
4, Swami Sivanand Salai, Chennai, TN 600 002
Google 15169 Delhi Tata Delhi VSB, Bangla Sahib Road, New Delhi 110001
Google 15169 Mumbai Tata Mumbai IDC LVSB, Opposite Kirti College
6th floor, Prabhadevi
Mumbai, MH, 400 028
Limelight 55439 / 22822 Chennai Bharti Airtel Santhome Bharti Towers, 101 Santhome High Road, Chennai, 600 028
Limelight 55439 / 22822 Mumbai Netmagic Vikhroli Mehra Industrial Estate
LBS Marg, Vikhroli
Mumbai, 400 079
Microsoft 8075 Mumbai Bharti Airtel Mumbai Plot No, TPS-2, 14/3, 2nd floor
Dattatray Road, Linking Road Extension
Mumbai, 400054
Microsoft 8075 Chennai Bharti Airtel Santhome Bharti Towers, 101 Santhome High Road, Chennai, 600 028
Microsoft 8075 Chennai TATA Communications Ltd 14th floor, 2nd block
4, Swami Sivanand Salai, Chennai, TN 600 002
Microsoft 8075 Delhi Tata Communications Ltd – GK1 Greater Kailash-1
New Delhi, 110048
Microsoft 8075 Mumbai Tata Mumbai IDC LVSB, Opposite Kirti College
6th floor, Prabhadevi
Mumbai, MH, 400 028

 

Besides these Google also has an option of GGC, Akamai has an option of Akamai Caching server, Facebook has the option for caching server which is hosted inside ISP’s network and Netflix has an option for OCAs. Besides these networks there are known nodes of Verizon’s Edgecast in Delhi, Mumbai &¬†Chennai (as per this map), Cloudflare has nodes in Delhi, Mumbai & Chennai (as per this map), PCH & K-root server have a node with Web Werks available on MCH peering fabric and Dyn has a node in Mumbai (as per this map).

Go ahead and peer as after all it all starts with a handshake. ūüôā

06 Oct

K root server – Noida anycast and updates

K root in Noida seems to be not getting enough traffic from quite sometime and connectivity does seems bit broken. This is a blog post following up to Dyn’s excellent and detailed post about how TIC leaked the world famous¬†193.0.14.0/24 address space used by AS25152. It was good to read this post from RIPE NCC written by¬†my friend Emile (and thanks to him for crediting me to signal about traffic hitting outside!)

 

The route leak…

TIC¬†AS48159 was supposed to keep the route within it’s IGP but it leaked it to¬†Omantel AS8529 – a large International backbone which propagated route leak further to global table. It was mistake at by both players primarily by TIC for leaking route.

 

If we look at IPv4 route propagation graph of Omatel AS8529 on Hurricane Electric BGP tool kit, it shows two import ASNs:

 

Omantel IPv4 routing

 

 

This has AS9498 (Bharti Airtel) and AS6453 (Tata Communications). Both of these are extremely important networks and two of large International and domestic IP transit providers in India. Very likely Omantel is customer of Bharti Airtel and if we look at IRR record of Airtel as published in their peeringdb record: AS9498:AS-BHARTI-IN

 

Anurags-MacBook-Pro:~ anurag$ whois -h whois.apnic.net AS9498:AS-BHARTI-IN |grep -w AS8529
members: AS38476,AS45219,AS45264,AS45283,AS45514,AS45451,AS37662,AS45491,AS7642,AS45517,AS45514:AS-TELEMEDIA-SMB,AS45609,AS38740,As131210,AS45335,AS23937,AS132045,AS8529,AS132486,AS8164,AS133967,AS37048
Anurags-MacBook-Pro:~ anurag$

 

This also confirms the same. Airtel did picked this route and since it was a customer route, it had a higher local preference then the peering route Airtel learnt from NIXI Noida peering with  K root. For now route leak fixed and Airtel seems to be having good routing with K root anycast instance in Noida.

 

Current status

From Tata Communications – it’s yet not picking announcement of K root anycast instance from Noida since their peering session at NIXI Noida has been down from long time. NIXI moved over from STPI to Netmagic Sector 63 Noida in August¬†(see heavy drop of traffic in NIXI Noida graphs here). From that time onwards Tata’s domestic backbone AS4755’s peering session seems down.

 

As per NIXI’s connected parties page, Tata Comm’s IP is¬†218.100.48.30. From NIXI’s looking glass there seems to no peer on that IP !

 

Hence for now Tata Comm isn’t getting route at all from Noida instance and that explains reason for bad outbound path.

 

Example of trace from Tata Comm to K root:

 

even same stays for its downstream customers who have outbound via TCL:

 

 

Another issue which is causing serious trouble around K root is the fact that it appears to be broken IP transit pipe of K root Noida. Due to the way NIXI works, K root must have a IP transit pipe. I pointed long back about broken connectivity of root DNS servers due return path problems. After that both K root and i root got transit but seems like after NIXI moved over, IP transit has been broken for current setup in Netmagic.

 

Why “local node” of root server needs IP transit?

It needs transit because:

    1. NIXI has a weird pricing of “x-y” where requester pays and this leads to a quite high settlement amount for a network which has a high inbound traffic (eyeball network) – even few x times than that of transit! (paying 5Rs/GB!). This leads to scenario where networks do “partial prefix announcement” to keep their traffic balanced (or slightly in outbound direction) to avoid high settlement cost. Hence most of such eyeball networks announce their regional routes but avoid all routes while they still do learn K root’s route and inject in their IGP.This leads in case where K root’s¬†193.0.14.0/24 is leant by networks in West and South India and hence there’s a forward path from customers >>> K root Noida node. Now since these networks aren’t announcing their West or South Indian routes at NIXI Noida, there’s no return path for packets. Thus for root DNS to stay operationally stable (which they should since they are critical) they must have transit / default route to return packets as last resort to IP’s which aren’t visible via peering.
    2. Similar case of some other random leaked routes. E.g if a large ISP decided to learn K root route and announce to customer’s table thus leading to Customer > Large network > K root Noida path while not announcing that customer’s route at NIXI resulting in no return path.

 

 

So in short – It does needs transit but just for outbound traffic, not for announcing routes on the transit.

I have informed of broken connectivity issue to RIPE NCC and their team is actively working on the fix. Hopefully it would be fixed very soon!

 

With hope that your DNS is not getting resolved from other side of world, good night! ūüôā

 

Disclaimer: As usual – thoughts & comments are completely personal.