Tata-Communications

A friend of mine buzzed me yesterday about his missing route objects. Later multiple other ISPs told the same story which triggered me to put this as a question on INNOG Mailing list. Many folks replied of missing route objects there and it seems to be limited to IRINN members only. I also asked the same question on APNIC mailing list and it was again confirmed about the issue.
Before I proceed further, here’s what it is all about.

I am in Chiang Mai, Thailand for APNIC 48 conference. Earlier today attended APIX meeting where many IX members from Asian community gave an update including NIXI i.e National Internet Exchange of India.

As per the update NIXI now allows content players to peer at the exchange. NIXI earlier had a strict requirement of telecom license for anyone to peer but as of now it allows anyone with IP address and AS number to be part of the exchange just like all other exchanges. This is a really good development coming this year after their announcement of the removal of x-y charge. One strange thing remains that their website is still not updated to reflect that which is probably just work in progress. As per representative from NIXI they now openly welcome all content players to peer at NIXI.

And it’s less than a week before the INNOG 2 i.e Indian Network Operators Group Conference 2. We (Indians) are little late to start a NOG but it’s finally working out and this is the 2nd event. First one happened last year.

Event website: www.innog.net

 

Why INNOG is important and why we should care?

Well, having a functional NOG is as important for local community as a working Internet Exchange Point. In absence of either people just start peering outside for an expensive price. There a lot of things which Indian Network Operators need to work on and without knowledge sharing that’s just not going to happen.

And here goes first blog post of 2018. Last few months went busy with some major changes in personal life. :) I looked into Amazon’s India connectivity with various ASNs tonight. Here’s how it looks like. (Note: Jump to bottom most to skip traces and look at the summary data).  

 

Traceroutes

Amazon India to Vodafone India

traceroute to 118.185.107.1 (118.185.107.1), 30 hops max, 60 byte packets
 1 ec2-52-66-0-128.ap-south-1.compute.amazonaws.com (52.66.0.128) 21.861 ms ec2-52-66-0-134.ap-south-1.compute.amazonaws.com (52.66.0.134) 19.244 ms 19.233 ms
 2 100.64.2.200 (100.64.2.200) 14.789 ms 100.64.0.200 (100.64.0.200) 20.731 ms 100.64.3.12 (100.64.3.12) 13.187 ms
 3 100.64.0.193 (100.64.0.193) 14.418 ms 100.64.3.69 (100.64.3.69) 15.469 ms 100.64.3.67 (100.64.3.67) 15.946 ms
 4 100.64.16.67 (100.64.16.67) 0.343 ms 100.64.17.165 (100.64.17.165) 0.312 ms 100.64.17.199 (100.64.17.199) 0.313 ms
 5 52.95.67.213 (52.95.67.213) 1.942 ms 52.95.67.209 (52.95.67.209) 1.967 ms 52.95.67.213 (52.95.67.213) 1.935 ms
 6 52.95.66.218 (52.95.66.218) 4.998 ms 4.694 ms 52.95.66.130 (52.95.66.130) 4.650 ms
 7 52.95.66.67 (52.95.66.67) 1.752 ms 52.95.66.89 (52.95.66.89) 1.850 ms 1.806 ms
 **8 52.95.217.183 (52.95.217.183) 3.111 ms 3.102 ms 3.088 ms <- Amazon India**
 **9 182.19.106.204 (182.19.106.204) 3.426 ms 4.547 ms 4.537 ms <- Vodafone India**
10 118.185.107.1 (118.185.107.1) 2.035 ms 2.059 ms 2.039 ms

 

One of frequent email and contact form message I get my blog is about available content networks in India and where one can peer. There are certain content networks in India and of course most of the content networks have open peering policy and are usually happy with direct inter-connection (we call as “peering”) with the ISP networks (often referred to as “eyeball networks”). Some of these networks have a backbone which connects back to their key datacenter locations on their own circuits via Singapore/Europe, some other have simply placed their caching server where cache fill happens over IP transit. Based on publically known information across community and of course peeringdb, following content players are available in India and known to be open for peering:

So it seems like apart from voice interconnect issues, Jio is also facing routing issues on the backbone. I ran a trace to one of IP’s on Jio network allocated to end customer - 169.149.212.122. I ran trace from all Indian RIPE Atlas probes measurement here. There seem quite a few RIPE Atlas probes which are giving latency on 150ms + range. Seems like they are downstream or downstream of downstream of Tata Comm’s AS4755 and routing is happening via Singapore!    

It’s great to see Amazon announcement two days back about launch of their region in Mumbai. In past I was quite happy to see their Cloudfront CDN PoPs in Mumbai & Chennai (blog post here). Now it’s just great to see a full AWS region out of Mumbai. :) Though it’s going to eat most of important customers from the smaller players still it’s good for industry as industry is too big and we need more & more of such large Cloud players in India to bring more and more content hosting in India.

So last month I had a wonderful trip to Bangladesh for bdNOG. This is bit delayed.  

Some thoughts on infrastructure

1. In terms of infrastructure - roads & traffic, power, quality of builds - it seemed like India in 2000’s.
2. Specifically roads and traffic was bit terrible and even as an Indian (who manages to drive in Indian traffic!) I still got scared out of traffic in Dhaka. Speeds, roughness and overtaking is pretty high.
3. There was no Uber and app based services are still pretty low. It was mostly usual “yellow taxi” which one had to call. (And it was expensive by local standards).
4. There was excessive, just excessive amounts of overhead cabling in Dhaka and most of key city areas. It’s worth noting that there is way more overhead fiber than India. I guess most of it was running “active ethernet” based solutions (not a PON).  Most was just via media converters on both ends.
5. I got 30Mbps speeds in cheap budget hotel in Dhaka which was more higher then what I have ever seen in India! (Speedtest here)
6. Bangladesh currently is connected to outside world via SEA-ME-WE4 (landing at Cox’s Bazaar) and a terrestrial cable route via Kolkata.
7. Overall network connectivity with India is decent since many large Bangladeshi networks buy transit from Tata Communications (AS6453) and Airtel (AS9498). So mostly there’s direct path to India and if not direct then via Singapore which added bit of latency but was not as bad as India-China routes.
8. Bangladesh has a real & functional internet exchange :)

   

There seems be an ongoing route leak by AS49505 (Selectel, Russia) for K root server.

K root server’s IP: 193.0.14.129
Origin Network: AS25152  

Here’s trace from Airtel Looking Glass, Delhi PoP

Mon Oct 26 16:21:18 GMT+05:30 2015
traceroute 193.0.14.129
Mon Oct 26 16:21:22.053 IST
Type escape sequence to abort.
Tracing the route to 193.0.14.129
 1   \*
    203.101.95.146 19 msec  4 msec
 2  182.79.224.73 14 msec  3 msec  1 msec
 3  14.141.116.89.static-Delhi.vsnl.net.in (14.141.116.89) 7 msec  3 msec  2 msec
 4  172.23.183.134 26 msec  45 msec  26 msec
 5  ix-0-100.tcore1.MLV-Mumbai.as6453.net (180.87.38.5) 151 msec  153 msec  152 msec
 6  if-9-5.tcore1.WYN-Marseille.as6453.net (80.231.217.17) \[MPLS: Label 383489 Exp 0\] 160 msec  163 msec  155 msec
 7  if-2-2.tcore2.WYN-Marseille.as6453.net (80.231.217.2) \[MPLS: Label 595426 Exp 0\] 161 msec  162 msec  162 msec
 8  if-7-2.tcore2.FNM-Frankfurt.as6453.net (80.231.200.78) \[MPLS: Label 399436 Exp 0\] 149 msec  151 msec  155 msec
 9  if-12-2.tcore1.FNM-Frankfurt.as6453.net (195.219.87.2) 164 msec  163 msec  159 msec
 10 195.219.156.146 153 msec  151 msec  160 msec
 11 spb03.transtelecom.net (188.43.1.226) 190 msec  192 msec  189 msec
 12 Selectel-gw.transtelecom.net (188.43.1.225) 185 msec  185 msec  185 msec
 13 k.root-servers.net (193.0.14.129) 183 msec  204 msec  196 msec
RP/0/8/CPU0:DEL-ISP-MPL-ACC-RTR-9#

The routing information (show route 193.0.14.129 output) from their looking glass doesn’t seems useful since it shows that it’s learning K root Noida route via NIXI. This is likely because routing information is different from actual forwarding information in that device. So the trace looks extremely weird. It’s leading traffic to K root which does has anycast instance in Noida, landing into Russia!   Why is that happening? Let’s look at what Tata Communications (AS6453) routing table has for K root’s prefix. I am looking at feed of AS6453 which it’s putting into RIPE RIS RRC 03 collector.

K root in Noida seems to be not getting enough traffic from quite sometime and connectivity does seems bit broken. This is a blog post following up to Dyn’s excellent and detailed post about how TIC leaked the world famous 193.0.14.0/24 address space used by AS25152. It was good to read this post from RIPE NCC written by my friend Emile (and thanks to him for crediting me to signal about traffic hitting outside!)