Isp-Column

Lately, I have been struggling to keep latency in check between my servers in India and Europe. Since Nov 2021 multiple submarine cables are down impacting significant capacity between Europe & India. The impact was largely on Airtel earlier but also happened on Tata Comm for a short duration. As of now Airtel is still routing traffic from Europe > India towards downstream networks via the Pacific route via EU > US East > US West > Singapore path. Anyways, this blog post is not about the submarine cable issue.

Background

Lately, NIXI has been making a bit of news in the Indian peering ecosystem. NIXI for those who may not be aware is the National Internet Exchange of India. It was founded in 2003 with the idea to provide inter-connection layer 2 peering fabric for local Indian ISPs. They were supposed to ensure domestic Indian traffic is exchanged within India and not outside of India. In my previous post, I did cover how that is not true for now. They never picked up much interconnection due to a number of fundamental issues with their policies.

After my last post about home networking, I am jumping back into global routing. More specifically how Indian traffic is hitting the globe when it does not need to. This is an old discussion across senior management folks in telcos, policymakers, and more. It’s about “Does Indian internet traffic routes from outside of India?” and if the answer is yes then “Why?” and “How much?”

It became a hot topic, especially after the Snowden leaks. There was even an advisory back in 2018 from Deputy National Security Advisor to ensure Indian internet traffic stays local (news here). Over time this has come up a few dozen times in my discussion with senior members from the Indian ISP community, individuals, and even latency-sensitive gamers. So I am going to document some of that part here. I am going to put whatever can be verified publically and going to avoid putting any private discussions I had with friends in these respective networks. The data specially traceroutes will have measurement IDs from RIPE Atlas so they can be independently verified by other network engineers.

Over last couple of years I posted updates on Facebook caching nodes (FNA) deployment across the world. If you would like to read the logic I am using to pull the data, you can check the original post here. While the data is about Facebook FNA, it’s highly likely that networks would have Google GGC nodes alongside (a bit less) Akamai caches.

My last post about it was back in Nov 2019 and it seems just about the time to do a fresh check. So here we go…

Earlier today I saw twitter feed of bgpstream about Vodafone AS55410 hijacking a prefix from Brazil.

BGP,HJ,hijacked prefix AS270497 24.152.117.0/24, RUTE MARIA DA CUNHA, BR,-,By AS55410 VIL-AS-AP Vodafone Idea Ltd, IN, https://t.co/WvDvQMMDCf

— Cisco BGPStream (@bgpstream) April 16, 2021

 

Soon my friend Doug Madory tweeted about large scale hijack coming from Vodafone AS55410.

Large BGP routing leak out of India this morning.

AS55410 mistakenly announced over 30,000 BGP prefixes causing a 13x spike in inbound traffic to their network according to @kentikinc netflow data.

(cc: @anurag_bhatia, @aftabsiddiqui, @jaredmauch) pic.twitter.com/PQ4iiTKD2Q

Last month I did a short webinar with Indian ISPs talking about DNS servers in detail. The idea of the session was to make network engineers from fellow ISPs familiar with root DNS servers, DNS hierarchy, anycast etc. As we went through slides it was clear from RIPE Atlas data that Indian networks are not reaching local DNS servers due to routing! (Data from RIPE Atlas here).

This may come as a surprise for policymakers (where there seem to be ongoing discussions around how India can have its own root DNS servers even though) we are not hitting existing local root DNS instances. Anyways does that statement of having own root DNS servers even possible?

And a blog post after a while. Last few months went busy with RPKI. After my last post about RPKI and the fact that India was lacking a little bit on RPKI ROA front, we started with a major push by a set of like-minded folks like us. For now, Indian signed table has jumped from 12% since Aug to 32% now in Oct. Detailed graphs and other data can be found here on the public Grafana instance.

So based on my friend - Abdul Awal’s tweet, I started looking at the latest RPKI ROA data for India. His Tweet came when I was in the middle of moving my blog from WordPress running over LXC containers to now WordPress over docker with Bitnami image. Bit of optimisation is still pending.

My routing security project with @nsrcworld & @mozilla started in Oct-19 to improve #RPKI deployment in South Asia+Myanmar. The two graphs show tremendous growth in RPKI and how #INDIA is lagging behind. @anurag_bhatia @RoutingMANRS @routinator3000 @AbuseIp @routingtablepod pic.twitter.com/HAb93XED5C

A friend of mine buzzed me yesterday about his missing route objects. Later multiple other ISPs told the same story which triggered me to put this as a question on INNOG Mailing list. Many folks replied of missing route objects there and it seems to be limited to IRINN members only. I also asked the same question on APNIC mailing list and it was again confirmed about the issue.
Before I proceed further, here’s what it is all about.

Back in 2017 Google shared details about Espresso which is their SDN solution for scaling up their routing.
Saw this fascinating presentation from Google at SIGCOMM 2017. This blog post covers it in detail besides the talk.

 

Key design principles for their routing platform

1. Hierarchical control plane consisting of both global as well as local control. Global takes care of overall traffic flow, inputs coming from performance metric etc while local take care of failure of BGP sessions, port/device failure etc.