Isp-Column

After my last post about home networking, I am jumping back into global routing. More specifically how Indian traffic is hitting the globe when it does not need to. This is an old discussion across senior management folks in telcos, policymakers, and more. It’s about “Does Indian internet traffic routes from outside of India?” and if the answer is yes then “Why?” and “How much?”

It became a hot topic, especially after the Snowden leaks. There was even an advisory back in 2018 from Deputy National Security Advisor to ensure Indian internet traffic stays local (news here). Over time this has come up a few dozen times in my discussion with senior members from the Indian ISP community, individuals, and even latency-sensitive gamers. So I am going to document some of that part here. I am going to put whatever can be verified publically and going to avoid putting any private discussions I had with friends in these respective networks. The data specially traceroutes will have measurement IDs from RIPE Atlas so they can be independently verified by other network engineers.

Over last couple of years I posted updates on Facebook caching nodes (FNA) deployment across the world. If you would like to read the logic I am using to pull the data, you can check the original post here. While the data is about Facebook FNA, it’s highly likely that networks would have Google GGC nodes alongside (a bit less) Akamai caches.

My last post about it was back in Nov 2019 and it seems just about the time to do a fresh check. So here we go…

Earlier today I saw twitter feed of bgpstream about Vodafone AS55410 hijacking a prefix from Brazil.

BGP,HJ,hijacked prefix AS270497 24.152.117.0/24, RUTE MARIA DA CUNHA, BR,-,By AS55410 VIL-AS-AP Vodafone Idea Ltd, IN, https://t.co/WvDvQMMDCf

— Cisco BGPStream (@bgpstream) April 16, 2021

Soon my friend Doug Madory tweeted about large scale hijack coming from Vodafone AS55410.

Large BGP routing leak out of India this morning.

AS55410 mistakenly announced over 30,000 BGP prefixes causing a 13x spike in inbound traffic to their network according to @kentikinc netflow data.

(cc: @anurag_bhatia, @aftabsiddiqui, @jaredmauch) pic.twitter.com/PQ4iiTKD2Q

Last month I did a short webinar with Indian ISPs talking about DNS servers in detail. The idea of the session was to make network engineers from fellow ISPs familiar with root DNS servers, DNS hierarchy, anycast etc. As we went through slides it was clear from RIPE Atlas data that Indian networks are not reaching local DNS servers due to routing! (Data from RIPE Atlas here).

This may come as a surprise for policymakers (where there seem to be ongoing discussions around how India can have its own root DNS servers even though) we are not hitting existing local root DNS instances. Anyways does that statement of having own root DNS servers even possible?

And a blog post after a while. Last few months went busy with RPKI. After my last post about RPKI and the fact that India was lacking a little bit on RPKI ROA front, we started with a major push by a set of like-minded folks like us. For now, Indian signed table has jumped from 12% since Aug to 32% now in Oct. Detailed graphs and other data can be found here on the public Grafana instance.

So based on my friend - Abdul Awal’s tweet, I started looking at the latest RPKI ROA data for India. His Tweet came when I was in the middle of moving my blog from WordPress running over LXC containers to now WordPress over docker with Bitnami image. Bit of optimisation is still pending.

My routing security project with @nsrcworld & @mozilla started in Oct-19 to improve #RPKI deployment in South Asia+Myanmar. The two graphs show tremendous growth in RPKI and how #INDIA is lagging behind. @anurag_bhatia @RoutingMANRS @routinator3000 @AbuseIp @routingtablepod pic.twitter.com/HAb93XED5C

A friend of mine buzzed me yesterday about his missing route objects. Later multiple other ISPs told the same story which triggered me to put this as a question on INNOG Mailing list. Many folks replied of missing route objects there and it seems to be limited to IRINN members only. I also asked the same question on APNIC mailing list and it was again confirmed about the issue.
Before I proceed further, here’s what it is all about.

Back in 2017 Google shared details about Espresso which is their SDN solution for scaling up their routing.
Saw this fascinating presentation from Google at SIGCOMM 2017. This blog post covers it in detail besides the talk.

Key design principles for their routing platform

1. Hierarchical control plane consisting of both global as well as local control. Global takes care of overall traffic flow, inputs coming from performance metric etc while local take care of failure of BGP sessions, port/device failure etc.

In Melbourne for the week for APRICOT 2020. Someone jokingly said it’s should be “APRICOT and RPKI 2020”. :-)

It seems like both JPNIC and TWNIC are doing a good job at promoting their member operators in Japan & Taiwan for signing ROA. I thought to check for the status in India to find how India is doing.

RPKI ROA status for India

1. Total prefixes: 40,834 (IPv4 + IPv6)
2. Prefixes with valid ROA: 4693
3. Prefixes with invalid ROA: 354
4. Prefixes without ROA: 35,787

I had calls with a couple of friends over this week and somehow discussion IPv6 deployment came up. “How much has been IPv6 deployment in India now in 2020” is a very interesting question. It’s often added with - “how much of my traffic will flow over IPv6 once it is enabled”?

Game of numbers

There is a drastic difference in IPv6 deployment depending on which statistic we are looking at here in India. There can be a bunch of factors based on which we can try to judge IPv6 deployment: