Bgp

Background

Lately, NIXI has been making a bit of news in the Indian peering ecosystem. NIXI for those who may not be aware is the National Internet Exchange of India. It was founded in 2003 with the idea to provide inter-connection layer 2 peering fabric for local Indian ISPs. They were supposed to ensure domestic Indian traffic is exchanged within India and not outside of India. In my previous post, I did cover how that is not true for now. They never picked up much interconnection due to a number of fundamental issues with their policies.

A developer friend recently asked me about the design of redundancy on servers. He had a valid point - running BGP can be tricky and expensive since most colo & datacenter host would offer simple static routing & usually with just a couple of IP addresses. Furthermore, due to IPv4 exhaustion, the prices of /24 have shot off pretty massively. On top of this burning, a /24 on single or multiple servers is also a questionable design practice unless one of hosting & selling hundreds of virtual machines on those servers.

After my last post about home networking, I am jumping back into global routing. More specifically how Indian traffic is hitting the globe when it does not need to. This is an old discussion across senior management folks in telcos, policymakers, and more. It’s about “Does Indian internet traffic routes from outside of India?” and if the answer is yes then “Why?” and “How much?”

It became a hot topic, especially after the Snowden leaks. There was even an advisory back in 2018 from Deputy National Security Advisor to ensure Indian internet traffic stays local (news here). Over time this has come up a few dozen times in my discussion with senior members from the Indian ISP community, individuals, and even latency-sensitive gamers. So I am going to document some of that part here. I am going to put whatever can be verified publically and going to avoid putting any private discussions I had with friends in these respective networks. The data specially traceroutes will have measurement IDs from RIPE Atlas so they can be independently verified by other network engineers.

So based on my friend - Abdul Awal’s tweet, I started looking at the latest RPKI ROA data for India. His Tweet came when I was in the middle of moving my blog from WordPress running over LXC containers to now WordPress over docker with Bitnami image. Bit of optimisation is still pending.

My routing security project with @nsrcworld & @mozilla started in Oct-19 to improve #RPKI deployment in South Asia+Myanmar. The two graphs show tremendous growth in RPKI and how #INDIA is lagging behind. @anurag_bhatia @RoutingMANRS @routinator3000 @AbuseIp @routingtablepod pic.twitter.com/HAb93XED5C

Back in 2017 Google shared details about Espresso which is their SDN solution for scaling up their routing.
Saw this fascinating presentation from Google at SIGCOMM 2017. This blog post covers it in detail besides the talk.

 

Key design principles for their routing platform

1. Hierarchical control plane consisting of both global as well as local control. Global takes care of overall traffic flow, inputs coming from performance metric etc while local take care of failure of BGP sessions, port/device failure etc.