Networking

I thought about this long back - “Who pays to whom in case of internet bandwidth?” I have been working in this domain from sometime and so far I have learnt that it’s really complex. I will try to put a series of blog post to give some thoughts on this subject. Firstly we have to understand that when we talk about “bandwidth price” it’s often layer 3 bandwidth which you buy in form of capacity over ethernet GigE, Ten-GigE and so on (or STMs if you are in India). As we know from back school class in networking - layer 3 works over layer 2 and so to deliver “bandwidth” on layer 3, one needs layer 2 physical circuit. Price paid by companies on layer 2 Vs layer 3 varies significantly based on their location, type of business, their target goal etc. E.g a content heavy company like Google pays hell lot of money on layer 2 circuits while it is strongly believed among networking community that Google is a tier 1 network and hence a “transit free” zone and they do not pay any amount on layer 3. In general the trend is pretty much as big networks have larger network footprint and connected “PoPs” over layer 2 (leading to a higher layer 2 bill) while relatively lower layer 3 bill while small networks depend significantly just on transit bandwidth (in form of layer3) and have very low layer 2 footprint.  

Some free time here in Europe and thus time for another quick blog post & to take my mind away from depressing people!

One of impressive features of major European networks is support for BGP communities. In India it’s almost non-existent. Setting it up isn’t hard technically but from capacity management side, Indian ISPs are somewhat shy in setting it up.

Let’s put a case where we have a Customer router (R1 with AS1), upstream of customer (R2 with AS2), upstream of upstream (R3 with AS3), peer of upstream (R4 with router4). Let’s try to setup communities so that customer at AS1 can control his BGP announcements and announce some prefixes to AS3 and some to AS4 selectively to control inbound traffic flow. 

Today I was reading news on news.google.co.in. I follow news on many topics like datacenter, ISPs, IPv6 etc via Google News. Under the datacenter section I came across an interesting news. So two news : First about “Second datacenter in India by eBay” and other about second “eBay launches development centre”. I clicked on first news about datacenter and came to page which had title - “eBay launches 2nd development centre in India”. I thought I clicked wrong link, I closed it and opened datacenter link again and I landed on same page again. Then I opened both URL’s and here are the URL’s I saw in browser:  

Earlier this month Dyn started with it’s Indian PoP. I came across news from Dyn’s blog post. It’s very good to see first Amazon AWS and now Dyn in India. With a warm welcome to Dyn let’s look at their Indian deployment.

Dyn using AS33517 which seems to be having upstream from Tata-VSNL AS4755 and Airtel AS9498. 

Dyn seems to be announcing 103.11.203.0/24 to both networks in Mumbai to transit. There are routes in global IPv4 routing table which show Tata & Airtel as transit for Dyn. It cannot be just a /24. I am sure there are more prefixes which are very likely locally announced. Since deployment is at Mumbai, let’s try to look at NIXI Mumbai for prefixes.We can see Tata AS4755 is using 218.100.48.85 and Airtel is using 218.100.48.86 from NIXI route server at Mumbai with simple “sh ip bgp sum” query. I tried taking entire table of Tata as well as of Airtel from NIXI route server but not able to get it beyond few thousand routes. 

Few days back I noticed F root server (which is with ISC) brought it’s anycasted node in NIXI Chennai back live. They have taken that down as per my interaction with them over mailing list. My last post about F root coming back live was with guess work on who’s providing upstream.   Today I spent sometime in finding who’s providing transit to that node. It is very important to note that most of these key infrastructure related nodes rely on peering for most of traffic but a transit in form of full table or default stays so that one can push packets to a route if it is not in table learnt from peering. In case of Indian deployment which was at NIXI Chennai - many ISPs were following regional routes clause of NIXI and were announcing just their regional routes (to ISC’s F root router) but quite a few of them (like BSNL) were still learning routes from one region and exporting them into their other region via their IGP. This brought case where my router (sitting on BSNL link) was getting a forward path to NIXI Chennai for F root but there was return path from F root to my system because BSNL wasn’t announcing Northern prefixes in Chennai based NIXI. As I noted earlier F root is back live in India and I am getting consistant and direct routes. It seems very much the case of addition of transit on that node. Today I was looking at global table dump and I came across some interesting routes which revealed who is probably the transit for ISC’s F root in India. :)