Tata-Communications

Today I met a good friend and he has recently moved back into Rohtak (like me!) and was crying over BSNL’s issues. He has issues of unstable DSL due to last mile and I told him that even if last mile works well, BSNL still has got ton of issues with their IP backbone traffic.   It’s Sunday late night out here in India and I am having really pathetic connectivity with just everywhere except Google. With Google only key difference I noted is that my TCP session to Google’s services is terminating at Mumbai and not Delhi anymore. First and formost, I did trace to spectranet.in (which is last company I was working for) to see how is my latency with server hosting it:

And my post on 1st April. Don’t take it as April fool post ;)

Multiple times NIXI’s AS24029 has been reported as acting like transit ASN for multiple networks. I have analysed it in past and this is very much because of route leaks by few specific networks. I have explained difference in peering Vs transit routes and their handling previously on my blog.

In short: A network is supposed to re-announce it’s peering and transit routes only to customer and not to any other peer or upstream. Whenever NIXI’s ASN appears in global routing table, its always the case where one or more networks are re-announcing routes learnt via NIXI to their upstream transits. 

I thought about this long back - “Who pays to whom in case of internet bandwidth?” I have been working in this domain from sometime and so far I have learnt that it’s really complex. I will try to put a series of blog post to give some thoughts on this subject. Firstly we have to understand that when we talk about “bandwidth price” it’s often layer 3 bandwidth which you buy in form of capacity over ethernet GigE, Ten-GigE and so on (or STMs if you are in India). As we know from back school class in networking - layer 3 works over layer 2 and so to deliver “bandwidth” on layer 3, one needs layer 2 physical circuit. Price paid by companies on layer 2 Vs layer 3 varies significantly based on their location, type of business, their target goal etc. E.g a content heavy company like Google pays hell lot of money on layer 2 circuits while it is strongly believed among networking community that Google is a tier 1 network and hence a “transit free” zone and they do not pay any amount on layer 3. In general the trend is pretty much as big networks have larger network footprint and connected “PoPs” over layer 2 (leading to a higher layer 2 bill) while relatively lower layer 3 bill while small networks depend significantly just on transit bandwidth (in form of layer3) and have very low layer 2 footprint.  

Earlier this month Dyn started with it’s Indian PoP. I came across news from Dyn’s blog post. It’s very good to see first Amazon AWS and now Dyn in India. With a warm welcome to Dyn let’s look at their Indian deployment.

Dyn using AS33517 which seems to be having upstream from Tata-VSNL AS4755 and Airtel AS9498. 

Dyn seems to be announcing 103.11.203.0/24 to both networks in Mumbai to transit. There are routes in global IPv4 routing table which show Tata & Airtel as transit for Dyn. It cannot be just a /24. I am sure there are more prefixes which are very likely locally announced. Since deployment is at Mumbai, let’s try to look at NIXI Mumbai for prefixes.We can see Tata AS4755 is using 218.100.48.85 and Airtel is using 218.100.48.86 from NIXI route server at Mumbai with simple “sh ip bgp sum” query. I tried taking entire table of Tata as well as of Airtel from NIXI route server but not able to get it beyond few thousand routes. 

Today I spotted some routes from Amazon AWS Cloud services -  AS16509 in Indian tables. AS16509 was originating prefixes while sitting in downstream of Tata-VSNL AS4755 and Reliance AS18101. I almost missed Amazon AWS's announcement on their blog about Indian PoPs for their DNS service - Route53 and CDN service - Cloudfront.

New PoP’s of Amazon in India are at Mumbai and Chennai and I see pretty much consistent BGP announcements to Tata and Reliance from these locations. Prefixes I have seen so far:

Sometimes we see interesting IP’s in traceroute & they confuse lot of people.

I have seen this topic in discussion twice on NANOG and once on Linux Delhi user group. 

OK - let’s pick an example: 

anurag:~ anurag$ traceroute 71.89.140.11
traceroute to 71.89.140.11 (71.89.140.11), 64 hops max, 52 byte packets
1 router (10.10.0.1) 1.176 ms 0.993 ms 0.941 ms
2 117.220.160.1 (117.220.160.1) 20.626 ms 29.101 ms 19.216 ms
3 218.248.169.122 (218.248.169.122) 23.983 ms 43.850 ms 45.057 ms
4 115.114.89.21.static-mumbai.vsnl.net.in (115.114.89.21) 118.094 ms 81.447 ms 66.838 ms
5 172.31.16.193 (172.31.16.193) 115.979 ms 90.947 ms 90.491 ms
6 ix-4-2.tcore1.cxr-chennai.as6453.net (180.87.36.9) 95.778 ms 98.601 ms 98.920 ms
7 if-5-2.tcore1.svw-singapore.as6453.net (180.87.12.53) 321.174 ms
if-3-3.tcore2.cxr-chennai.as6453.net (180.87.36.6) 331.386 ms 326.671 ms
8 if-6-2.tcore2.svw-singapore.as6453.net (180.87.37.14) 317.442 ms
if-2-2.tcore2.svw-singapore.as6453.net (180.87.12.2) 334.647 ms 339.289 ms
9 if-7-2.tcore2.lvw-losangeles.as6453.net (180.87.15.26) 318.003 ms 328.334 ms 309.234 ms
10 if-2-2.tcore1.lvw-losangeles.as6453.net (66.110.59.1) 306.500 ms 326.194 ms 341.537 ms
11 66.110.59.66 (66.110.59.66) 315.431 ms 330.417 ms 308.372 ms
12 dls-bb1-link.telia.net (213.155.136.40) 354.768 ms 344.360 ms 357.050 ms
13 chi-bb1-link.telia.net (80.91.248.208) 352.479 ms 358.751 ms 359.987 ms
14 cco-ic-156108-chi-bb1.c.telia.net (213.248.89.46) 367.467 ms 370.482 ms 377.280 ms
15 bbr01aldlmi-bue-4.aldl.mi.charter.com (96.34.0.98) 387.269 ms 385.362 ms 365.694 ms
16 crr02aldlmi-bue-2.aldl.mi.charter.com (96.34.2.11) 375.275 ms 375.356 ms 371.621 ms
17 dtr02grhvmi-tge-0-1-0-0.grhv.mi.charter.com (96.34.34.83) 383.539 ms 371.817 ms 383.804 ms
18 dtr02whthmi-tge-0-1-0-0.whth.mi.charter.com (96.34.34.85) 384.400 ms 391.197 ms 393.340 ms
19 dtr02ldngmi-tge-0-1-0-0.ldng.mi.charter.com (96.34.34.87) 371.192 ms 375.679 ms 378.457 ms
20 acr01mnplmi-tge-0-0-0-3.mnpl.mi.charter.com (96.34.40.75) 364.824 ms 385.534 ms 374.401 ms
21 * *^C
anurag:~ anurag$

Let’s try pinging IP on 14th hop (which is with a major backbone Telia) - 213.248.89.46

Today I noticed some traffic on my blog from a link from Broadband forum. 

Here’s what poster wrote:

I made a thread a few days ago complaining about BSNL’s horrible routing. Well it looks like it has been fixed. I thank all the guys who made efforts to bring this to BSNL’s notice. Especially Anurag Bhatia who highlighted the issue with much detail on his blog

Softlayer connectivity problem and possible fix.

And finally academic session over. Done with all vivas and related stuff. Next will be exams likely in June. Time for me to get ready for travel. :)   Anyways an interesting topic for today’s post - Google Public DNS. Lot of us are familier with popular (and free) DNS resolvers 8.8.8.8 and 8.8.4.4. I have covered reason in previous posts on why it tends to fail with Content Delivery networks like Akamai which rely on anycasting at bottom DNS layer and simple unicasting on application servers. Anycasted DNS nodes point to application servers based on various factors like distance, load, cost etc out of interesting algorithms these CDN networks use for load & cost management.   Anyways today’s post focus is not CDN issues with these resolvers but Google Public DNS itself. Are these servers located in India and everywhere else where Google has PoPs?   Let’s do a simple trace to get forward path from Airtel to Google’s 8.8.8.8:

Today a friend from Pakistan informed about SMW4 outage. He reported about issues in Pakistan.

It seems like SMW4 is damaged near Egypt and that is what causing high load on East Asian routes giving pretty high latency.

I am at my home and sitting BSNL’s network and latency with Europe has jumped terribly to 700-800ms. Right now I do not see a direct route to Europe and it’s rather taking East Asia > US > Europe routes right now on other cable networks.

Just was looking at routing tables of BSNL. They have a significant address space in /10 - 117.192.0.0/10. Overall this /10 address space is divided into /18 and /20 subnets.

Let’s pick two of such subnets and observe routing tables from route-views:

1. 117.192.0.0/18
2. 117.192.0.0/20 

Routing table for 117.192.0.0/18: 

* 117.192.0.0/18 193.0.0.56 0 3333 3356 6453 4755 9829 9829 9829 i  
* 194.85.102.33 0 3277 3216 6453 4755 9829 9829 9829 i  
* 194.85.40.15 0 3267 174 6453 4755 9829 9829 9829 i  
* 129.250.0.11 6 0 2914 6453 6453 4755 9829 9829 9829 i  
* 128.223.253.10 0 3582 3701 3356 6453 4755 9829 9829 9829 i  
* 4.69.184.193 0 0 3356 6453 4755 9829 9829 9829 i  
* 209.124.176.223 0 101 101 3356 6453 4755 9829 9829 9829 i  
* 69.31.111.244 3 0 4436 2914 6453 6453 4755 9829 9829 9829 i  
* 207.46.32.34 0 8075 6453 4755 9829 9829 9829 i  
* 66.59.190.221 0 6539 6453 4755 9829 9829 9829 i  
* 12.0.1.63 0 7018 6453 4755 9829 9829 9829 i  
* 208.74.64.40 0 19214 2828 6453 4755 9829 9829 9829 i  
* 203.181.248.168 0 7660 2516 6453 4755 9829 9829 9829 i  
* 66.185.128.48 111 0 1668 6453 4755 9829 9829 9829 i  
* 134.222.87.1 0 286 6453 4755 9829 9829 9829 i  
* 157.130.10.233 0 701 6453 4755 9829 9829 9829 i  
* 114.31.199.1 0 0 4826 6939 1299 6453 4755 9829 9829 9829 i  
* 89.149.178.10 10 0 3257 6453 4755 9829 9829 9829 i  
* 154.11.98.225 0 0 852 3561 6453 4755 9829 9829 9829 i  
* 202.249.2.86 0 7500 2497 6453 4755 9829 9829 9829 i  
* 154.11.11.113 0 0 852 3561 6453 4755 9829 9829 9829 i  
* 144.228.241.130 0 1239 6453 4755 9829 9829 9829 i  
* 217.75.96.60 0 0 16150 1299 6453 4755 9829 9829 9829 i  
* 207.172.6.20 0 0 6079 3356 6453 4755 9829 9829 9829 i  
* 206.24.210.102 0 3561 6453 4755 9829 9829 9829 i  
* 195.66.232.239 0 5459 6453 4755 9829 9829 9829 i  
* 208.51.134.254 2523 0 3549 6453 4755 9829 9829 9829 i  
* 207.172.6.1 0 0 6079 3356 6453 4755 9829 9829 9829 i  
* 216.218.252.164 0 6939 1299 6453 4755 9829 9829 9829 i  
* 203.62.252.186 0 1221 4637 6453 4755 9829 9829 9829 i  
*> 66.110.0.86 0 6453 4755 9829 9829 9829 i  
* 164.128.32.11 0 3303 6453 4755 9829 9829 9829 i  
* 202.232.0.2 0 2497 6453 4755 9829 9829 9829 i

Routing table for 117.192.0.0/20: