26 Feb

APRICOT 2016 – Auckland, New Zealand

First and foremost before talking about APRICOT, I must say I am deeply moved with impact Rohtak (and Haryana) as whole had because of recent Jat agitation. What I find extremely depressing is way current Govt. of Haryana completely failed to control it and the way previous Govt. ministers did best in their interest and completely against the interest of people of Haryana. For now quite hopeful with news that Mr Prakash Singh (one of my favorite IPS officers) who did quite well during his various terms is looking into failure of police. More details about the news here.

I will write more on this later on, not good time right now since tensions have yet to get normal.

 

Coming to APRICOT 2016 – I had a wonderful time in attending event and this was my first visit to New Zealand as well. This time I presented on “Misused top ASNs” and it went good overall. Link to my presentation here. Also, I had a short video interview with APNIC about my presentation (embedded below).

 

 

Also, last month I attended SANOG 27 at Kathmandu, Nepal. My detailed guest blog post about event on APNIC blog here.

15 Mar

BSNL routing tables and upstreams

Just was looking at routing tables of BSNL. They have a significant address space in /10 – 117.192.0.0/10. Overall this /10 address space is divided into /18 and /20 subnets.

 

Let’s pick two of such subnets and observe routing tables from route-views:

  1. 117.192.0.0/18
  2. 117.192.0.0/20 

 

Routing table for 117.192.0.0/18

* 117.192.0.0/18 193.0.0.56 0 3333 3356 6453 4755 9829 9829 9829 i
* 194.85.102.33 0 3277 3216 6453 4755 9829 9829 9829 i
* 194.85.40.15 0 3267 174 6453 4755 9829 9829 9829 i
* 129.250.0.11 6 0 2914 6453 6453 4755 9829 9829 9829 i
* 128.223.253.10 0 3582 3701 3356 6453 4755 9829 9829 9829 i
* 4.69.184.193 0 0 3356 6453 4755 9829 9829 9829 i
* 209.124.176.223 0 101 101 3356 6453 4755 9829 9829 9829 i
* 69.31.111.244 3 0 4436 2914 6453 6453 4755 9829 9829 9829 i
* 207.46.32.34 0 8075 6453 4755 9829 9829 9829 i
* 66.59.190.221 0 6539 6453 4755 9829 9829 9829 i
* 12.0.1.63 0 7018 6453 4755 9829 9829 9829 i
* 208.74.64.40 0 19214 2828 6453 4755 9829 9829 9829 i
* 203.181.248.168 0 7660 2516 6453 4755 9829 9829 9829 i
* 66.185.128.48 111 0 1668 6453 4755 9829 9829 9829 i
* 134.222.87.1 0 286 6453 4755 9829 9829 9829 i
* 157.130.10.233 0 701 6453 4755 9829 9829 9829 i
* 114.31.199.1 0 0 4826 6939 1299 6453 4755 9829 9829 9829 i
* 89.149.178.10 10 0 3257 6453 4755 9829 9829 9829 i
* 154.11.98.225 0 0 852 3561 6453 4755 9829 9829 9829 i
* 202.249.2.86 0 7500 2497 6453 4755 9829 9829 9829 i
* 154.11.11.113 0 0 852 3561 6453 4755 9829 9829 9829 i
* 144.228.241.130 0 1239 6453 4755 9829 9829 9829 i
* 217.75.96.60 0 0 16150 1299 6453 4755 9829 9829 9829 i
* 207.172.6.20 0 0 6079 3356 6453 4755 9829 9829 9829 i
* 206.24.210.102 0 3561 6453 4755 9829 9829 9829 i
* 195.66.232.239 0 5459 6453 4755 9829 9829 9829 i
* 208.51.134.254 2523 0 3549 6453 4755 9829 9829 9829 i
* 207.172.6.1 0 0 6079 3356 6453 4755 9829 9829 9829 i
* 216.218.252.164 0 6939 1299 6453 4755 9829 9829 9829 i
* 203.62.252.186 0 1221 4637 6453 4755 9829 9829 9829 i
*> 66.110.0.86 0 6453 4755 9829 9829 9829 i
* 164.128.32.11 0 3303 6453 4755 9829 9829 9829 i
* 202.232.0.2 0 2497 6453 4755 9829 9829 9829 i

 

Routing table for 117.192.0.0/20:

Network Next Hop Metric LocPrf Weight Path
* 117.192.0.0/20 193.0.0.56 0 3333 3356 6453 9829 i
* 194.85.102.33 0 3277 3216 6453 9829 i
* 194.85.40.15 0 3267 174 6453 9829 i
* 129.250.0.11 6 0 2914 6453 9829 i
* 128.223.253.10 0 3582 3701 3356 6453 9829 i
* 4.69.184.193 0 0 3356 6453 9829 i
* 209.124.176.223 0 101 101 2914 6453 9829 i
* 69.31.111.244 3 0 4436 2914 6453 9829 i
* 207.46.32.34 0 8075 6453 9829 i
* 66.59.190.221 0 6539 6453 9829 i
* 12.0.1.63 0 7018 6453 9829 i
* 208.74.64.40 0 19214 2828 6453 9829 i
* 203.181.248.168 0 7660 2516 209 6453 9829 i
* 66.185.128.48 111 0 1668 6453 9829 i
* 134.222.87.1 0 286 6453 9829 i
* 157.130.10.233 0 701 6453 9829 i
* 114.31.199.1 0 0 4826 6939 1299 6453 9829 i
* 144.228.241.130 0 1239 6453 9829 i
* 89.149.178.10 10 0 3257 6453 9829 i
* 154.11.98.225 0 0 852 2914 6453 9829 i
* 202.249.2.86 0 7500 2497 6453 9829 i
* 154.11.11.113 0 0 852 2914 6453 9829 i
* 217.75.96.60 0 0 16150 1299 6453 9829 i
* 207.172.6.20 0 0 6079 3356 6453 9829 i
* 195.66.232.239 0 5459 6453 9829 i
*> 66.110.0.86 0 6453 9829 i
* 216.218.252.164 0 6939 1299 6453 9829 i
* 206.24.210.102 0 3561 6453 9829 i
* 203.62.252.186 0 1221 4637 6453 9829 i
* 202.232.0.2 0 2497 6453 9829 i
* 208.51.134.254 2523 0 3549 6453 9829 i
* 207.172.6.1 0 0 6079 3356 6453 9829 i

 

 

Looking at table 1 for /18 – Clearly path is always via 6453 > 4755 > 9829 and then prepending. This is for sure BSNL’s transit links inside India since AS4755 is Tata-VSNL and is present only inside India (as far as I know!). Although it seems like /18 is not being announced to anyone else other then Tata-VSNL in for this subnet but I am sure there are some other prefixes which are announced to Reliance and Airtel as well. 

 

Next looking at table 2 for /20 – path is always 6453 > 9829 with no prepending. Clearly this is IPLC link since AS6453 is Tata Comm outside India and they usually do not connect to anyone inside India on 6453 but rather via Tata-VSNL AS4755 backbone. Thus likely 6453 > 9829 (direct with no AS4755 in between) is where BSNL is purchasing bandwidth outside India over IPLC. 

 

Now since /18 is less specific and even has long AS path – direct transit upstream link via Tata > Tata-VSNL > BSNL is being less preferred and IPLC is more preferred. 

 

There are also cases of /20’s where BSNL is not doing any prepends and announcing it to VSNL directly which actually is good since Tata AS6453 is connected to more networks in Asia, Europe and North America. While BSNL’s IPLC link seems only to London, New York and Los Angeles which causes issues in return paths (world > BSNL) entery points and eventually increases latency for all Asian traffic.

 

One of prefixes which is announced correctly –  117.200.48.0/20

Network Next Hop Metric LocPrf Weight Path
* 117.200.48.0/20 66.59.190.221 0 6539 6453 4755 9829 ?
* 164.128.32.11 0 3303 6453 4755 9829 ?
* 128.223.253.10 0 3582 3701 3356 6453 4755 9829 i
* 193.0.0.56 0 3333 3356 6453 4755 9829 i
* 194.85.102.33 0 3277 3216 6453 4755 9829 ?
* 194.85.40.15 0 3267 3356 6453 4755 9829 i
* 129.250.0.11 6 0 2914 6453 6453 4755 9829 ?
* 4.69.184.193 0 0 3356 6453 4755 9829 i
* 209.124.176.223 0 101 101 3356 6453 4755 9829 i
* 69.31.111.244 3 0 4436 2914 6453 6453 4755 9829 ?
* 207.46.32.34 0 8075 6453 4755 9829 ?
* 12.0.1.63 0 7018 6453 4755 9829 ?
* 208.74.64.40 0 19214 2828 6453 4755 9829 ?
* 203.181.248.168 0 7660 2516 6453 4755 9829 ?
* 66.185.128.48 111 0 1668 6453 4755 9829 ?
* 134.222.87.1 0 286 6453 4755 9829 ?
* 157.130.10.233 0 701 6453 4755 9829 ?
* 114.31.199.1 0 0 4826 6939 1299 6453 4755 9829 i
* 89.149.178.10 10 0 3257 6453 4755 9829 ?
* 154.11.98.225 0 0 852 1299 6453 4755 9829 ?
* 202.249.2.86 0 7500 2516 6453 4755 9829 ?
* 154.11.11.113 0 0 852 1299 6453 4755 9829 ?
* 144.228.241.130 0 1239 6453 4755 9829 ?
* 217.75.96.60 0 0 16150 1299 6453 4755 9829 i
* 207.172.6.20 0 0 6079 3356 6453 4755 9829 i
* 195.66.232.239 0 5459 6453 4755 9829 ?
* 208.51.134.254 2523 0 3549 6453 4755 9829 ?
* 207.172.6.1 0 0 6079 3356 6453 4755 9829 i
* 202.232.0.2 0 2497 6453 4755 9829 ?
*> 66.110.0.86 0 6453 4755 9829 ?
* 216.218.252.164 0 6939 1299 6453 4755 9829 i
* 206.24.210.102 0 3561 6453 4755 9829 ?
* 203.62.252.186 0 1221 4637 6453 4755 9829 ?

 

Well that’s all for now. With hope that your section for /20 is announced to everyone – have a good night! 🙂

10 Apr

BSNL routing tables screw up

It has been super boring evening considering my sessional tests tomorrow. Test time is dull as always. I have been precisely measnuring latency on BSNL link from BSNL Haryana to Singapore based servers. The fluctuation in latency is pretty much common now. Someones we get 120ms latency to Singapore (an expected number based on distance) while other time it goes off as high as 310ms. Latency with openDNS nodes in Singapore makes it pretty much poor to use openDNS here.

 

Based on my collected data and BGPlay’s routing records, here’s what’s happening. My IP is coming /20 BGP annoucement from BSNL Autonomous System 9829 – 117.207.48.0/20. Looking at BGP table records for that block from BGPlay’s routing data archive source.

 

On Sunday Morning 00:00am UTC, BSNL was found to be announcing 117.207.48.0/20 from AS9829 which was carried over via it’s upstream ISPs – Tata Communications (AS6453) and Reliance Globalcom (AS18101). From Tata’s AS6453 most of other Tier 1 backbones and many other small ISPs were getting routes. Similarly in case of Reliance, AS18101 was announcing blocks to it’s other network FLAG Telecom AS15412 which was further passing routes across many ISPs in the world. ISPs like Tinet, AT&T, Savvis, Seabone, Sprint etc were getting announcement via AS6453 while Level3, NTT, Hurricane Electric, Swisscom & many others were getting annoucements via Reliance-FLAG backbone.

At this instance – connectivity to network via Reliance route looks pretty good. All traffic comes via direct path – entering Reliance’s network from nearest point & next reaching BSNL. While at the same time, unfortunately in case of other path via Tata Communications – it seems like no matter where traffic orignates from, it is always routed via US. That is even if traffic is orignating from Singapore, it is being routed to India via US.

 

Quick check on Tata Communications AS6453 PoPs at this instant:

Router: gin-svq-core1
Site: SG, Singapore – SVQ, EQUNIX
Command: show ip bgp 117.207.48.0/20

BGP routing table entry for 117.207.48.0/20
Bestpath Modifiers: deterministic-med
Paths: (3 available, best #2)
Multipath: eBGP
9829
nyy-mcore4. (metric 3777) from tv2-core1. (tv2-core1.)
Origin IGP, valid, internal
Community:
Originator: nyy-mcore4.

9829
nyy-mcore4. (metric 3777) from hk2-core3. (hk2-core3.)
Origin IGP, valid, internal, best
Community:
Originator: nyy-mcore4.

9829
nyy-mcore4. (metric 3777) from s9r-core1. (s9r-core1.)
Origin IGP, valid, internal
Community:
Originator: nyy-mcore4.

 

Router: gin-lhx-core1
Site: GB, London – LHX, TATA COMM. HARBOR EXCHANGE
Command: show ip bgp 117.207.48.0/20

BGP routing table entry for 117.207.48.0/20
Bestpath Modifiers: deterministic-med
Paths: (2 available, best #1)
Multipath: eBGP
9829
nyy-mcore4. (metric 3036) from ldn-mcore3. (ldn-mcore3.)
Origin IGP, valid, internal, best
Community:
Originator: nyy-mcore4.

9829
nyy-mcore4. (metric 3036) from l78-tcore1. (66.110.10.237)
Origin IGP, valid, internal
Community:
Originator: nyy-mcore4.

 

 

Thus we can see in both cases – Tata’s router is getting updates from nyy-mcore4 which is their router in New York city. This forces packets all way down to New York before they are routed back in Asia to BSNL.

 

Next, same thing goes for next few hours. On Sunday noon time 12:55pm, we can see a path change from multiple providers like Savvis AS3561. We can see change in routing table and now between Tata AS6453 and BSNL AS9829, a new network comes in. It is AS4755 which is Tata Communications other network VSNL. Tata still uses AS4755 in India and AS6453 everywhere else.

Within next few seconds, we can see similar changes from AS293 – ESnet,

Path Change from 3561 6453 9829
to 3561 6453 4755 9829

Path Change from 293 6453 9829
to 293 6453 4755 9829

Path Change from 3303 15412 18101 9829
to 3303 6453 4755 9829

Path Change from  812 6453 9829
to 812 6453 4755 9829

Path Change from 3257 6453 9829
to 3257 6453 4755 9829

Path Change from 3130 1239 6453 9829
to 3130 1239 6453 4755 9829 

Path Change from 7018 6453 9829
to 7018 6453 4755 9829 

Path Change from 1299 6453 9829
to 1299 6453 4755 9829 

Within a min, we can see half of routes are going via AS4755 rather then AS6453 directly.

Next, we see a route withdrawal – 701 6453 9829 followed by route re-annoucement 701 6453 4755 9829 and with this we see whole block being announced via AS4755 (which exists only in India as per as I know). Next on 12:58pm, we see another series of path changes all reversing back to AS6453 – AS9829 skipping AS4755 in between. On 12:50pm we can see half of routes being back with AS6453 link directly and half still with AS6453. Very soon all routes get back on direct link AS4755 goes out of picture again.

Summary of what’s happening:

  1. BSNL’s network is having high latency on various routes including Singapore and Europe.
  2. For specific block in testing, we can see BSNL is announcing them via Tata Communications & Reliance Globalcom.
  3. Tata Communications uses two autonomous systems – AS4755 (VSNL) for Indian operations & AS6453 for everywhere else.
  4. We can see routing table changes almost daily which bring AS4755 between AS9829 and AS6453 on random basis.
  5. AS4755 is believed to be operated only in India and I can see whenever AS4755 is coming in picture, packets to BSNL-AS9829 are handed off directly and latency is pretty good.
  6. For other times when we have AS6453 > AS9829 routing, we can see block is being annouced from AS6453 in New York.
  7. Very likely problem is NOT from Tata Communications end, but rather from BSNL’s end. BSNL is constantly switching annoucing peers – AS4755 at one end in India while AS6453 router on other end in New York. Though it is hard to confirm this speculation.

 

 

 

Glad Airlines don’t route flights in the manner in which wrong routing goes! 🙂