27 Aug

Welcome to India Dyn!

Earlier this month Dyn started with it’s Indian PoP. I came across news from Dyn’s blog post. It’s very good to see first Amazon AWS and now Dyn in India. With a warm welcome to Dyn let’s look at their Indian deployment.

 

Dyn using AS33517 which seems to be having upstream from Tata-VSNL AS4755 and Airtel AS9498

 

Dyn seems to be announcing 103.11.203.0/24 to both networks in Mumbai to transit. There are routes in global IPv4 routing table which show Tata & Airtel as transit for Dyn. It cannot be just a /24. I am sure there are more prefixes which are very likely locally announced. Since deployment is at Mumbai, let’s try to look at NIXI Mumbai for prefixes.We can see Tata AS4755 is using 218.100.48.85 and Airtel is using 218.100.48.86 from NIXI route server at Mumbai with simple “sh ip bgp sum” query. I tried taking entire table of Tata as well as of Airtel from NIXI route server but not able to get it beyond few thousand routes. 

One thing for sure is that there cannot be any prefix which is announced ONLY in India and not outside. That is because that is not the way authoritative DNS selling works. 🙂 

 

Let’s look at ALL prefixes originated by AS33517 as observed by Oregon route-views (via it’s archive):

 

anurags-macbook-pro-lan:ASN-data anurag$ grep -w ‘33517 i’ oix-full-snapshot-latest.dat|cut -f 3 -d ‘ ‘ |sort -u
103.11.200.0/24
103.11.201.0/24
103.11.203.0/24
198.153.192.0/22
198.153.192.0/23
198.153.192.0/24
198.153.194.0/23
198.153.194.0/24
203.62.195.0/24
204.13.248.0/24
204.13.249.0/24
204.13.250.0/24
204.13.251.0/24
208.76.56.0/24
208.76.57.0/24
208.76.58.0/24
208.76.59.0/24
208.76.60.0/24
208.76.61.0/24
208.76.63.0/24
208.78.68.0/22
208.78.68.0/24
208.78.69.0/24
208.78.70.0/24
208.78.71.0/24
216.146.32.0/24
216.146.33.0/24
216.146.34.0/23
216.146.34.0/24
216.146.35.0/24
216.146.36.0/23
216.146.36.0/24
216.146.37.0/24
216.146.38.0/24
216.146.39.0/24
216.146.40.0/24
216.146.41.0/24
216.146.42.0/24
216.146.43.0/24
216.146.44.0/24
216.146.45.0/24
216.146.46.0/24
216.146.47.0/24
80.231.219.0/24
80.231.25.0/24
91.198.22.0/24
anurags-macbook-pro-lan:ASN-data anurag$ 

 

Now finding which of these are announced in India seems to be slightly laborus task. I am quite confident that Dyn would have asked it’s transit to carry prefix in Mumbai but not to export routes outside India. My initial idea was to simply write a macro script to visit NIXI’s Looking Glass and run “sh ip bgp $IP” one by one.

 

A Macro script which can do something like this in a loop:

 

VERSION BUILD=8300326 RECORDER=FX
TAB T=1
URL GOTO=http://www.nixi.in/lookingglass.php
FRAME F=1
TAG POS=1 TYPE=INPUT:RADIO FORM=ACTION:/lg/ ATTR=ID:bgp
TAG POS=1 TYPE=INPUT:TEXT FORM=ACTION:/lg/ ATTR=ID:addr CONTENT=204.13.251.0/24
TAG POS=1 TYPE=SELECT FORM=ACTION:/lg/ ATTR=ID:router CONTENT=%NIXI<SP>Mumbai
TAG POS=1 TYPE=INPUT:SUBMIT FORM=ACTION:/lg/ ATTR=VALUE:Submit
TAG POS=1 TYPE=PRE ATTR=TXT:%<SP>Network<SP>not<SP>in<SP>table
SAVEAS TYPE=CPL FOLDER=/Users/anurag/Downloads/ASN-data/dyn-routes FILE=+_{{!NOW:yyyymmdd_hhnnss}}
TAG POS=1 TYPE=A ATTR=TXT:Back

 

This somewhat works but few major limitations:

  1.  I am not able to take output in TXT. For TYPE=TXT I get a blank file. And for HTML output – the file has a frame which takes me further to a lg.html file in subdirectory. Script to read such output would be quite crazy!
  2. Limited time I have to code this (well this is personal fun work…not a serious one!).
  3. My limited programming skills 🙁

 

 

OK – so based on that I think Macro isn’t the way to go. What other way I can think off is to simply count hops or latency. Counting latency is do-able but hard because I have list of prefixes announced by Dyn and thus I would have to scan 256 (or 512) IP’s per prefix to see which ones reply for ICMP and based on that taking out their latency values. Hop count seems to be lot more easy and possible. Logic here would be to simply “extract” prefixes out of these 46 which are at less then say 10 hops distance. 

 I can write a quick script to extract all prefixes out of 46 which are at a hop count of less then 10 from me. Based on that number likely I can do some manual checks via NIXI Looking Glass.

 

Bash scripting time!

 

Logic of this simple bash script 

  1.  Read list of IP prefixes of Dyn (which I will simply provide via a text file).
  2. Reverse them and then cut “three characters” and reverse them again. This will turn 103.11.203.0/24 to 103.11.203.0
  3. Run mtr -wrc 1 $IP |wc -l on all these and extract ones which gives number less then 10 or even simply print hop count for all prefixes.

 

 

So here’s a simple script. 🙂

 

#!/bin/bash
cat org-list.txt | while read Prefix
do
IP=$(echo $Prefix |rev |cut -f 2 -d / | rev)

#echo IP from $Prefix is $IP

hopcount=$(mtr -wrc 1 $IP |wc -l)
echo “Hopcount for $Prefix is $hopcount”
done

 

Output is:

Anurags-MacBook-Pro:dyn-routes anurag$ ./hop-count.sh
Hopcount for 103.11.200.0/24 is 11
Hopcount for 103.11.201.0/24 is 10
Hopcount for 103.11.203.0/24 is 9
Hopcount for 198.153.192.0/22 is 11
Hopcount for 198.153.192.0/23 is 11
Hopcount for 198.153.192.0/24 is 11
Hopcount for 198.153.194.0/23 is 16
Hopcount for 198.153.194.0/24 is 16
Hopcount for 203.62.195.0/24 is 11
Hopcount for 204.13.248.0/24 is 17
Hopcount for 204.13.249.0/24 is 18
Hopcount for 204.13.250.0/24 is 17
Hopcount for 204.13.251.0/24 is 15
Hopcount for 208.76.56.0/24 is 17
Hopcount for 208.76.57.0/24 is 9
Hopcount for 208.76.58.0/24 is 13
Hopcount for 208.76.59.0/24 is 12
Hopcount for 208.76.60.0/24 is 15
Hopcount for 208.76.61.0/24 is 12
Hopcount for 208.76.63.0/24 is 13
Hopcount for 208.78.68.0/22 is 13
Hopcount for 208.78.68.0/24 is 13
Hopcount for 208.78.69.0/24 is 13
Hopcount for 208.78.70.0/24 is 9
Hopcount for 208.78.71.0/24 is 10
Hopcount for 216.146.32.0/24 is 28
Hopcount for 216.146.33.0/24 is 30
Hopcount for 216.146.34.0/23 is 11
Hopcount for 216.146.34.0/24 is 11
Hopcount for 216.146.35.0/24 is 11
Hopcount for 216.146.36.0/23 is 16
Hopcount for 216.146.36.0/24 is 16
Hopcount for 216.146.37.0/24 is 9
Hopcount for 216.146.38.0/24 is 16
Hopcount for 216.146.39.0/24 is 12
Hopcount for 216.146.40.0/24 is 17
Hopcount for 216.146.41.0/24 is 13
Hopcount for 216.146.42.0/24 is 11
Hopcount for 216.146.43.0/24 is 13
Hopcount for 216.146.44.0/24 is 16
Hopcount for 216.146.45.0/24 is 21
Hopcount for 216.146.46.0/24 is 12
Hopcount for 216.146.47.0/24 is 17
Hopcount for 80.231.219.0/24 is 10
Hopcount for 80.231.25.0/24 is 10
Anurags-MacBook-Pro:dyn-routes anurag$

 

So we have 103.11.201.0/24, 103.11.203.0/24, 208.76.57.0/24, 208.78.70.0/24, 216.146.37.0/24 and few more. I see from one trace that Dyn is sitting at VSNL-Andheri-Mumbai. I think it makes more sense to extract routes which go to that. It will be better then plain hop count. Modifying script:

 

#!/bin/bash
cat org-list.txt | while read Prefix
#cat dyn-prefix-list.txt | while read Prefix
do
IP=$(echo $Prefix |rev |cut -f 2 -d / | rev)

#echo IP from $Prefix is $IP

route=$(mtr -wrc 1 $IP |grep andheri)
#echo “Prefix $Prefix seems to be having route in $route”
if [[ “$route” =~ ‘andheri-mumbai‘ ]]; then
echo “$Prefix seems to be anycasted at Mumbai”

else echo “$Prefix is likely not anycasted at Mumbai PoP”
fi

done

 

OK – so here’s the output:

Anurags-MacBook-Pro:dyn-routes anurag$ ./hop-count.sh
103.11.200.0/24 is likely not anycasted at Mumbai PoP
103.11.201.0/24 is likely not anycasted at Mumbai PoP
103.11.203.0/24 is likely not anycasted at Mumbai PoP
198.153.192.0/22 is likely not anycasted at Mumbai PoP
198.153.192.0/23 is likely not anycasted at Mumbai PoP
198.153.192.0/24 is likely not anycasted at Mumbai PoP
198.153.194.0/23 is likely not anycasted at Mumbai PoP
198.153.194.0/24 is likely not anycasted at Mumbai PoP
203.62.195.0/24 is likely not anycasted at Mumbai PoP
204.13.248.0/24 is likely not anycasted at Mumbai PoP
204.13.249.0/24 is likely not anycasted at Mumbai PoP
204.13.250.0/24 is likely not anycasted at Mumbai PoP
204.13.251.0/24 is likely not anycasted at Mumbai PoP
208.76.56.0/24 is likely not anycasted at Mumbai PoP
208.76.57.0/24 is likely not anycasted at Mumbai PoP
208.76.58.0/24 is likely not anycasted at Mumbai PoP
208.76.59.0/24 is likely not anycasted at Mumbai PoP
208.76.60.0/24 is likely not anycasted at Mumbai PoP
208.76.61.0/24 is likely not anycasted at Mumbai PoP
208.76.63.0/24 is likely not anycasted at Mumbai PoP
208.78.68.0/22 is likely not anycasted at Mumbai PoP
208.78.68.0/24 is likely not anycasted at Mumbai PoP
208.78.69.0/24 is likely not anycasted at Mumbai PoP
208.78.70.0/24 seems to be anycasted at Mumbai
208.78.71.0/24 seems to be anycasted at Mumbai
216.146.32.0/24 is likely not anycasted at Mumbai PoP
216.146.33.0/24 is likely not anycasted at Mumbai PoP
216.146.34.0/23 is likely not anycasted at Mumbai PoP
216.146.34.0/24 is likely not anycasted at Mumbai PoP
216.146.35.0/24 is likely not anycasted at Mumbai PoP
216.146.36.0/23 is likely not anycasted at Mumbai PoP
216.146.36.0/24 is likely not anycasted at Mumbai PoP
216.146.37.0/24 is likely not anycasted at Mumbai PoP
216.146.38.0/24 is likely not anycasted at Mumbai PoP
216.146.39.0/24 is likely not anycasted at Mumbai PoP
216.146.40.0/24 is likely not anycasted at Mumbai PoP
216.146.41.0/24 is likely not anycasted at Mumbai PoP
216.146.42.0/24 is likely not anycasted at Mumbai PoP
216.146.43.0/24 is likely not anycasted at Mumbai PoP
216.146.44.0/24 is likely not anycasted at Mumbai PoP
216.146.45.0/24 is likely not anycasted at Mumbai PoP
216.146.46.0/24 is likely not anycasted at Mumbai PoP
216.146.47.0/24 is likely not anycasted at Mumbai PoP
80.231.219.0/24 is likely not anycasted at Mumbai PoP
80.231.25.0/24 is likely not anycasted at Mumbai PoP
Anurags-MacBook-Pro:dyn-routes anurag$

 

So here we go!

I found two prefixes 208.78.70.0/24 and 208.78.71.0/24 which seem to be anycasted via Indian Dyn PoP in Mumbai along with many other PoPs. One strong assumption here is that my ISP i.e BSNL is getting routes to all possible Indian PoPs. If BSNL is missing route to any of prefixes then I am out of luck. I see both of these prefixes are available at NIXI via Tata-VSNL AS4755 in Mumbai.

 

I wish anyone from any network who is peering at NIXI is reading this post could share output of “sh ip bgp nei <NIXI-route-server-Mum-IP> received-routes regexp 33517$

Only that can technically  confirm my guess work. 🙂

 

Note to self: My programming skills suck big time. Need to improve!

20 Aug

F root server transit in Chennai

Few days back I noticed F root server (which is with ISC) brought it’s anycasted node in NIXI Chennai back live. They have taken that down as per my interaction with them over mailing list. My last post about F root coming back live was with guess work on who’s providing upstream.
 
Today I spent sometime in finding who’s providing transit to that node. It is very important to note that most of these key infrastructure related nodes rely on peering for most of traffic but a transit in form of full table or default stays so that one can push packets to a route if it is not in table learnt from peering. In case of Indian deployment which was at NIXI Chennai – many ISPs were following “regional routes” clause of NIXI and were announcing just their regional routes (to ISC’s F root router) but quite a few of them (like BSNL) were still learning routes from one region and exporting them into their other region via their IGP. This brought case where my router (sitting on BSNL link) was getting a forward path to NIXI Chennai for F root but there was return path from F root to my system because BSNL wasn’t announcing Northern prefixes in Chennai based NIXI.
As I noted earlier F root is back live in India and I am getting consistant and direct routes. It seems very much the case of addition of transit on that node. Today I was looking at global table dump and I came across some interesting routes which revealed who is probably the transit for ISC’s F root in India. 🙂
 
 
F root server uses IP 192.5.5.241 which comes under BGP announcement of 192.5.5.0/24 as well as 192.5.4.0/23 from quite a few autonomous system numbers of ISC. In India ISC is announcing 192.5.5.0/24 from AS3557. It seems like ISC does not uses AS3557 for direct peering with external networks but instead it peers with other ASNs of ISC. In ISC’s worldwide deployment of F root it seems like there are as many as 15+ ASNs with direct peering/upsteram for AS3557. In case of India AS24049 is used by ISC for inter-connection with external networks.
 
Here’s a routing table entry from NIXI Chennai:
 

show ip bgp 192.5.5.241
Number of BGP Routes matching display condition : 1
Status codes: s suppressed, d damped, h history, * valid, > best, i internal
Origin codes: i – IGP, e – EGP, ? – incomplete
Network Next Hop Metric LocPrf Weight Path
*> 192.5.5.0/24 218.100.48.142 10 100 0 24049 3557 i

 
Looking into global IPv4 table from Orgeon route-views for AS24049 routes, we get:

route-views>sh ip bgp regexp 24049$
BGP table version is 911752723, local router ID is 128.223.51.103
Status codes: s suppressed, d damped, h history, * valid, > best, i – internal,
r RIB-failure, S Stale
Origin codes: i – IGP, e – EGP, ? – incomplete
Network Next Hop Metric LocPrf Weight Path
* 203.119.18.0 12.0.1.63 0 7018 6453 4755 37986 24049 i
* 193.0.0.56 0 3333 3356 6453 4755 37986 24049 i
* 208.74.64.40 0 19214 174 6453 4755 37986 24049 i
* 154.11.11.113 0 0 852 3257 6453 4755 37986 24049 i
* 128.223.253.10 0 3582 3701 3356 6453 4755 37986 24049 i
* 157.130.10.233 0 701 6453 4755 37986 24049 i
* 164.128.32.11 0 3303 6453 4755 37986 24049 i
*> 66.110.0.86 0 6453 4755 37986 24049 i
* 208.51.134.254 2523 0 3549 6453 4755 37986 24049 i
* 203.181.248.168 0 7660 2516 6453 4755 37986 24049 i
* 144.228.241.130 0 1239 6453 4755 37986 24049 i
* 114.31.199.1 0 0 4826 6939 1299 6453 4755 37986 24049 i
* 206.24.210.102 0 3561 6453 4755 37986 24049 i
* 194.85.102.33 0 3277 3267 174 6453 4755 37986 24049 i
* 217.75.96.60 0 0 16150 1299 6453 4755 37986 24049 i
* 202.232.0.2 0 2497 6453 4755 37986 24049 i
* 207.172.6.20 0 0 6079 3356 6453 4755 37986 24049 i
* 207.172.6.1 0 0 6079 3356 6453 4755 37986 24049 i
* 4.69.184.193 0 0 3356 6453 4755 37986 24049 i
* 66.59.190.221 0 6539 6453 4755 37986 24049 i
* 194.85.40.15 0 3267 9002 6453 4755 37986 24049 i
* 154.11.98.225 0 0 852 3257 6453 4755 37986 24049 i
* 69.31.111.244 3 0 4436 3257 6453 4755 37986 24049 i
* 202.249.2.86 0 7500 2516 6453 4755 37986 24049 i
* 89.149.178.10 10 0 3257 6453 4755 37986 24049 i
* 129.250.0.11 6 0 2914 6453 6453 4755 37986 24049 i
* 216.218.252.164 0 6939 1299 6453 4755 37986 24049 i
* 203.62.252.186 0 1221 4637 6453 4755 37986 24049 i
* 66.185.128.48 7 0 1668 6453 4755 37986 24049 i
* 209.124.176.223 0 101 101 3356 6453 4755 37986 24049 i
* 134.222.87.1 0 286 6453 4755 37986 24049 i
* 207.46.32.34 0 8075 6453 4755 37986 24049 i
route-views>
 

And here we go!
 
AS37986 i.e Tulip Telecom seems to be transit. I think this is not case of any route leak – it’s just that Tulip telecom is providing transit.
 

route-views>sh ip bgp regexp 24049 3557
route-views>

 
 
Clearly there’s no announcement of F root prefix directly to Tulip which seems good to avoid any external (outside India) traffic hitting Chennai node. I am quite sure that default on router of AS24049 (or likely AS3557) does points to Tulip’s gateway.
Here we can see the sites under their management subnet – http://route.robtex.com/203.119.18.0-24—internet-systems-consortium.html#sites
 
Well, thanks to Tulip telecom for helping to bring F root node back live. 🙂
 
 
Disclaimer: This is my personal blog and post is completely a reflection of personal thoughts. It has no relation with my employer.