It has been super boring evening considering my sessional tests tomorrow. Test time is dull as always. I have been precisely measnuring latency on BSNL link from BSNL Haryana to Singapore based servers. The fluctuation in latency is pretty much common now. Someones we get 120ms latency to Singapore (an expected number based on distance) while other time it goes off as high as 310ms. Latency with openDNS nodes in Singapore makes it pretty much poor to use openDNS here.   Based on my collected data and BGPlay’s routing records, here’s what’s happening. My IP is coming /20 BGP annoucement from BSNL Autonomous System 9829 - 117.207.48.0/20. Looking at BGP table records for that block from BGPlay’s routing data archive source.

Subnetting IPv6 sounds very complex but to be true - it is very easy! All you need to do is to understand basics of IPv6 addressesing - how an address is formed and how to efficiently use CIDR notation.   Firstly how an IPv6 address looks like? (good to clear fundamentals first!) An IPv6 address has 8 sections seprated by coloums and each sections has carries 4 hexadecimal digits. So an IPv6 address is something like: xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx - Each x can have a hexa decimal value i.e from 0 to 9 and a to f. Thus 16 possible values for each x. Since each each x is stored in binary i.e 0 or 1 (that is 2 possible value) - number of bits per section turns out to be 2x2x2x2 = 16bits. Thus we have now each section with 16 bits per section and 8 sections in total. This turns out to be 16 + 16 + 16 + 16 + 16 + 16 + 16 + 16 bits = 128bit. This is why an IPv6 address has 128bits. This means total possible addresses in IPv6 space is 2^128 = 340 282 366 920 938 463 463 374 607 431 768 211 456 addresses. Next, an important point to remember here is  - in IPv6 address clients are mostly based on /64 subnet which means first 64 bits go to network part while next 64 bits go to the host part i.e usage IPv6 addresses which are allocated to end machines.  

In last few days, I have been pushing discussion on APNIC & NANOG mailing lists about poor DNS infrastructure in India.

Thought to put a quick blog post on the issue.

So what’s exactly wrong?

To understand what’s wrong, let’s understand how DNS works at core level. DNS relies on a hierarchy model with . (dot) on top which is Root and TLD i.e Top Level Domains below Root, which further  follow 2nd level domains which are popularly domain names we use. So e.g mail.google.com is actually like

I am blogging after quite a few days. Spent bit of time at events like APRICOT 2012 - New Delhi, followed by (unexpected) college start (on time) with heavy fine!

Overall time is moving fast. Last week TeamARIN posted a guest blog post from me on DNS in case of IPv6. Apart from that I was excited with small article from RIPE NCC about K-root server connectivity issue as reported by me on mailing list.

 

Few days back I mentioned how reverse DNS setup of Airtel was incorrect. Sad to say it has not been fixed yet. In meanwhile I was looking at domain name - airtel.in the main domain which runs website for Bharti Airtel’s Indian operations. I am little surprised to find that DNS server of airtel.in are failing randomly!  

Problem:

airtel.in uses 4 DNS servers from Mantra Online - a small ISP which Bharti took over years back. Here are the DNS servers used by domain name: aaadel.mantraonline.com. dnsbom.mantraonline.com. dnsdel.mantraonline.com. dnsblr.mantraonline.com.   Now interesting part here is that out of these 4, only 1 behaves normally. DNS server - dnsblr.mantraonline.com. seems working fine but rest all are rejecting queries “randomly” which is interesting. I have mostly seen DNS servers being up or down. This is probably first case when I can see DNS servers failing in random fashion.

Yesterday I came across a very interesting case of network hijacking of an ISP from wrong BGP announcements by another network. This issue was reported to NANOG mailing list. 

Issue was reported by Kevin, Senior Engineer at Altus Communications (AS11325). Problem was that SBJ Media LLC (AS33611) was making a /24 block announcement for specific slices of Altus -  208.110.48.0/20, 63.246.112.0/20, and 68.66.112.0/20 which are allocated to Altus Communications (as per ARIN whois).

Sify is one of really interesting companies. One time pioneer of Indian internet market via chain of cyber cafes. Good old days. Present situation of Sify in consumer market is not significant. Latest earing figures clearly state company is moving towards enterprise segment.

Company is quite aggressive in enterprise segment offerings specially datacenters & corporate leased lines.
Is consumer market really over for Sify or there’s still some hope?

Well, consumer broadband market isn’t really over! Infact this is the main market which is yet to explode in India!

Interestingly routing issues didn’t spare one of top CDN provider - Akamai!

So what’s wrong?

(from my BSNL connection):

PING akamai.com (61.213.189.49) 56(84) bytes of data.  
64 bytes from 61.213.189.49: icmp_req=1 ttl=52 time=492 ms  
64 bytes from 61.213.189.49: icmp_req=2 ttl=52 time=492 ms  
64 bytes from 61.213.189.49: icmp_req=3 ttl=52 time=474 ms  
64 bytes from 61.213.189.49: icmp_req=4 ttl=51 time=492 ms  
64 bytes from 61.213.189.49: icmp_req=5 ttl=51 time=489 ms

\--- akamai.com ping statistics --- 
5 packets transmitted, 5 received, 0% packet loss, time 22236ms  
rtt min/avg/max/mdev = 474.296/488.469/492.837/7.183 ms

~ 500ms is way too high. Even US is at like 300ms latency.

Looking at traceroute: 

traceroute to akamai.com (61.213.189.49), 30 hops max, 60 byte packets  
1 router.local (192.168.1.1) [AS8151/AS28513] 4.223 ms 4.979 ms 5.879 ms  
2 117.200.48.1 (117.200.48.1) [AS9829] 45.241 ms 46.384 ms 52.839 ms  
3 218.248.173.46 (218.248.173.46) [AS9829] 87.089 ms \* \*  
4 115.114.57.165.static-Mumbai.vsnl.net.in (115.114.57.165) [AS4755] 74.675 ms 76.970 ms 80.856 ms  
5 if-0-100.tcore2.MLV-Mumbai.as6453.net (180.87.39.25) [\*] 83.234 ms 84.403 ms 87.742 ms  
6 if-6-2.tcore1.L78-London.as6453.net (80.231.130.5) [AS6453] 230.777 ms 185.553 ms 194.288 ms  
7 \* Vlan704.icore1.LDN-London.as6453.net (80.231.130.10) [AS6453] 203.104 ms \*  
8 Vlan522.icore1.LDN-London.as6453.net (195.219.83.22) [AS6453] 308.973 ms 310.324 ms 311.038 ms  
9 ae-4.r23.londen03.uk.bb.gin.ntt.net (129.250.5.40) [AS2914] 311.799 ms 333.841 ms 313.348 ms  
10 as-0.r22.osakjp01.jp.bb.gin.ntt.net (129.250.5.35) [AS2914] 499.075 ms 501.158 ms 512.657 ms  
11 ae-5.r24.tokyjp01.jp.bb.gin.ntt.net (129.250.3.221) [AS2914] 484.258 ms 485.401 ms 499.039 ms  
12 \* \* \*  
13 xe-2-3.a17.tokyjp01.jp.ra.gin.ntt.net (61.213.169.214) [AS2914] 488.807 ms 489.543 ms 495.396 ms  
14 61.213.189.49 (61.213.189.49) [AS2914] 506.170 ms 501.504 ms 507.296 ms

So route is like Mumbai (India) - London (UK) - Tokyo (Japan).

This is a very popular mistake admins make - it’s missing . i.e dot in the end of hostname. This causes serious problems (and lot of frustration!).

E.g taking example of popular Google’s cname record ghs.google.com. As we know if one would like to use mail.domain.com., he has to point the CNAME record to “ghs.google.com”. Now here if one misses dot in the end of ghs.google.com. - it will give a real value like:

Seems like k-root servers are having issue again. This is not the first time BSNL is having such issues. Last year I reported issue with K root server (which was actually because of downtime at Delhi node).  

Here’s some data for today’s case:

PING 193.0.14.129 (193.0.14.129) 56(84) bytes of data. 
64 bytes from 193.0.14.129: 
icmp_req=1 ttl=44 time=309 ms 
64 bytes from 193.0.14.129: icmp_req=2 ttl=44 time=312 ms 
64 bytes from 193.0.14.129: icmp_req=3 ttl=44 time=312 ms 
64 bytes from 193.0.14.129: icmp_req=4 ttl=44 time=312 ms 
64 bytes from 193.0.14.129: icmp_req=5 ttl=44 time=313 ms 
--- 193.0.14.129 ping statistics --- 
5 packets transmitted, 5 received, 0% packet loss, time 4001ms 
rtt min/avg/max/mdev = 309.687/312.019/313.333/1.289 ms