Isp-Column

An excellent presentation by James Quinn from Facebook on “Being Open How Facebook Got Its Edge” at NANOG68. YouTube link here and video is embedded in the post below.

Some key points mentioned by James:

1. BGP routing is inefficient as scale grows especially around distributing traffic. They can get a lot of traffic concentrated to a specific PoP apart from the fact that BGP best AS_PATH can simply be an inefficient low AS_PATH based path.
2. Facebook comes with a cool idea of “evolving beyond BGP with BGP” where they use BGP concepts to beat some of the BGP-related problems.
3. He also points to fact that IPv6 has much larger address space and huge summarization can result in egress problems for them. A single route announcement can just have almost entire network behind it!
4. Traffic management is based on local and a global controller. Local controller picks efficient routes, injects them via BGP and takes care of traffic balancing within a given PoP/city, balancing traffic across local circuits. On the other hand, Global PoP is based on DNS logic and helps in moving traffic across cities.

It’s wonderful to see that Facebook is solving the performance and load related challenges using fundamental blocks like BGP (local controller) and DNS (global controller). :)

Bangladesh’s .bd ccTLD faced another outage. As I mentioned in one of the previous posts - .bd domain seems to be primarily on BTCL (AS17494).  Zone delegation of .bd is still pending with PCH and while PCH is mentioned in NS records of the authoritative DNS servers but delegation is pending in the root DNS servers as per reply from Kabindra from PCH on the bdNOG mailing list during the last outage.

So it seems like apart from voice interconnect issues, Jio is also facing routing issues on the backbone. I ran a trace to one of IP’s on Jio network allocated to end customer - 169.149.212.122. I ran trace from all Indian RIPE Atlas probes measurement here. There seem quite a few RIPE Atlas probes which are giving latency on 150ms + range. Seems like they are downstream or downstream of downstream of Tata Comm’s AS4755 and routing is happening via Singapore!    

Day before yesterday i.e on 18th August 2016 Bangladesh’s TLD .bd went had an outage. It was originally reported by Jasim Alam on bdNOG mailing list.

dig btcl.com.bd @8.8.8.8
; <<>> DiG 9.10.4-P2 <<>> btcl.com.bd @8.8.8.8
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 8114
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;btcl.com.bd.                   IN      A
;; Query time: 76 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Thu Aug 18 14:24:25 Bangladesh Standard Time 2016
;; MSG SIZE  rcvd: 40

His message shows that DNS resolution of BTCL (Bangladesh Telecommunications Company Ltd) was failing. Later Alok Das that it was the power problem resulting in outage. Let’s look ask one of 13 root DNS server about NS records on who has the delegation for .bd.

Just noticed that Reliance Jio (AS55836) seems to be originating a /16 which is for Charter Communications (AS20115) - 47.35.0.0/16.  

route-views>sh ip bgp 47.35.0.0/16 long | exclude 20115
BGP table version is 18764390, local router ID is 128.223.51.103
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,
              x best-external, a additional-path, c RIB-compressed,
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
     Network          Next Hop            Metric LocPrf Weight Path
 *   47.35.0.0/16     195.208.112.161                        0 3277 3267 174 64049 55836 i
 *                    217.192.89.50                          0 3303 6762 64049 55836 i
 *                    212.66.96.126                          0 20912 1267 64049 55836 i
 *                    162.243.188.2                          0 393406 6453 6762 64049 55836 i
 *                    192.241.164.4                          0 62567 2914 174 64049 55836 i
 *                    129.250.0.11          1007             0 2914 174 64049 55836 i
 *                    104.192.216.1                          0 46450 174 64049 55836 i
 *                    202.93.8.242                           0 24441 3491 3491 174 64049 55836 i
 *                    66.59.190.221                          0 6539 577 6762 64049 55836 i
 *                    144.228.241.130         80             0 1239 174 64049 55836 i
 *                    207.172.6.20             0             0 6079 3356 174 64049 55836 i
 *                    203.62.252.83                          0 1221 4637 174 64049 55836 i
 *                    93.104.209.174                         0 58901 51167 3356 6762 64049 55836 i
     Network          Next Hop            Metric LocPrf Weight Path
 *                    162.250.137.254                        0 4901 174 64049 55836 i
 *                    4.69.184.193             0             0 3356 174 64049 55836 i
 *                    208.51.134.254           1             0 3549 3356 174 64049 55836 i
 *                    89.149.178.10           10             0 3257 174 64049 55836 i
 *                    66.110.0.86                            0 6453 6762 64049 55836 i
 *                    134.222.87.1           650             0 286 174 64049 55836 i
 *                    95.85.0.2                              0 200130 6453 174 64049 55836 i
 *                    12.0.1.63                              0 7018 174 64049 55836 i
 *                    173.205.57.234                         0 53364 3257 174 64049 55836 i
 *                    206.24.210.80                          0 3561 174 64049 55836 i
 *                    5.101.110.2                            0 202018 2914 174 64049 55836 i
 *                    207.172.6.1              0             0 6079 3356 174 64049 55836 i
 *                    154.11.98.225            0             0 852 174 64049 55836 i
 *                    194.85.40.15                           0 3267 174 64049 55836 i
 *                    208.74.64.40                           0 19214 174 64049 55836 i
 *                    209.124.176.223                        0 101 101 174 64049 55836 i
 *                    66.185.128.48            6             0 1668 174 64049 55836 i
 *                    203.181.248.168                        0 7660 2516 6762 64049 55836 i
 *                    202.232.0.2                            0 2497 701 6762 64049 55836 i
 *                    103.247.3.45                           0 58511 64049 55836 i
 *                    193.0.0.56                             0 3333 1103 64049 55836 i
     Network          Next Hop            Metric LocPrf Weight Path
 *                    80.241.176.31                          0 20771 47872 64049 55836 i
 *>                   216.218.252.164                        0 6939 64049 55836 i
 *                    132.198.255.253                        0 1351 174 64049 55836 i
 *                    103.255.249.22                         0 58443 45177 64049 55836 i
 *                    114.31.199.1                           0 4826 174 64049 55836 i
route-views>

This shows Reliance Jio’s ASN AS55836 announcing 47.35.0.0/16. Charter Communications (AS20115) is originating multiple of /18s out of the same pool.  

It’s great to see Amazon announcement two days back about launch of their region in Mumbai. In past I was quite happy to see their Cloudfront CDN PoPs in Mumbai & Chennai (blog post here). Now it’s just great to see a full AWS region out of Mumbai. :) Though it’s going to eat most of important customers from the smaller players still it’s good for industry as industry is too big and we need more & more of such large Cloud players in India to bring more and more content hosting in India.

I just presented data on Performance of Root DNS anycast in UK & Ireland at INEX meeting.

Last month India & Bangladesh went into an agreement for power and bandwidth. India stated export of an additional 100MW of power to Bangladesh while Bangladesh started a 10Gbps link to Indian state of Tripura. (News article on this here)

Tripura is an Indian state having its boundaries with Bangladesh as you can see in above map. Coming to routing side of things setup is that BSNL (AS9829) is buying IP transit from Bangladesh Submarine Cable Co. Ltd (BSCCL) at $1.2 million / year. This means a cost of around $10/Mbps/month or 662Rs/Mbps/month. It’s hard to say if it’s good or bad since other link from BSNL is via it’s other links. But yes it’s good to see a layer 3 connectivity in terms of IP transit relationship rather then leasing dark fiber or L1 waves as they would have caused bit inefficient routing in the area. In order to do this BSNL has setup a “gateway node” at Agartala. I think it would be pretty much a node with approvals under ILD from doT and extremely likely a LIM device for lawful interception.   Months before it actually came up, Dyn research tweeted about this visible routing relationship.  

First and foremost before talking about APRICOT, I must say I am deeply moved with impact Rohtak (and Haryana) as whole had because of recent Jat agitation. What I find extremely depressing is way current Govt. of Haryana completely failed to control it and the way previous Govt. ministers did best in their interest and completely against the interest of people of Haryana. For now quite hopeful with news that Mr Prakash Singh (one of my favorite IPS officers) who did quite well during his various terms is looking into failure of police. More details about the news here. I will write more on this later on, not good time right now since tensions have yet to get normal.  

An interesting promo video on Internet peering and why it is important in context of Africa by Internet Society.