Posts

A while back I posted about routing filter generation via bgpq3 for Cisco (ios and XR) and Juniper JunOS based routers. I have received a number of emails in last few months about automated filter generation for Mikrotik routeros. Since Mikrotik’s CCRs are getting quite popular across small to mid-sized ISPs. So this blog post is about ways for generating filter config for a given ASN via IRR. One can use such logic with some kind of remote login mechanism like rancid (look for mtlogin here). I tried building around bgpq3 but it seems more easy with another popular tool in the domain called IRR Power Tools. Once IRR Power Tools (IRRPT) is setup, it allows us to fetch prefixes based via Internet Routing Registries and also aggregates them.   So, for instance, let’s pick AS54456:

India has a slum problem as many of us know. Slums are a serious problem and there’s just no easy way to fix them. One cannot just push thousands and thousands of people out while at the same time quality of life in slums is terrible. One thing which happens a lot in India is the fact that Govt. does nothing when slums are getting established and once they are established situation gets out of control.    

Earlier this month I got an opportunity to be part of IXP workshop in Kolkata. It was a 3-day event organised by ISOC Kolkata and supported by APNIC. There was also a workshop on DNSSEC and Champika Wijayatunga (from ICANN) was the instructor along with Anand Raje. It was a nice event and I come to know of other interesting projects ISOC Kolkata is doing like Indian IETF capacity building program apart from the IXP they are running in Kolkata. Mr Anupam Aggarwal and Anand showed the IX and it looks very good. I think it’s the first and only IX I know in India which is a real IX with proper policy. It’s an IX by a non-for-profit group, allows anyone to connect, a real layer 2 IX and welcomes anyone including ISPs, content players and root DNS servers. Presently IIFON-IX in Kolkata has few member ISPs besides the L root from ICANN and one of Verisign gTLD nodes (which host zones for .com, .net etc). I also saw a rack with some of Akamai CDN servers. This brings decent content right there. IX’es play an extremely important part of current internet infrastructure ecosystem. It’s very likely that content of this blog is travelling from my server to your browser from an Internet Exchange. :)  

Today BGPmon reported about possible BGP prefix hijack of Amazon’s IP address space. Amazon announces 50.16.0.0/16 from AS14618.

At 13:45:44 UTC / 19:15:44 IST D-Vois broadband started originating a more specific 50.16.226.0/24 in the table from AS45769. One of example AS_PATH of this announcement: 198290 197264 197264 197264 29467 1299 9583 45769 Clearly, this leak was carried over by AS9583 (Sify) to AS1299 (Telia) and was carried over to rest of internet from there. There was a visible withdrawal of this request by 14:17:37 UTC / 19:47:37 IST.  

Last week I was on way from Kolkata to Delhi after spending time at ISOC IXP workshop. It was Jet Airways 9W 946 on 10th March 2017. The flight took off slightly late and 90% time went just fine. Original arrival time was 16:30 though it was expected to be little late.

Landing time…

At around 16:15 pilots announced “to prepare for landing” and cabin crew started collecting waste and getting people ready. I was sitting on the window seat and looking at clouds on my right. It was a usual view from the window and I usually like when the plane crosses over the clouds. It was little bumpy by 16:20 and that is usual behaviour. As it proceeded it went from “slightly bumpy” to “very bumpy” ride. Somewhere around 16:25 or so plane gave the first feeling of “free fall”. It’s was the same feeling as we get going down the roller coaster but 10x of that. Few people screamed. It reminded me of one of my previous travels from Malaysia to India in 2013 after APRICOT 2013 when the flight was very bumpy and also faced free fall at that time. Next after 2-3 mins plane had another free fall and it went for 10-15 seconds. A Japanese gentleman sitting next to me wasn’t wearing his shoes. One of his shoes literally touched my chest and fell on me. That was the first time I also got quite scared.

And here goes my first post for 2017. The start of this year did not go well as I broke my hand in Jan and that resulted in a lot of time loss. Now I am almost recovered and in much better condition. I just attended HKNOG 4.0 at Hong Kong followed by APRICOT 2017 at Ho Chi Minh, Vietnam. an event and I enjoyed the both. Here’s my presentation from APRICOT 2017. I recently I came across some of crazy confusing traceroutes as passed by one of my friends. I cannot share that exact traceroute on this blog post but can produce the same effect about which I am posting by doing a trace from one of large network like Telia London PoP to one of the Indian destinations via their looking glass

Yesterday Google’s Bangladeshi website google.com.bd was hacked and this happened via DNS. It was reported on the bdNOG mailing list at morning in a thread started by Mr Omar Ali.

This clearly shows how authoritative DNS for “com.bd.” (which is same as bd. btw) was poisoned and was reflecting attackers authoritative DNS. Later Mr Farhad Ahmed posted a screenshot of google.com.bd showing hackers page:

Later Mr Sumon Ahmed mentioned that it happened because web frontend of .bd was compromised. This was an interesting hijack as attacker attacked the key infrastructure of the registry instead of Google or Facebook servers. It’s also a warm reminder of the way DNS depends on the hierarchal structure by design and at this stage, we need to focus on DNSSEC to add on the security to the current system.   Lately .bd domain faced issues multiple time this year. I hope it will have a good stable time in the upcoming year. In terms of stability it is being backed by PCH anycast infrastructure but PCH’s DNS servers are just published in NS records of it’s existing auth servers, but not on the parent zone (which is root zone). Thus the point of failure remains and is yet to be fixed.

Today Google (AS15169) seem to be facing issues in their Indian network due to tropical cyclone Vardah. Their traffic at PoPs in Mumbai dipped considerably for a number of ISPs. My guess is that it’s likely because of outage in a large Govt. operator’s network who has overhead fibres along with utility lines.

It’s very important to note that “Google PoP” faced the issue and it’s no where close to saying that Google services went down. Google has a large network across the globe where they peer with networks. If one segment of this network goes down, traffic is re-routed via other parts and as per design even if the network goes down completely in say Mumbai or Chennai, services should stay live. While in real practice considerable degradation occurs because most of the Indian networks get a very large amount of traffic from Google and usually do not have that much extra capacity on their IP transit links, resulting in choking of transits during issues on their PNI with Google.   This shows how traffic of an ISP connected to Google in Mumbai dipped during peak time around at 4pm on Monday 12th Dec (IST) and went to zero little before midnight. I triggered a trace to aspmx.l.google.com. which is outside India from RIPE atlas probes in India and in general routing to that goes via Google’s backbone. Cluster with hostname aspmx.l.google.com (and few others) carry the Gmail/Google Apps traffic and it’s published by Google Apps users in their domain’s MX records.

One of frequent email and contact form message I get my blog is about available content networks in India and where one can peer. There are certain content networks in India and of course most of the content networks have open peering policy and are usually happy with direct inter-connection (we call as “peering”) with the ISP networks (often referred to as “eyeball networks”). Some of these networks have a backbone which connects back to their key datacenter locations on their own circuits via Singapore/Europe, some other have simply placed their caching server where cache fill happens over IP transit. Based on publically known information across community and of course peeringdb, following content players are available in India and known to be open for peering:

An excellent presentation by James Quinn from Facebook on “Being Open How Facebook Got Its Edge” at NANOG68. YouTube link here and video is embedded in the post below.

Some key points mentioned by James:

1. BGP routing is inefficient as scale grows especially around distributing traffic. They can get a lot of traffic concentrated to a specific PoP apart from the fact that BGP best AS_PATH can simply be an inefficient low AS_PATH based path.
2. Facebook comes with a cool idea of “evolving beyond BGP with BGP” where they use BGP concepts to beat some of the BGP-related problems.
3. He also points to fact that IPv6 has much larger address space and huge summarization can result in egress problems for them. A single route announcement can just have almost entire network behind it!
4. Traffic management is based on local and a global controller. Local controller picks efficient routes, injects them via BGP and takes care of traffic balancing within a given PoP/city, balancing traffic across local circuits. On the other hand, Global PoP is based on DNS logic and helps in moving traffic across cities.

It’s wonderful to see that Facebook is solving the performance and load related challenges using fundamental blocks like BGP (local controller) and DNS (global controller). :)