While I am spending time on APNIC’s security workshop here at APNIC 46, I got curious about DNSSEC deployment across ccTLDs.
For those who may be unaware, DNSSEC adds signature the DNS responses making it possible to cryptographically verify a DNS query response.
Out of 254 ccTLDs, 125 support DNSSEC with a published DS record (at least that is what I get when I check their zone) and 129 do not support it as yet. So, for now, it is at 49.21%.
Earlier this year after APRICOT 2018, I posted a list of visible Facebook FNA (CDN caching) nodes across the world with IPv4, IPv6 and the AS name. I got quite a few mails in following months about people mentioning that they installed nodes but do not see their names in the list (and that was normal since list was static).
I re-ran my script to see emailslatest status of nodes. During last check I saw 1689 nodes (3rd March). Now on 26th Aug i.e after close to 6 months, the total number of nodes has increased to 2204.
Suddenly the voice market in India is becoming very interesting. Earlier it was the case of Jio (and competitors) launching unlimited voice plans and now it’s the case of Govt. of India permitting IP telephony. IP Telephony i.e networks where telephony happens over IP (not to be confused with IP to IP calls but) where IP to PSTN interconnects happen. Till a few months ago IP telephony (or IP-PSTN) interconnection was allowed only under certain conditions like doing it inside a building only for purpose of call centres (with OSP license) or running SIP trunks over private networks. Things like termination of calls originated from the apps was not allowed (where IP-PSTN was happening within India) as well as DID or Direct Inward Dialing numbers were not allowed. There were even cases where apps/businesses had to shut down due to confusing regulation. Here’s a nice article from Medianama about it. But all those were things of past. In May Wifi calling or calls via Wifi where wifi is used loosely and it’s essentially called via any sort of Internet connections were permitted (news here). Later after TRAI’s clarification it now has been formally allowed. While it may not look as attractive as it should have been in the age of WhatsApp calling (IP to IP, not PSTN mess involved!), it still is quite interesting and going to bring some major change.
Railtel (the telecom arm of Indian railways) is running free wifi hotspots across the country in collaboration with Google. It’s there since last two years and started with the MoU between Railtel and Google (news here) back in 2015. Fast forward to 2018 - the free wifi project railway stations seems to be doing quite well with so many users using it. The project covers 361 stations and is expected to reach it’s target of 400 stations soon. The IP network for the service is under the name “Mahataa Information India Private Limited” and originates IP pools from AS134426 - https://bgp.he.net/AS134426#_asinfo. It is a single homed network behind Railtel’s AS24186. https://qz.com/715143/googles-free-wifi-at-indian-railway-stations-is-better-than-most-of-the-countrys-paid-services/ I put an RTI to Railtel asking them about MoU details as well as bandwidth consumption for each state. In their reply, Railtel denied the request for MoU under the exemption from disclosure as well as NDAs they have with Google but they did share detailed of state wise bandwidth consumption.
Out of curiosity, I put an RTI asking President of India Secretariat about connectivity at President’s office.
Question 1: Who is the Internet service provider?
Reply: National Informatics Centre MEITY is providing internet service at President Office.
Question 2: Speed of Internet connection at President’s Office?
Reply: Presently the speed is 100 Mbps.
Question 3: (And my favourite!) Is IPv6 deployed?
Reply: IPv6 supported by most devices. However, IPv4 addressing is used at present.
Just back from APRICOT 2018. As I mentioned in my previous blog post, APNIC had its first Hackathon and it was fun (blog post of APNIC here). There was one project on the ranking of CDNs using RIPE Atlas data. To achieve this team was trying to find strings/hostnames which they can trace to and figure out nearby CDN. As part of that, I suggested them to look into www.facebook.com and carefully noting the sources from where elements get loaded. It’s quite common that Facebook.com (or Google.com for the logic) would be hosted on some server at a large PoP while FNA (or GGC) would serve only specific static content out of it. FNA, of course, sits on the IPs of the ISP hosting it. So in the source list, we found scontent.fktm1-1.fna.fbcdn.net and that gives an idea that FNA strings are around logic: scontent.fxxx1-1.fna.fbcdn.net where xxx is the airport code. 1-1 means 1st PoP in 1st ISP over there probably (strong guess!). If there are more FNA nodes in a given area, the number goes further up. The team used it and for now, the project is over. But while I was on the way back to India, I thought that this is very interesting data if we pull the full picture by querying all possible IATA airport codes with a logic. This logic can be used for two things:
Writing this post from my hotel room in Kathmandu. I found that many of the servers appear to be DNS resolvers which is unusual.
Have a look at these weird DNS replies:
dig @anuragbhatia.com . ns +short
a.root-servers.net.
b.root-servers.net.
c.root-servers.net.
d.root-servers.net.
e.root-servers.net.
f.root-servers.net.
g.root-servers.net.
h.root-servers.net.
i.root-servers.net.
j.root-servers.net.
k.root-servers.net.
l.root-servers.net.
m.root-servers.net.
dig @google.com . ns +short
b.root-servers.net.
c.root-servers.net.
d.root-servers.net.
e.root-servers.net.
f.root-servers.net.
g.root-servers.net.
h.root-servers.net.
i.root-servers.net.
j.root-servers.net.
k.root-servers.net.
l.root-servers.net.
m.root-servers.net.
a.root-servers.net.
This seems unusual and is the result of basically port 53 DNS hijack. Let’s try to verify it using popular “whoami.akamai.net” query.
APNIC and RIPE NCC are doing a hackathon at APRICOT 2018. It just started today with some light interaction with various participating members yesterday. The theme of the hackathon is around IPv6. Many cool projects were suggested yesterday and teams started working today on certain shortlisted projects like:
Let’s see how things go in next 12hrs. Super fun. Things should show up on Github in next few hours. :)
And here goes first blog post of 2018. Last few months went busy with some major changes in personal life. :) I looked into Amazon’s India connectivity with various ASNs tonight. Here’s how it looks like. (Note: Jump to bottom most to skip traces and look at the summary data).
traceroute to 118.185.107.1 (118.185.107.1), 30 hops max, 60 byte packets
1 ec2-52-66-0-128.ap-south-1.compute.amazonaws.com (52.66.0.128) 21.861 ms ec2-52-66-0-134.ap-south-1.compute.amazonaws.com (52.66.0.134) 19.244 ms 19.233 ms
2 100.64.2.200 (100.64.2.200) 14.789 ms 100.64.0.200 (100.64.0.200) 20.731 ms 100.64.3.12 (100.64.3.12) 13.187 ms
3 100.64.0.193 (100.64.0.193) 14.418 ms 100.64.3.69 (100.64.3.69) 15.469 ms 100.64.3.67 (100.64.3.67) 15.946 ms
4 100.64.16.67 (100.64.16.67) 0.343 ms 100.64.17.165 (100.64.17.165) 0.312 ms 100.64.17.199 (100.64.17.199) 0.313 ms
5 52.95.67.213 (52.95.67.213) 1.942 ms 52.95.67.209 (52.95.67.209) 1.967 ms 52.95.67.213 (52.95.67.213) 1.935 ms
6 52.95.66.218 (52.95.66.218) 4.998 ms 4.694 ms 52.95.66.130 (52.95.66.130) 4.650 ms
7 52.95.66.67 (52.95.66.67) 1.752 ms 52.95.66.89 (52.95.66.89) 1.850 ms 1.806 ms
**8 52.95.217.183 (52.95.217.183) 3.111 ms 3.102 ms 3.088 ms <- Amazon India**
**9 182.19.106.204 (182.19.106.204) 3.426 ms 4.547 ms 4.537 ms <- Vodafone India**
10 118.185.107.1 (118.185.107.1) 2.035 ms 2.059 ms 2.039 ms
A few weeks back an Indian ISP contacted me via a contact form on my blog. That ISP has been struggling with a targetted DDoS attack. For the reason of privacy as well as the stability of their network, I will not put their name or AS number. The attack on that ISP was much higher than their bandwidth levels. Their upstream did not really share the volume of attack but I could tell from the screenshots they shared was that it was distributed volumetric attack choking their upstream bandwidth. I suggested that ISP get the blackholing option from his upstream (preferred way) or buy a cheap server/VM somewhere outside India with BGP (and BGP blackholing) and manually blackhole traffic when the attack comes. They were able to get blackholing enabled from their upstream and it did work. They started blackholing traffic and it helped to manually drop traffic going towards the IPs which were being attacked. It’s important to have BGP blackholing because it helps a network to signal upstream ISP about the pools which are under attack and to drop traffic towards them. ISPs further signal the same to their upstream and larger networks typically drop that traffic on all their edge routers i.e closer to the entry fo attack. Next, the problem which hit that ISP was that it was a pain to manually find IPs which were under attack and quickly drop them. I suggested them to try fastnetmon. I heard of fastnetmon from the presentation of Job Snijders at NLNOG (slides here & video here). Fastnetmon is developed and maintained by Pavel Odintsov (who works at Cloudflare AS13335).