Tag Archives: Akamai

06 Sep

CDN Caching Panel discussion at APNIC 46

I am in Noumea in New Caledonia in the Pacific Islands. Next week we have APNIC 46 conference and I would be moderating an exciting panel discussion with friends from Akamai, Cloudflare, Facebook and more about working of CDNs. 

If attending APNIC 46, please come & join this session –https://conference.apnic.net/46/program/schedule/#/day/7/panel-cdn-caching

If you are interested in connecting to Hurricane Electric (AS6939) in this region, please do drop me a message. (List of our PoPs in the region here)

01 Mar

Encrypted DNS using DNSCrypt

Writing this post from my hotel room in Kathmandu. I found that many of the servers appear to be DNS resolvers which is unusual.
E.g:

dig @anuragbhatia.com . ns +short
a.root-servers.net.
b.root-servers.net.
c.root-servers.net.
d.root-servers.net.
e.root-servers.net.
f.root-servers.net.
g.root-servers.net.
h.root-servers.net.
i.root-servers.net.
j.root-servers.net.
k.root-servers.net.
l.root-servers.net.
m.root-servers.net.
dig @google.com . ns +short
b.root-servers.net.
c.root-servers.net.
d.root-servers.net.
e.root-servers.net.
f.root-servers.net.
g.root-servers.net.
h.root-servers.net.
i.root-servers.net.
j.root-servers.net.
k.root-servers.net.
l.root-servers.net.
m.root-servers.net.
a.root-servers.net.

 
This seems unusual and is the result of basically port 53 DNS hijack. Let’s try to verify it using popular “whoami.akamai.net” query.

dig @8.8.8.8 whoami.akamai.net a +short
202.79.32.164
dig @9.9.9.9 whoami.akamai.net a +short
202.79.32.164
dig @1.2.3.4 whoami.akamai.net a +short
202.79.32.164

So clearly something in middle is hijacking DNS queries and no matter whichever DNS resolver I try to use, the queries actually hit authoritative DNS via 202.79.32.164. This belongs to WorldLink Communications (ISP here in Nepal) and I am just 5 hops away from it.
 
So what can be done about these cases? Well, one way is VPN of course but with a setup where VPN server’s IP address is hardcoded in the client and not using DNS. It works and does the task but performance can vary greatly depending on how far is the tunnel server. A better and more modern way out of it is by using encryption in DNS by using a protocol named “DNSCrypt“. DNSCrypt offers to encrypt of DNS queries from clients to the DNS resolvers. (Beyond that resolver still, follow usual non-encrypted root chain to reach authoritative DNS servers).
 
So how does it work?
There’s no integrated support of DNSCrypt in OS’es at this time. There are number of projects like dnscrypt-osxclient available on GitHub which enable this support.  Once configured, the client changes system’s DNS resolver to a local IP which listens for port 53 (regular/non-encrypted) requests.

cat /etc/resolv.conf |grep nameserver
nameserver 127.0.0.54

The client often offers support of various open resolvers like OpenDNS, Quad9 etc.

dig @127.0.0.54 whoami.akamai.net a +short
67.215.80.66

 
 
Here it shows that DNS resolver in my case happens to be Cisco’s OpenDNS. As soon as the client gets port 53 DNS queries, it encrypts it and sends via UDP port 443 (UDP or TCP depending on provider and client configuration). The encyption is based on trusted root CA’s and associated chain as popularly used in HTTPS. This is also one of reasons why DNSCrypt is also known as DNS over HTTPS.
 
Here’s an example of a DNS query to resolve A record of google.com while running tcpdumps in parallel:

sudo tcpdump -i lo0 'dst port 53' -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on lo0, link-type NULL (BSD loopback), capture size 262144 bytes
04:36:04.429212 IP 127.0.0.54.50966 > 127.0.0.54.53: 31576+ A? prd.col.aria.browser.skypedata.akadns.net. (59)
04:36:04.532015 IP 127.0.0.54.54914 > 127.0.0.54.53: 623+ [1au] A? google.com. (39)
^C
2 packets captured
4 packets received by filter
0 packets dropped by kernel

This shows request went in clear text to 127.0.0.54 which is configured on loopback. While in parallel if I watch for traffic towards OpenDNS public IPs, I get:

sudo tcpdump -i en0 'dst 208.67.220.220 or dst 208.67.222.222' -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on en0, link-type EN10MB (Ethernet), capture size 262144 bytes
04:39:56.827824 IP 192.168.0.4.53763 > 208.67.220.220.443: UDP, length 512
^C
1 packet captured
63 packets received by filter
0 packets dropped by kernel

Thus all that appears here is just an encrypted packet to Cisco OpenDNS over UDP port 443.
I ran another query and saved it in pcap file. Here’s how it looks like in wireshark:

 
 
 
That’s all about it for now. I am going to keep encryption enabled especially when travelling from now onwards. Time to get some sleep. 🙂
 
Useful Links:

  1. dnscrypt-osxclient – https://github.com/alterstep/dnscrypt-osxclient
  2. DNSCrypt Wikipedia – https://en.wikipedia.org/wiki/DNSCrypt
  3. DNS Over HTTPS (Google Public DNS) – https://developers.google.com/speed/public-dns/docs/dns-over-https
  4. DNS over TLS (Quad9) – https://quad9.net/faq/#Does_Quad9_support_DNS_over_TLS