16 May

Backend of Google's Public DNS

And finally academic session over. Done with all vivas and related stuff.
Next will be exams likely in June. Time for me to get ready for travel. 🙂
 
Anyways an interesting topic for today’s post – Google Public DNS. Lot of us are familier with popular (and free) DNS resolvers 8.8.8.8 and 8.8.4.4. I have covered reason in previous posts on why it tends to fail with Content Delivery networks like Akamai which rely on anycasting at bottom DNS layer and simple unicasting on application servers. Anycasted DNS nodes point to application servers based on various factors like distance, load, cost etc out of interesting algorithms these CDN networks use for load & cost management.
 
Anyways today’s post focus is not CDN issues with these resolvers but Google Public DNS itself. Are these servers located in India and everywhere else where Google has PoPs?
 
Let’s do a simple trace to get forward path from Airtel to Google’s 8.8.8.8:

traceroute 8.8.8.8
Type escape sequence to abort.
Tracing the route to google-public-dns-a.google.com (8.8.8.8)
1 182.79.254.242 [MPLS: Label 550027 Exp 0] 0 msec
203.101.100.29 [MPLS: Label 550027 Exp 0] 4 msec
182.79.254.238 [MPLS: Label 354133 Exp 0] 0 msec
2 203.101.100.181 0 msec
182.79.247.13 0 msec 0 msec
3 74.125.51.34 44 msec 44 msec 48 msec
4 72.14.237.3 [AS 15169] 52 msec 56 msec 52 msec
5 google-public-dns-a.google.com (8.8.8.8) [AS 15169] 52 msec * 116 msec
DEL-ISP-MPL-ACC-RTR-9#

 
50ms latency. Clearly destination is within India and based on my experience with latency values, I strongly guess that’s Chennai.
 

Location of Google Public DNS servers

Anyways so does that means Google’s DNS server is within India?
 
A clear answer is no. This is just a DNS caching server and Google does not use it for originating actual queries further to root, TLDs nodes and authoritative DNS servers. This seems like a interesting distributed setup.
As per Google Public DNS FAQ page, there are quite a few locations from where DNS servers originate queries but India is not in the list yet. Google has PoPs in Delhi, Mumbai and Chennai and they peer with pretty much every Indian ISP out from there.
 
We can actually test which node is serving us here in India.
This can be achieved in multiple ways:

  1. Running a authoritative zone on a server with basic BIND installation. I tried this with my own Linux server by having testing-google-dns.anuragbhatia.com. DNS zone. I delegated NS for this zone on auth. DNS servers for “anuragbhatia.com” zone. Next I sent a DNS query with dig @8.8.8.8  testing-google-dns.anuragbhatia.com. a +short to ask my DNS server for IP and this gave me source IP of Google’s resolver. 
  2. The other easy way out is to simply use Akamai’s “whoami.akamai.net” service. It is designed in a way to return A record of DNS resolver which queries it. This gives IP of Google’s server which sent the DNS query for resolution.

 

Anurags-MacBook-Pro:~ anurag$ dig whoami.akamai.net a @8.8.8.8 +short
173.194.97.22
Anurags-MacBook-Pro:~ anurag$

 
 
In both cases I saw IP was 173.194.97.22. It belongs to 173.194.97.0/24 announced by Google’s AS15169. As per Google’s FAQ page (which has IPs too!) the prefix 173.194.97.0/24 belongs to Kuala Lumpur, Malaysia. So that’s the actual DNS resolver node which serves users here in India. Machines with IPs 8.8.8.8 and 8.8.4.4 are just caching replies and more over taking the IP traffic to Google within India.
 
Now one can ask why Google is not having DNS resolver within India?
 
 

 

Guess work time!

I don’t know exactly but I can do a strong guess work here. Google is a tier 1 transit free network. It relies on paying on layer 2, building PoPs and connected them together. It does not pays on layer 3 for bandwidth to any ISP. So Google’s routers in India learn traffic from just peering sessions with all major telcos (except BSNL). Google is peering with Tata-VSNL AS4755, Reliance AS18101, Airtel AS9498, MTNL AS17813, Spectranet AS10029 etc. One interesting thing here is that these are all tier 2 networks. Tata Communications is a tier 1 network but their domestic backbone VSNL AS4755 is technically not a tier 1 network and technically it sits in downstream of Tata AS6453 (which is their tier 1 IP backbone). Thus Google does not gets full global table feed from any of these links and possibly nearest PoP of Google which has full table feed from Tier 1 networks is in Malaysia.
 
What I am not able to answer from my guess work is that when Google is relying on East Asian PoPs for such stuff and mantaining a backbone between East Asia and India directly then why they could not feed Indian routers routing table with routes learnt from outside?  It could be just to ensure direct delivery in India and avoid routing loops. E.g BSNL has IP port from Tata-VSNL AS4755 within India and IPLC port from Tata-AS6453 to outside Indian PoPs. Thus if tables are combined Google might see paths like  AS6453 > AS9829 and AS4755 > AS9829 which seem identical as per AS path but one is direct India to India traffic while other via India > Singapore > India or India > US > India. It’s not just about BSNL but Sify also lately has weird routing loops going from outside India for Indian destinations.
 
That’s about it. Can’t do any guess work beyond this point unless someone gives me access to a router of AS15169 to see table! 🙂

15 May

Do connected interface ping?

And an interesting day full of bit frustrating drama. Today was “External viva” for Major Project at college. It went good with external teacher but “internal ones” tend to cause un-necessary issues. Quite a few people put personal egos and frustration on top priority to an extent that they violate their own points for which they are arguing. They go completely unethical in way they deal with world.
 

 
I am saying this with full responsibility for couple of teachers from my college who have completely lost some “fundamentals of life” as taught in childhood to most of us. Some key principles like staying cool & calm, being humble, making best possible use of time and just being good with everyone. In last 4 years they haven’t learn how to give respect & talk with sense and they expect students to be learning “technology” from them? What an absurd!
 
Anyways not much I can do about it. My own external viva went fine and that’s good enough to stay happy. 🙂
All this reminds of an old amazing poem by Former Prime Minister of India (find it embedded below):
 


 
 

Today’s post…

Personal frustration aside – An interesting topic for today. One of my friend came up with an interesting question. Are all connected routers supposed to be pinging?
It’s simply about two routers – say router A and router B which whose interfaces are connected by a cable. Should they ping? If yes then under what circumstances?
 
Answer is …..mmm…let’s first dive in a little before I gave plain answer.
 
 

Ping/ICMP

Ping works on ICMP protocol and that comes on layer 3 on OSI model.
ICMP simply sends a packet (in a form of “Hello there!”) to destination host machine and if everything is OK then host machine is expected to be replying back with “Hi!” to the source. Time taken in this is counted in round trips (because a non-round trip based counting is logically not possible). This provides with connectivity confirmation + roundtrip latency in the connection.
 
So what are conditions required for ICMP to work?
 
Going into fundamentals of TCP/IP networking, a packet can route only when it knows answer to three simple questions:

  1. Who I am?
  2. Where I need to go?
  3. How I will get there?

 
 
Answer to “Who I am?” comes from the unique IP address (on broadcast domain), “Where I need to go” comes from the user himself who is pinging destination machine IP and “How I will get there?” comes from routing table.
 
So #1 and #3 are important here.  There’s not really BGP, or any IGP protocol when two routers are connected directing on same broadcast (layer 2) and this is where other low level protocol ARP comes into picture. ARP is used in creating a simple table which keeps a reference between Mac address, IP address and interface.
 

How does ARP work?

ARP simply works using “broadcast address” we use while configuring IP address on a interface. E.g if I am putting IP as 10.0.0.1 coming from /29 subnet (or call it 255.255.255.248), it assumes last IP to be “broadcast IP”. A /29 here means 2^32-29 = 8 IPs. Starting from 10.0.0.1, it goes till 10.0.0.7. So last IP = 10.0.0.7 is broadcast IP. This is used by all machines under same broadcast domain to “announce/advertise” their IP address from their Mac address.
 
Let’s play around and connect three routers to a switch (same layer 2 broadcast, single VLAN).
We have router A, router B and router C.
I am putting A and B on 10.0.0.1 and 10.0.0.2 under same subnet (a /29) while router C will be on 10.0.0.100 coming from /24 subnet (for fun!).
 

Screen Shot 2013-05-15 at 6.51.34 PM

 
 

Router>
Router>en
Router>enable
Router#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)#host
Router(config)#hostname A
A(config)#in
A(config)#interface F
A(config)#interface FastEthernet0/0
A(config-if)#ip add 10.0.0.1 255.255.255.248
A(config-if)#no shut
A(config-if)#no shutdown
A(config-if)#end
A#
00:11:44: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
00:11:45: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
00:11:46: %SYS-5-CONFIG_I: Configured from console by console
Building configuration…
[OK]
A#
A#

 
OK – let’s go on B:
 

Router>en
Router>enable
Router#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)#host
Router(config)#hostname B
B(config)#int
B(config)#interface F
B(config)#interface FastEthernet0/0
B(config-if)#ip add 10.0.0.2 255.255.255.248
B(config-if)#no shutdown
B(config-if)#end
B#
00:16:06: %SYS-5-CONFIG_I: Configured from console by console
B#
B#
00:16:06: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
00:16:07: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state to up
B#
B#write
Building configuration…
[OK]
B#

 
Now comes the “different one” i.e router C:

Router>
Router>en
Router>enable
Router#conf t
Enter configuration commands, one per line. End with CNTL/Z.
Router(config)#rou
Router(config)#host
Router(config)#hostname C
C(config)#int
C(config)#interface F
C(config)#interface FastEthernet0/0
C(config-if)#ip add 10.0.0.100 255.255.255.0
C(config-if)#no shu
C(config-if)#no shutdown
C(config-if)#write
00:01:38: %LINK-3-UPDOWN: Interface FastEthernet0/0, changed state to up
00:01:39: %LINEPROTO-5-UPDOWN: Line protocol on Interface FastEthernet0/0, changed state
C(config-if)#
C(config-if)#end
C#wr
00:01:44: %SYS-5-CONFIG_I: Configured from console by consoleite
Building configuration…
[OK]
C#

 
 
OK – now we have three routers with two on same /29 subnet and third one on a /24 subnet. Let’s try to ping:

A#ping 10.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/21/24 ms

 
Worked. OK – let’s try C now:
 

A#ping 10.0.0.100
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.100, timeout is 2 seconds:
…..
Success rate is 0 percent (0/5)
A#

 
Failed.
 
Let’s look at routing table on A:

A#sh ip route connected
10.0.0.0/29 is subnetted, 1 subnets
C 10.0.0.0 is directly connected, FastEthernet0/0
A#

 
OK – so only one entry for 10.0.0.0/29 which is connected directly to Ethernet interface 0/0.
 
If we look at ARP table, we get:

A#
A#sh ip arp
A#sh ip arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.0.0.2 12 ca03.35dc.0008 ARPA FastEthernet0/0
Internet 10.0.0.1 – ca02.35dc.0008 ARPA FastEthernet0/0
A#

 
This answers the question completely. 🙂
We have only A and B it ARP table of A. Both B and C are connected to same switch, same VLAN but C is not “visible” to A because C is on /24 subnet and that means last IP of 10.0.0.0/24 i.e 10.0.0.255 is acting as broadcast. C is sending it’s live updates for ARP on 10.0.0.255 while A and B are doing that on broadcast IP 10.0.0.7 (last IP from 10.0.0.0/29) and are not “hearing” on 10.0.0.255. Hence A and B are “hearing” on same broadcast and are updating ARP table while C even connected on same switch, same layer 2 is not connected on layer 3 and thus ICMP ping does not work.
 
It pretty much about those three fundamental questions. Again, third question was “How I will get there?” needs an answer from routing table. So let’s go ahead and tell A about lonely router C. 🙂
 

A(config)#ip route 10.0.0.0 255.255.255.0 F
A(config)#ip route 10.0.0.0 255.255.255.0 FastEthernet0/0
A(config)#end
A#writ
00:38:50: %SYS-5-CONFIG_I: Configured from console by consolee
Building configuration…
[OK]
A#
A#

 
Now there’s a static route entry on A which tells where it for subnet 10.0.0.0/24 and IP 10.0.0.100 belongs to that.
Let’s try pinging again:

A#ping 10.0.0.100
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.100, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/21/24 ms
A#

 
And A can now ping C.  What about C?

C#ping 10.0.0.1
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.1, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/21/24 ms

 

C#ping 10.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
…..
Success rate is 0 percent (0/5)
C#

 
So now C is able to reach A but not B because again B doesn’t knows where is C. If we look at ARP table of C now:

C#
C#sh ip arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 10.0.0.1 3 ca02.35dc.0008 ARPA FastEthernet0/0
Internet 10.0.0.100 – ca06.35dc.0008 ARPA FastEthernet0/0
C#

 
So 10.0.0.2 i.e B is still missing. Let’s tell B to “default” all traffic via A.
 

B>
B>en
B>enable
B#conf t
Enter configuration commands, one per line. End with CNTL/Z.
B(config)#ip ro
B(config)#ip route
B(config)#ip route 0.0.0.0 0.0.0.0 10.0.0.1
B(config)#end
B#write
Building configuration…
[OK]
B#

 
Checking again on B now:
 

C#ping 10.0.0.2
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.0.0.2, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 20/21/24 ms
C#

 
 
So that’s about it.
 

Summary

Ping works on ICMP and it is on layer 3. For it to work, layer 2 should be connected along with working logical layer 3 with entries in routing table. Unless two directly connected machines are on same subnet they won’t ping because different subnet will get them on different broadcast IP and thus different ARP tables all together.
And don’t miss that lot of firewalls block ICMP partially as well as completely because it tends to overload routers in processing those packets rather then doing normal switching operation.
 
Time for me to get back to work!
 
Note: My comments on college teachers are specifically for two teachers (and everyone around me knows whom I am referring too) and one should not make a general impression out from those. There are some very good teachers in dept. as well.