Routing Crunch: Reliance-Flagtel to Neotel South Africa

15-12-2011

Quick traceroute between Reliance and Neotel (South African subsidiary of Tata Communications)

Using Reliance Mumbai node:

You requested a Traceroute from Mumbai (62.216.135.130) to neotel.co.za (41.168.1.110)

   1 ge-0-2-0.0.pjr02.mmb004.flagtel.com (85.95.25.70) 80.252 ms 79.822 ms 79.737 ms  
   MPLS Label=351552 CoS=0 TTL=1 S=1  
   2 so-1-3-0.0.pjr02.hkg005.flagtel.com (85.95.25.118) 79.688 ms 79.717 ms 79.779 ms  
   MPLS Label=301328 CoS=0 TTL=1 S=1  
   3 so-5-0-0.0.cjr04.hkg003.flagtel.com (85.95.25.214) 79.790 ms 81.271 ms 84.868 ms  
   MPLS Label=300272 CoS=0 TTL=1 S=1  
   4 xe-1-2-0.0.cji01.hkg003.flagtel.com (62.216.128.66) 79.504 ms 79.587 ms 79.452 ms  
   5 62.216.145.194 (62.216.145.194) 85.660 ms 79.769 ms *  
   6 if-2-0-0-1123.core3.HK2-HongKong.as6453.net (216.6.95.137) [AS 6453] 79.930 ms 79.884 ms 79.775 ms  
   7 180.87.15.81 (180.87.15.81) [AS 6453] 295.413 ms 295.675 ms 295.695 ms  
   MPLS Label=300800 CoS=0 TTL=1 S=1  
   8 if-5-2.tcore2.CXR-Chennai.as6453.net (180.87.15.70) [AS 6453] 296.944 ms if-2-2.tcore1.SVW-Singapore.as6453.net (180.87.12.1) [AS 6453] 311.951 ms 311.784 ms  
   MPLS Label=299968 CoS=0 TTL=1 S=1  
   9 if-3-3.tcore1.CXR-Chennai.as6453.net (180.87.36.5) [AS 6453] 311.636 ms if-5-2.tcore1.CXR-Chennai.as6453.net (180.87.12.54) [AS 6453] 312.689 ms 312.041 ms  
   MPLS Label=300688 CoS=0 TTL=1 S=1  
  10 if-4-0-0.core1.CFO-Chennai.as6453.net (180.87.36.18) [AS 6453] 144.922 ms if-15-1-0.core1.CFO-Chennai.as6453.net (180.87.36.14) [AS 6453] 145.223 ms if-4-0-0.core1.CFO-Chennai.as6453.net (180.87.36.18) [AS 6453] 144.627 ms  
   MPLS Label=1578 CoS=0 TTL=1 S=1  
  11 116.0.86.1 (116.0.86.1) [AS 6453] 155.796 ms 155.857 ms 154.918 ms  
   MPLS Label=3107 CoS=0 TTL=1 S=1  
  12 116.0.86.30 (116.0.86.30) [AS 6453] 341.019 ms 312.304 ms 322.348 ms  
   MPLS Label=306816 CoS=0 TTL=1 S=1  
  13 if-2-2.tcore1.JSO-Johannesburg.as6453.net (216.6.55.85) [AS 6453] 312.133 ms 311.860 ms 311.718 ms  
  14 216.6.55.38 (216.6.55.38) [AS 6453] 324.450 ms 295.062 ms 296.033 ms  
  15 41.160.0.243 (41.160.0.243) [AS 36937] 312.371 ms 313.074 ms 311.998 ms  
  16 * * *  
  17 * * *  
  18 * * *

Reasons for crazy routing

Packets were taken from Mumbai to Chennai via Hong Kong!
Reliance has direct connectivity* to Johannesburg, South Africa via SEACOM cable link from Mumbai.

Why packets were taken outside India? Because Flagtel AS15412 and Tata Comm’s AS6453 are pretty much isolated. They mostly peer at niXi via their regional ASN’s which are Reliance Infocomm AS18101 and Tata-VSNL AS4755 = Main Problem.

Why Reliance didn’t used SEACOM rather then handling packets to Tata at Hong Kong?

Simply because of destination. Destination network is Neotel which is subsidary of Tata and relies for almost all of it’s International bandwidth on Tata’s AS6453. We can see routes via AS18101 just terminate - thus no local connectivity option for Reliance to handle packets directly to Tata within India (avoiding extra round trip to Hong Kong). Apart from that it seems like Reliance has no nodes/PoP’s in Africa. I tried bgp lookups to lot of African ISP’s and most are via London because Reliance has no local presence in African market and hence Reliance connects to African Networks at London where most of upstream providers of African Networks meet. Apart from that SECOM does provides in-direct link between Mumbai & South Africa as SECOM ring which starts from Mumbai lands in Tanzania, and Tanzania is connected to South Africa via Mozambique. Have a look at SEACOM map below:

So what’s the real world effect of such routing?

I can see neotel.co.za is blocking ICMP pings. Let’s use other IP which is before firewall - 41.160.0.243 which replies to ping. Now round trip latency to this IP from Reliance Mumbai node is ~ 311ms.

This could be well under 150ms if a direct route is established because actual distance is near around 5000miles. We actually get such performance between Tata Comm (Indian PoP’s) to Neotel as Tata is routing traffic via Cochin PoP where SAFE cable lands.

Result:

Router: gin-mlv-core1   
Site: IN, Mumbai - MLV, VSNL   
Command: ping 41.160.0.243

Sending 5, 100-byte ICMP Echos to 41.160.0.243, timeout is 2 seconds:  
!!!!!  
Success rate is 100 percent (5/5), round-trip min/avg/max = 128/128/128 ms

With hope that eventually we will have better routing between Indian ISP’s time for me to get back to Java learning! (well Practical test a day after tomorrow!) :(