06 Jul

ISC F root server – IPv6 issue at NIXI Chennai

Last week I noticed that F root was showing poor connectivity with Indian RIPE Atlas probes for F-root. The graph looked really terrible.

 

 

 

I traced to it from one of RIPE Atlas probes and saw this trace:

 

Here the last hop before timeout i.e hop 6 is of NIXI Chennai peering subnet 2001:de8:1:2::/64. As soon as I saw it, it reminded me older issue which happened and broke IPv4 connectivity to root DNS servers. I blogged about it here, here and here. So the problem remains that NIXI is broken cost wise due to charge on in – out policy. This leads to people accepting routes at all NIXI’s but they do not announce their routes. Thus return path is broken and essentially traffic is being blackholed. Earlier this issue was fixed by adding IP transit support to these root DNS servers so that a default route stays in case of all other failures.

It seems like same is missing in IPv4 world and routes are not being announced.

During this time, I saw two BGP sessions at NIXI Chennai for F root:

2001:de8:1:2::3 24049 ESTAB 25d 3h10m 1 0 2263 0
2001:de8:1:2::4 24049 ESTAB 125d18h30m 1 0 2264 0

 

Both were announcing prefix covering F root server’s pool:

 

I posted about it on SANOG and APNIC mailing list. Though there hasn’t been any reply by ISC, Sunny from APNIC passed info to them and I noticed that prefix announcement from NIXI has been withdrawn. Connectivity to F root now works to the instances outside India.

 

 

 

Waiting to hear from ISC as of now. Time to get back to work!

 

15 Jun

Routing with North East India!

A few weeks back I got in touch with Marc from Meghalaya. He offered to host RIPE Atlas probe at Shillong and that’s an excellent location which isn’t there on RIPE Atlas coverage network yet. It took around 5 days for the probe to reach Shillong from Haryana. I think probably this probe is the one at the most beautiful place in India. 🙂

Now that probe is connected, I thought to look into routing which is super exciting for far from places like Shillong. Marc has a BSNL FTTH connection & mentioned about not-so-good latency. Let’s trace to 1st IP of the corresponding /24 pool on which probe is hosted:

 

This is interesting output. So there are two parts of it:

  1. Traffic going via Bangladesh
  2. Traffic to Bangladesh going via Europe!

 

While #1 may look like a routing issue, it’s actually desired result of a deal between BSNL & BSCCL. I blogged about it in last year when it was visible in BGP tables. Eventually, this link was launched by Indian Prime Minister Modi.

 

From the map, it seems like an ideal choice but I really wish BSNL went for some kind of circuits instead of transit with BSCCL. Reason being poor routing across Asian backbones which we see in reason #2.

 

Coming to #2 – this clearly is bad and broken. Traffic is hitting from Siti broadband > Airtel > Telecom Italia > BSCCL and this is resulting in traffic going from India to Europe first before returning to South Asia.

 

Let’s trace to same 1st IP of the pool from all Indian RIPE Atlas probes for a detailed picture:

Measurement result: https://atlas.ripe.net/measurements/8844267/

 

As we can see latency numbers are quite decent from BSNL’s AS9829 itself. 60-70ms seems fine considering it’s from the probes which are in North or South India to far away in North East. Let’s look at some of these traces from probes on BSNL itself:

 

 

This shows that there is indeed a direct backbone circuit of BSNL to that location. There’s a low chance of it being on top of BSCCL infra.

 

Except for BSNL, rest all other Indian networks are routing towards that BSNL segment in Meghalaya from Europe or Singapore/Hong Kong. All the ones from Europe are from Marseille in France. That’s the landing station for 11 cable systems:

  • SEACOM
  • SEA-ME-WE-4
  • EIG
  • I-ME-WE
  • Ariane 2
  • Atlas Offshore
  • Med Cable
  • TE North
  • Tamares Telecom
  • Alexandros
  • AAE-1 (Asia Africa Europe)

 

Out of these Se-Me-We-4 lands in Bangladesh and I guess that is being used by BSCCL for traffic. So coming back to why routing is so terrible from Indian networks towards BSNL in North East? To understand that we need to look at uplinks of BSCCL.

Well, BSNL is announcing 117.247.134.0/24 to BSCCL AS132602 only. BSCCL is buying transit from Telecom Italia AS6762 and NTT AS2914.

http://bgp.he.net/AS132602#_graph4

 

Looking at one of few traces from Europe:

 

213.144.176.194/31 TIS – BSCCL connectivity
213.144.176.194 – 10Gig port on TIS AS6762 router in Marseille
213.144.176.195 – TIS’s IP on BSCCL router in somewhere in Bangladesh

 

Next, looking at NTT AS2914 transit of BSCCL:

 

Here as traffic handoff from Tata AS6453 is happening to NTT AS2914 in Singapore (logical and correct!) and NTT to BSCCL also within Singapore.  The latency is high due to bad return. Here forward is slightly bad but not as bad as return possibly.

Let’s look at return trace to 2nd hop 115.118.168.1 from RIPE probe at destination (measurement here):

So clearly return path i.e Shillong to Hyderabad is via Europe because BSCCL used TIS for forwarding path.

 

So keeping above traces in mind, here’s the reason for high latency:

  1. BSNL is routing traffic over its backbone but rest all traffic i.e which is not going towards BSNL is being routed from Bangladeshi provider BSCCL.
  2. BSCCL is announcing routes to NTT AS2914 in Singapore & TIS AS6762 in France. Thus to send any traffic to BSNL’s segment in Meghalaya, one has to send it either via TIS router in Marseille, France or NTT Singapore. This adds up latency significantly for Indian networks (excluding) towards BSNL Meghalaya.
  3. BSCCL is using TIS AS6762 to reach Tata AS6453 and this is resulting in very bad return route and thus Meghalaya to any other network in India who is Tata AS6453 downstream is via Marseille, France.

Quite a lot seems messed up. BSNL’s should at least start announcing 117.247.134.0/24 immediately across NIXI’s subject to capacity between their core network in North East. If there’s a capacity constrained, they should use L1 circuits from BSCCL to connect network in Shillong instead of IP transit.

 

How is BSNL in North East reaching Google?

Seems direct to BSNL’s PNI with Google within India.