Jio-Airtel PNI up in Delhi NCR!
I visited Delhi earlier today and noticed latency from my phone on Jio 5G to my home (on an ISP behind Airtel) was just 20ms. It varied a bit (as one would expect on a wireless radio network) but 20ms is special because until now it was at least 80-90ms. After all, Jio and Airtel were not connected in Delhi NCR until now. There were sometimes jokes about them being connected at NIXI Noida but that never pushed any traffic because NIXI injects its route server AS24029 in the AS_PATH and while their direct PNIs (i.e Private Network Interconnect) in Mumbai & Chennai would have a direct (short) AS_PATH.
Delhi - Mumbai latency
20ms between Jio Delhi and Airtel Haryana is impossible unless they exchange traffic within NCR. This is because round trip latency from Delhi - Mumbai is between 23ms to 40ms depending on the network. I have seen 23-24ms on Tata Comm (AS4755), 30ms on Airtel (AS9498) and 40ms on Jio (AS55836). So for packets between Jio (in North India) and Airtel (in North India), it was at least (30+30) = 60ms besides radio latency (30-150ms) when looking from 4G/5G.
While I was curious, I could not trace and confirm routing. On Jio 5G I don’t have IPv4 (it’s 464XLAT) & at home I don’t have IPv6. Direct packet flow works but traceroutes do not. Remember when tracing the tool would send a packet with TTL 1, next hop will reduce it to 0 (as IP forwarding standard), drop the packet and reply with “TTL exceeded” with its IP and utility at my end is expected to know the IP of that first hope and same with TTL 2, TTL 3 onwards. This behaviour does not happen when there is no IPv4 on the access network.
I just checked the smokeping instance running at home behind IAXN (which is behind Airtel) towards the Jio Delhi speed test node and as expected, there is a massive drop in the latency!
A zoomed-in view shows that latency went down from 40ms to 12ms on 29th March at 00:00 hrs (GMT+5.5).
A direct trace from IAXN > Airtel > Jio within NCR reflects the same:
anurag@desktop ~> mtr -wr speeddlh1.jioconnect.com
Start: 2023-04-14T03:45:44+0530
HOST: desktop.rtk Loss% Snt Last Avg Best Wrst StDev
1.|-- _gateway 0.0% 10 0.2 0.2 0.1 0.2 0.0
2.|-- 172.16.52.1 0.0% 10 1.3 1.2 0.9 1.6 0.3
3.|-- 182.74.20.9 0.0% 10 2.0 2.1 1.2 2.5 0.4
4.|-- 116.119.72.112 0.0% 10 7.4 5.1 3.9 9.1 1.7
5.|-- 116.119.61.49 0.0% 10 17.2 26.5 13.0 52.3 12.5
6.|-- 49.44.187.164 0.0% 10 10.7 10.6 10.3 10.9 0.2
7.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
8.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
9.|-- ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
10.|-- 49.44.56.202 0.0% 10 12.0 11.6 11.3 12.0 0.2
anurag@desktop ~>
Routing Trivia :-)
So which one is Jio in the trace? It’s either hop 6 (first Jio IP in the trace) or hop 5 (Airtel IP but on Jio’s router for interconnection). To find out, let’s look at the return trace i.e Jio -> Airtel -> IAXN
Return trace from Jio’s DEL-NLD-01 router towards my ISP’s loopback IP in the same city via their looking glass:
Connected to machine# : DEL-NLD-01
Command sent: traceroute ipv4 202.9.120.253 source SourceIP
traceroute ipv4 202.9.120.253 source DEL-NLD-01
Fri Apr 14 03:46:40.042 IST
Type escape sequence to abort.
Tracing the route to 202.9.120.253
1 172.X.X.X [MPLS: Label 287083 Exp 0] 2 msec
172.X.X.X 4 msec
172.X.X.X 2 msec
2 172.X.X.X 2 msec 3 msec
172.X.X.X 2 msec
3 49.44.187.165 3 msec 2 msec 8 msec
4 116.119.61.48 7 msec 8 msec 7 msec
5 116.119.72.113 8 msec 8 msec 8 msec
6 * * *
7 202.9.120.253 11 msec 11 msec 11 msec
This trace makes it tricky to guess because there are two direct pairs here. Both traces have the following IPs coming from the pair:
- 49.44.187.164/31 - 49.44.187.164 & 49.44.187.165
- 116.119.61.48/31 - 116.119.61.48 & 116.119.61.49
A reliable way to confirm if #1 is the interconnection IP is by tracing to each and comparing hops. Notice that I am tracing it from home from behind the Airtel side:
anurag@desktop ~> mtr 116.119.61.48 -wr
Start: 2023-04-14T04:13:19+0530
HOST: desktop.rtk Loss% Snt Last Avg Best Wrst StDev
1.|-- _gateway 0.0% 10 0.3 0.3 0.2 0.7 0.2
2.|-- 172.16.52.1 0.0% 10 1.4 1.4 1.1 1.9 0.3
3.|-- 182.74.20.9 0.0% 10 1.8 2.2 1.8 2.7 0.4
4.|-- 116.119.61.48 0.0% 10 4.3 4.8 4.3 5.7 0.5
anurag@desktop ~> mtr 116.119.61.49 -wr
Start: 2023-04-14T04:13:38+0530
HOST: desktop.rtk Loss% Snt Last Avg Best Wrst StDev
1.|-- _gateway 0.0% 10 0.7 0.4 0.2 0.7 0.2
2.|-- 172.16.52.1 0.0% 10 1.3 1.4 0.9 1.7 0.3
3.|-- 182.74.20.9 0.0% 10 2.3 2.0 1.5 2.4 0.3
4.|-- 116.119.72.112 0.0% 10 5.4 5.7 4.0 11.1 2.5
5.|-- 116.119.61.49 0.0% 10 11.3 9.9 9.2 11.3 0.7
anurag@desktop ~> mtr 49.44.187.164 -wr
Start: 2023-04-14T04:00:12+0530
HOST: desktop.rtk Loss% Snt Last Avg Best Wrst StDev
1.|-- _gateway 0.0% 10 0.3 0.2 0.2 0.3 0.0
2.|-- 172.16.52.1 0.0% 10 1.4 1.3 0.9 1.8 0.3
3.|-- 182.74.20.9 0.0% 10 2.4 2.1 1.6 2.7 0.4
4.|-- 116.119.72.112 0.0% 10 4.3 4.6 4.1 6.9 0.9
5.|-- 116.119.61.49 0.0% 10 9.0 10.3 8.7 15.2 2.5
6.|-- 49.44.187.164 0.0% 10 10.6 10.7 10.3 11.1 0.3
anurag@desktop ~> mtr 49.44.187.165 -wr
Start: 2023-04-14T04:00:32+0530
HOST: desktop.rtk Loss% Snt Last Avg Best Wrst StDev
1.|-- _gateway 0.0% 10 0.2 0.2 0.2 0.2 0.0
2.|-- 172.16.52.1 0.0% 10 1.5 1.3 0.8 1.8 0.4
3.|-- 182.74.20.9 0.0% 10 2.0 1.9 1.5 2.2 0.2
4.|-- 116.119.72.112 0.0% 10 4.6 4.8 4.1 8.0 1.1
5.|-- 49.44.187.165 0.0% 10 12.8 10.6 9.0 15.3 1.9
We can ignore 1st hop (my local router at home) and 2nd hop (my ISPs IAXN PPPoE termination device/router).
3rd hop 182.74.20.9 is upstream Airtel router either in Rohtak or Delhi NCR. Cannot be sure from latency because the distance is less than 100km. Likely Rohtak but can’t be sure. But anyway, that doesn’t matter. It’s just the router doing BGP with IAXN. Let’s merge IPs from the hop onwards which can give an answer to the question:
4th hop: 116.119.61.48 / 116.119.72.112 - two IPs on Airtel router somewhere in Delhi NCR for sure (based on latency) 5th hop: 116.119.61.49 / 49.44.187.165 - one Airtel & one Jio IP on the same router 6th hop: 49.44.187.164 - Jio IP on “some” (Airtel or Jio?) router
Hop 5 and 6 answers that it’s hop 5 (Airtel) on 49.44.187.165 and hop 6 Jio on 49.44.187.164. Hop 5 shows it has both Airtel as well as Jio’s IP and thus is an Airtel router. Hop 6 has only Jio IP & thus 5 & 6 are the ones doing the L3 interconnection. A cross-check of this can be done by looking at Airtel’s looking glass - any of their routers for the entry for 49.44.187.165:
Fri Apr 14 04:24:12 IST 2023
show route detail 49.44.187.165 | no-mor
lookingglass@MUM-GPX-ISP-RTR-189> ....187.165 | no-more
inet.0: 1056292 destinations, 2121817 routes (1056292 active, 0 holddown, 3646 hidden)
49.44.187.164/31 (2 entries, 1 announced)
*BGP Preference: 170/-101
Next hop type: Indirect, Next hop index: 0
Address: 0x1946283c
Next-hop reference count: 133
Source: 203.101.87.127
Next hop type: Router, Next hop index: 0
Next hop: 116.119.33.123 via xe-3/0/0.0 weight 0x1, selected
Label operation: Push 106773
Label TTL action: no-prop-ttl
Load balance label: Label 106773: None;
This confirms it 100%. :-) What would be the capacity of this PNI? Well, can’t tell from routing. I would guess it to be atleast a 100G interconnect because Mumbai & Chennai ones are known to be 100G each.
Impact of this new PNI
A few noticeable things for end users:
- Quality of VoWifi calls will increase significantly. This will shred 60ms of extra travel of packets to Mumbai from North India when someone is using Jio mobile & Airtel FTTH or vice versa.
- Peer-to-peer calling apps like WhatsApp will have much better quality between users of Airtel & Jio.
- Great for gaming folks who were tired of complaining about this issue.
- The impact would be more visible for Jio users because Airtel has a large number of downstream & thus suddenly latency has dropped from Jio North India to a few hundred networks in North which do not connect at NIXI but have IP transit from Airtel.
- Much better experience for non-Jio users to stream IPL on Jio’s streaming apps. :-)
Remember somwehere up the transit path, there is always peering!