Personal blog of Anurag Bhatia

IP Telephony in India: From regulation to reality

Mon, 28 Sep 2026 20:34:38 +0530

I spent some time looking at IP telephony in India, while also doing my own setup for a few use cases and some fun stuff. As I went through this, I decided to document what is possible and what is not when it comes to IP telephony in India.

History

IP telephony in India is getting exciting after a long time. For those who may not know, India has been extremely restrictive when it comes to voice. Part of it had to do with the fact that until 2016, voice was actively sold on a per-minute basis and was a key revenue source for mobile operators. The other part has been regulation.

On the technology front, India has been open to VoIP (Voice over Internet Protocol) as a technology, but extremely restrictive when it comes to VoIP-to-PSTN interconnection. When I use the term IP Telephony in this post, I mean VoIP to PSTN/mobile network interconnect and not IP-to-IP calling, where both end devices are on the internet — like in the early days with Skype-to-Skype calls, later Fring, Nimbuzz, Viber, and these days with WhatsApp, Signal, Arattai, etc. Those were always fine, except that telco lobby groups have, over time, argued that they should be brought under the licensing regime as well. But better sense prevailed, and they stayed outside the licensing requirements.

On the IP-to-PSTN front, for a very long time it was not allowed at all. I remember there was a time when Indian call centres had two desk phones — one for India and one for the rest of the world. The Indian one would be connected to a PBX, which in the early days connected to a PRI and later to a SIP trunk. The global one connected to a separate PBX, which connected to SIP trunk providers in the US, UK, Singapore and Australia over the internet.

Later, things cleaned up a bit with the OSP (Other Service Provider) licence, under which BPOs could run full IP infrastructure internally (like they were doing for global systems) with the Indian phone network. There were strict limitations: a) the SIP trunk had to terminate over private last-mile access at their office building, and b) their system interconnection with voice providers over SIP had to remain strictly inside the building. They were not allowed to carry it over the internet outside of that network. It was useful for call centre operations, as they could now run more integrated systems — single desk phones, CRM integration and telephony — but employees still had to be strictly in the office.

Meanwhile, as e-commerce, app-based taxis and similar services became popular, a major requirement for number masking emerged. Providers like Knowlarity and Exotel picked up this requirement, and number masking became a major source of demand for them. This was relatively straightforward from a regulatory perspective, as they were doing only PSTN-to-PSTN interconnection. The signalling part was somewhat of a grey area. Whether signalling could be done over the internet or not was unclear. All these apps surely needed this over the internet. For example, Uber itself runs over the internet, bookings happen over the internet, and the masked number has to be shared through the app with both sides. The actual call would then run over the regular phone network.

Ringo Vs telcos

One interesting case I remember is Ringo, from 2015. It was a company founded by Bhavin Turakhia — a billionaire who made his money from domain names, hosting and SaaS. He created an app called “Ringo” which would take a request from a caller for a destination number, then signal both the caller and the destination number through a central system. It would first trigger a call to the caller itself and, as soon as the caller picked up, call the destination and bridge the two calls.

Ringo was able to charge ₹0.19/min for this when market rates were much higher (I think ₹0.60 to ₹1/min back then). He did a deal with Quadrant Televentures (the telecom arm of Videocon) and had a setup in Mohali which would do the call bridging. Back then, the voice call termination rate was ₹0.14/min, but only between mobile operators. If one leg was fixed-line, it was free. This ran for only a few weeks before Airtel pushed back strongly. There was even a case where Mr Sunil Mittal complained to the telecom minister about this, and it was ultimately taken down, giving the impression that even signalling over the internet was not allowed.

Bhavin later got access licences and tried setting up Ringo as a full-fledged telecom operator with VoIP. That also did not work out, with reports that they were unable to get voice interconnects with anyone except BSNL.

I don’t remember what happened to Ringo, but I guess they were struggling with the telecom lobby. At the same time, Jio started in 2016, making voice free and unlimited from day one and killing much of the business that existed around retail voice. Even Jio had its share of struggles with voice interconnect capacity as well as termination charges in the early days. Later, in 2018, BSNL launched BSNL Wings — an IP telephony service. While the service itself was pretty buggy and had billing issues, it proved that, from a regulatory perspective, IP telephony was allowed.

Later, around 2019, operators started working on VoWiFi and launched it by early 2020. That moved a visible part of mobile voice traffic over to IP at homes, offices, etc. Another key development happened in early 2020 — COVID lockdowns. As that started, more and more people started working from home and found that the limitations around the OSP licence and locally declared PBX setups had become a bottleneck. Eventually, they were relaxed. Mr Sanjeev Sanyal has covered this a few times in his interviews. Until that point, DoT had rules that were too restrictive to let employees work remotely. Eventually, by the end of 2020 or early 2021, the OSP licence as a concept was removed.

Present situation

Regulation

While it’s hard to say anything with certainty when it comes to Indian telecom regulations, I think at this point it’s safe to say that IP telephony is now allowed much more broadly, but with two key limitations:

The IP-to-PSTN part of the call can be handled only by authorised access service providers. Anyone without a licence cannot connect IP with PSTN in India.
For IP telephony in India, both legs of the call need to be in India. One leg is obviously in India for a domestic service, but the other end cannot be outside India.

So it’s better than what it used to be, but it still carries some restrictions.

While there are many types of licences, the number of access providers actually offering voice at scale is quite small. These include Airtel, Jio, Vodafone Idea, Tata Tele (TTSL), BSNL, etc. There may be some smaller ones operating in limited areas, but these are pretty much the ones with full voice infrastructure, DID allocation and voice interconnects at national scale.

As far as I can see, many IP telephony players are holding Access Licenses in India now but are still working closely with an existing telecom player instead of getting DIDs and doing full autonomous rollout. Take, e.g this list from DoT for Mar 2026, which has RingCentral, Webex, Zoom (ZVC India - Zoom Video Communications), etc.

I think, commercially, these companies have chosen to work with existing telecom players rather than do a full rollout, avoiding the cost of PoIs, interconnection negotiations, lawful interception and the other compliance requirements that come with running the network themselves. Few players also seem to be working with a VNO licence with access, e.g. Plivo, Telnyx and Zoho, as per this list. Since these are VNOs, they essentially resell telecom voice services with their own API layer on top.

Market

Domestic retail voice is largely a dead market for IP telephony players. At this point, mobile operators offer voice-only plans for as little as ₹1,800 / $18.76 per year. That works out to an ARPU of around ₹150 / $1.56 a month, with mobility and pan-India coverage included. So only VoIP players like WhatsApp-to-WhatsApp calling can offer VoIP as a free service on top of a free product that is monetised on the other side, i.e. the business side with WhatsApp for Business. IP telephony is a hard sell at such low ARPUs, especially when IP telephony itself cannot replace the need for end-user voice. All data plans bundle unlimited calling anyway.

But there is a growing market among businesses that need features like multiple DIDs and multi-channel trunks. Traditionally, these have been tied to the last-mile provider, which had to serve them over its own local loop. But with a newer set of API-oriented players, telephony can be delivered over any internet connection.

An even bigger opportunity for many of these players is voice for AI agents. The focus is increasingly on connecting IP telephony to AI agents so they can make and receive calls for marketing, support and other use cases. A lot of these players focus and specialise in the API layer. Twilio outside of India, is the pioneer of this & same model is being copied over.

Even infrastructure-heavy players like Tata Communications talk and focus heavily on Kaleyra after its acquisition of the Italy-based voice API company for $250 million in 2023. They don’t offer it in India yet, but given that it is Tata Communications, they could in theory, tap into sister company Tata Tele’s access infrastructure and launch at some point

Players doing IP telephony in India as of now include: Tata Tele (TTSL), Plivo, Vobiz, Zoho Voice, Tryvox, BSNL, etc. Almost all of these sell only to businesses. Pure-play telephony is still somewhat missing for SMEs at scale, and it would be interesting to see how it develops.

Also, as far as I can see, traditional telcos are actually not doing it. E.g Jio on their SIP trunk page FAQ on the local loop:

What kind of local loops support SIP Trunk service?

SIP Trunk can be delivered on Ethernet and GPON last mile; also it can be delivered on UBR in some instances based on interface clearance and link stability.

My setup

So I wanted a basic IP telephony setup for myself for a number of reasons. One was to get calls over the internet, which I can control, rather than relying only on the mobile network, which I cannot.
VoWiFi exists, but can be tricky at times. E.g., at the recent APNIC 62 at the Westin Mumbai, I was getting really poor mobile signal and could not use VoWiFi because APNIC uses its own IPv4/IPv6 blocks for the conference, and VoWiFi normally does not work over these temporary IPs. Mobile networks offering VoWiFi maintain their own GeoIP-based filtering, and that can sometimes be slow to pick up new address ranges. Thus, I find VoWiFi useful but with very limited control. A pure IP telephony setup gives me much better control in situations like these. I was also looking for some advanced options, such as having calls reach me when I am offline through alternate alerting methods, voicemail, etc. And I wanted a DID that I could share when I don’t want SMS or WhatsApp spam reaching my primary number.

So here’s what I finalised:

Running a open source FreeSWITCH server at my end as the PBX
For IP telephony I am using Plivo - ₹200/month for the DID plus pay-as-you-go pricing (₹0.38/min) for calls
On the client side, I am using Linphone on Linux and macOS. For the mobile client, I am using Acrobits softphone on iOS.
For security, I am using the usual industry standard: TLS + SRTP between Plivo and FreeSWITCH, and the same TLS + SRTP between FreeSWITCH and my clients. This ensures that both signalling (via SIP) and media (via RTP) are encrypted, though there is no end-to-end encryption once the call reaches Plivo’s underlying telecom provider, Tata Tele, as the traditional telecom network is not end-to-end encrypted.

My son & his fixedline phone

So until this setup, I had an old fixedline phone at home connected to the Airtel CPE for calls. My son rarely used it, mostly to receive calls from his grandfather. He could theoretically dial his grandfather, but never did because dialling an 11-digit number (0 + mobile number) was hard to remember.

Since I already got FreeSWITCH running, it made sense to connect his phone via an old ATA I had. This gave me an excuse to add a bunch of fun features using FreeSWITCH XML configs:

My son can now dial 123 to reach me, and I can receive the call anywhere over IP (both endpoints are IP, with no PSTN involved).
He also has a shortcode, 456, for his grandfather, which FreeSWITCH translates in the backend into the actual mobile number.
I added a fun IVR menu for him that greets him, lets him enter numbers and plays MP3 songs over the landline phone. I was also able to use Piper TTS in Python to generate a personalised Hindi greeting for the IVR.
I created special shortcodes for him which directly land on my voicemail & vice versa. This allows us to leave messages for each other. Quite useful when travelling around.

Grey areas

Indian telecom is full of grey areas. Babus seem to love keeping it that way. The whole AGR mess, regulatory overreach and whatnot. Until recently, the applicable law was the super-old Telegraph Act, which by design was extremely restrictive. It had a licence requirement if one was transmitting a message from point A to point B, which was an extremely broad definition.

As far as I can think of, the following are grey areas as of now when it comes to IP telephony:

Unified Licence with Access authorisation for, say, Mumbai, or a VNO with Access authorisation for Mumbai, may or may not allow services to be sold outside Mumbai. I see some providers doing that and some not. One vague interpretation is also that if IP telephony is sold with just Mumbai access, then as long as a legally operating ISP carries it outside Mumbai (but within India), it is allowed. I am not sure about this interpretation.
Location of PBX: If the PBX is at the end user’s premises, it’s clear and likely OK, but imagine the PBX is also in the cloud. In this case, the setup would be Telecom operator → Licensed voice provider / API layer player → PBX → End user. Both ends, i.e. the PBX and end users, are not licensed. One interpretation can be that the end user-to-PBX connection is the IP leg of the call and the IP-to-PSTN connection is still happening at the licensed player, and hence is OK. Another interpretation can be that it’s part of the overall IP-to-PSTN call and therefore cannot touch unlicensed infrastructure.

Technical limitations

There are some serious limitations in the current design, with one key being too much use of and reliance on existing telcos. Here are some limitations of Indian IP telephony design:

Lack of number portability — there is no way to port a number from one provider to another, even though mobile number portability has been common for a long time.
Lack of DID ownership by API-layer companies. Players like Plivo, Vobiz, etc. are all using provider-specific DIDs, and they would be locked to the provider because of #1, i.e. no portability. Ideally, there should be an easy way for them to obtain DID allocations directly from DoT and run their own network, just like we do in the IP world — autonomous, with their own IPs, transit and peering setup.
Lack of IP-to-IP voice peering — in the US, it’s common to see large IP telephony providers peer with each other for voice over private cross-connects. In India, that seems much harder under the current access-licensing regime.

References

The following links are interesting and worth exploring:

Plivo India Calling regulations - Click here
Medianama discussion on IP in parts (here, here, here, here and here)
DoT’s list of VNOs with access authorisation has some interesting names, which give a hint of the various B2B companies in this business

Last but not least, special thanks to my friend and one of the experts on this stuff, Nitin Lochab, for all the advice as I worked through the FreeSWITCH deployment. 😀

Disclaimer: This is my personal blog, and hence, posts made here are in my personal capacity. These do not represent the views of my employer.

Airtel (AS9498) to take the lead on the RPKI RoV front

Mon, 14 Sep 2026 14:30:47 +0530

Just back from Mumbai after APNIC 62 and Equinix’s India Peering Forum. On excellent news: Airtel has started sending emails to its customers about RPKI RoV (Route Origin Validation). As per the mail, Airtel will start RPKI RoV soon and has mentioned 22 Sep 2026 as the timeline when they will start rolling out RoV. This is excellent news on the routing security front, and kudos to Airtel (AS9498) for taking the lead here among other large Indian backbones.

This brings up the important question of how many routes exist in their table which are RPKI RoV invalids as of today?

Looking at their announcements, I see the following:

RPKI Valids: 33,735
RPKI Unknown: 4632
RPKI invalids: 726

This is for their table, including downstream routes. One can view the complete list of those 726 invalids, including prefix and AS_PATH here. One interesting thing here is that out of 726, 140 are from Airtel fixedline AS24560 and 122 are from Airtel Mobility. In terms of address family, there are 108 invalids from IPv4 and 618 from IPv6.

How many of these invalids are single-homed vs multi-homed?

This is important to check, as often the argument against RoV given by various operators in past was that they were losing traffic to their competitors. When I checked, I do not see any of these prefixes visible outside of India behind a transit-free / tier 1 network without AS9498 in the path. It could be because Tata Comm AS6453 drops RPKI invalid already & most of these are not dual-homed via Jio AS64049. So unless a less specific RPKI valid announcement is there, these are anyway largely blackholed for now by networks & IXPs doing RoV outside of India.

Let’s hope other Indian backbones also start doing RPKI RoV soon. Happy routing! 😀

Indian backbones need AS_PATH filters

Tue, 08 Sep 2026 01:41:07 +0530

The security of India’s routing infrastructure is something that isn’t discussed as much as cybersecurity, software vulnerabilities, etc. Part of the blame for that has to go to our industry, which is relatively closed when it comes to sharing incidents and also lacks good documentation on the deployment side of things. Some of these things came to light after the RCom (AS18101) BGP hijack of Telegram prefixes in June 2026. One silent problem for a long time has been route leaks.

BGP route leak

Route leaks happen when a network ends up announcing a route to a BGP adjacency where it is not supposed to announce that route. Take, for example, a network “leaking” routes learnt from a peer to transit, or vice versa. One can filter downstreams which are small, but it’s very hard to filter large downstream networks which have further downstreams, or if one is far removed from that ASN. IRR AS-SETs exist for this reason, but they have not worked well due to tooling challenges. Furthermore, IRR (Internet Routing Registry) by design is a public register where one “publishes intent” before actually doing that in BGP, and then anyone can match the intent with the state in BGP. New tech ASPA will help address this issue, but it’s new, vendor support is still in the development phase and it will take its own adoption time.

AS_PATH filters / Peer lock

This is older tech which can play a very effective role in controlling leaks. If network A decides to peer with network B, they both can agree to announce all routes to each other over the peering & agree to reject each other’s ASN from all other BGP sessions. It’s a common practice between large transit-free Tier-1 networks as well as major backbones and is often referred to by the fancy name of “peer lock”. In India, I doubt backbones like Airtel, Jio, and Tata Comm can have AS_PATH filters rejecting each other from everywhere except direct sessions because there is some indirect routing visible all the time. It could be because of cold potato routing design or peering capacities.

Most of the leaks which these networks learn are not from their other large peers or upstreams but actually from smaller downstreams. Let’s look at an example of ongoing leaks right now:

Prefix	AS_PATH	Reason
103.166.14.0/24 & 103.166.15.0/24	9498 134674 134674 4755 17762 141846 141846	Tata Play broadband AS134674 learning route from its upstream Tata Comm and leaking it to Airtel
103.126.170.0/24	4755 17426 9498 138282	PRIMENET AS17426 leaking Airtel route to Tata Comm
163.128.230.0/23	4755 134674 9498 154622	Tata Play broadband AS134674 leaking Airtel route to Tata Comm
185.228.168.0/24	4755 45815 9498 20473 205157	ESDS AS45815 leaking route learnt from Airtel to Tata Comm
103.146.120.0/24 & 103.154.251.0/24	4755 136306 9498 45942 140114 140114	Texes Connect AS136306 leaking Airtel route to Tata Comm

All these leaks reach transit free / tier 1 networks and thus the larger internet. While these are small ones, the ability for larger leaks is there and they can impact a lot.

Take, for example, this leak reported on 28 May 2026 by friends at Qrator:

🚨BGP Route Leak at 2026-05-28 08:21 UTC

🇺🇸AS21859 (ZEN-ECN, Distributed Cloud for AI 🤖) leaked 1,334 prefixes learned from 🇮🇳AS4755 (TATACOMM-AS) towards 🇮🇳AS9498 (BBIL-AP), creating 1,334 conflicts with 1005 ASNs in 75 countries.

🌏Max propagation: 68%
⏱️Duration: ~5m pic.twitter.com/MO4TjosMBm
— Radar by Qrator (@Qrator_Radar) May 28, 2026

In this case AS21859 was able to leak 1,334 routes from Tata Comm to Airtel because Airtel’s filters accepted them.

What we logically need is basic AS_PATH rejection towards downstreams of Airtel, Tata Comm, Jio, Vodafone Idea, BSNL, Sify, etc., rejecting everyone else from their downstream side. That would kill 90% of these leaks, as they would be very localised. So something like this towards customer-facing sessions:

policy-options {
as-path Large-Indian-ASNs "_(9498|4755|55836)_";
policy-statement REJECT-LARGE-ASNS {
term BLOCK {
from {
as-path Large-Indian-ASNs;
}
then reject;
}
(...Existing rules...)
term ACCEPT {
then accept;
}
}
}

Happy routing! 😀

Incremental backup on AWS Glacier with rustic

Thu, 20 Aug 2026 02:34:01 +0530

I have been a heavy restic user for several years. In Jan of this year, I tried making restic work with AWS Glacier deep archive but it did not work out. AWS Glacier Deep Archive, for those who may not know, is one of the cheapest storage options, with costs as low as $1/TB, but with very expensive retrieval due to high egress charges from AWS, in addition to retrieval fees, API charges, etc. It can act as a backup of last resort, kind of like insurance if everything else fails, since it costs roughly 1/7th of what Backblaze B2 charges for the same data ($6.95/TB/month).

Restic by design is not compatible with Glacier because it stores a mix of data blobs and tree blobs inside /data. Thus, as AWS lifecycle rules move data from S3 Standard to Glacier, basic repo operations start failing. They do have some experimental support as per FAQ, but it did not work out for me when I tried in Jan.

A few days back, I came across an open-source project named rustic — it’s a restic implementation in Rust and supports cold storage. The rustic client is compatible with a traditional restic repo.

Rustic and cold storage

Rustic supports AWS Glacier as well as OVH cold storage. The way it works is: they keep all data in cold storage and a copy of the metadata in hot storage. This additional copy of metadata in hot storage allows operations like incremental backups, viewing snapshots, and requesting restores where only the specific data blobs needed for the restore are pulled from Glacier. This additional hot storage can sit on S3 standard or even Backblaze B2 (which I prefer due to zero API cost).

Demo

1) Create repo

Let’s create two buckets - one in S3 (to store data in Glacier) and one on Backblaze B2 to hold an additional copy of metadata.

> rclone mkdir s3:rustic-demo-cold
> rclone mkdir b2:rustic-demo-hot

2) Create rustic config profile

This file will define the cold and hot repo locations, along with the password file location, restoration command syntax, etc.

[repository]
repository = "rclone:s3:rustic-demo-cold"
repo-hot = "rclone:b2:rustic-demo-hot"
password-file = "/home/anurag/.demo-rustic-pass"
warm-up-command = 'bash -c "id=%id; rclone backend restore s3:rustic-demo-cold/data/${id:0:2}/ --include \"$id\" -o priority=Standard -o lifetime=7"'
warm-up-wait = "2m"

3) Generate password

This password will encrypt the entire repo, providing encryption at rest.

> tr -dc 'A-Za-z0-9!"#$%&()*+,-./:;<=>?@[]^_{|}~' < /dev/urandom | head -c 16 > /home/anurag/.demo-rustic-pass
>

4) Initial rustic repo

Next, initialize the repo. If you have used restic before, it’s the same step as init, but instead of creating a repo at only one location, it will create it at both hot & cold storage and ensure data is pushed only to cold storage (while metadata is updated on hot).

> rustic -P ~/.config/rustic.toml init
[INFO] using config /home/anurag/.config/rustic.toml
[INFO] using warm-up command bash -c 'id=%id; rclone backend restore s3:rustic-demo-cold/data/${id:0:2}/ --include "$id" -o priority=Standard -o lifetime=7' with batch size 1
[INFO] key b4e08cd6 successfully added.
[INFO] repository 2877a54b successfully created.
[INFO] using cache at /home/anurag/.cache/rustic/2877a54b3c9d7d6ba36a7e4cf4b201c4e10b6e7fdf4473db956a48bde8776c7e
> rclone ls s3:rustic-demo-cold
155 config
363 keys/b4e08cd66e06c4cf8f5be2831036beb2f4ad187a2a5985db6bdc563f901a2003
> rclone ls b2:rustic-demo-hot
169 config
363 keys/b4e08cd66e06c4cf8f5be2831036beb2f4ad187a2a5985db6bdc563f901a2003

5) Generating random files

At this step, we are ready to push data. For this demo, I will use AWS Glacier Instant Retrieval because I cannot wait 12 hours for data to be restored. In real commercial use, however, it doesn’t make sense to use Instant Retrieval. I will actually use glacier deep archive. I am going to set storage_class = GLACIER_IR in rclone s3 config. The price difference between Glacier Instant Retrieval and B2 is actually not that large, so it doesn’t make sense to use Instant Retrieval in production.

> dd if=/dev/random of=file.bin bs=1M count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0140004 s, 374 MB/s
> dd if=/dev/random of=file1.bin bs=1M count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0148932 s, 374 MB/s
> dd if=/dev/random of=file2.bin bs=1M count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0134629 s, 403 MB/s
> dd if=/dev/random of=file3.bin bs=1M count=5
5+0 records in
5+0 records out
5242880 bytes (5.2 MB, 5.0 MiB) copied, 0.0139815 s, 403 MB/s
> du -sh file*
5.0M file1.bin
5.0M file2.bin
5.0M file3.bin

6) Trigger a backup

> rustic -P ~/.config/rustic.toml backup file1.bin file2.bin
[INFO] using config /home/anurag/.config/rustic.toml
[INFO] using warm-up command bash -c 'id=%id; rclone backend restore s3:rustic-demo-cold/data/${id:0:2}/ --include "$id" -o priority=Standard -o lifetime=7' with batch size 1
[INFO] repository rclone:s3:rustic-demo-cold#rclone:b2:rustic-demo-hot: password is correct.
[INFO] using cache at /home/anurag/.cache/rustic/2877a54b3c9d7d6ba36a7e4cf4b201c4e10b6e7fdf4473db956a48bde8776c7e
[INFO] using no parent
[INFO] starting to backup ["file1.bin", "file2.bin"] ...
[INFO] Files: 2 new, 0 changed, 0 unchanged
[INFO] Dirs: 1 new, 0 changed, 0 unchanged
[INFO] Added to the repo: 10.0 MiB (raw: 10.0 MiB)
[INFO] processed 2 files, 10.0 MiB
[INFO] snapshot eab96cac successfully saved.
[INFO] backup of file1.bin,file2.bin done.
> rustic -P ~/.config/rustic.toml backup file2.bin
[INFO] using config /home/anurag/.config/rustic.toml
[INFO] using warm-up command bash -c 'id=%id; rclone backend restore s3:rustic-demo-cold/data/${id:0:2}/ --include "$id" -o priority=Standard -o lifetime=7' with batch size 1
[INFO] repository rclone:s3:rustic-demo-cold#rclone:b2:rustic-demo-hot: password is correct.
[INFO] using cache at /home/anurag/.cache/rustic/2877a54b3c9d7d6ba36a7e4cf4b201c4e10b6e7fdf4473db956a48bde8776c7e
[INFO] using no parent
[INFO] starting to backup ["file2.bin"] ...
[INFO] Files: 1 new, 0 changed, 0 unchanged
[INFO] Dirs: 1 new, 0 changed, 0 unchanged
[INFO] Added to the repo: 410 B (raw: 640 B)
[INFO] processed 1 files, 5.0 MiB
[INFO] snapshot 44d9e8aa successfully saved.
[INFO] backup of file2.bin done.

7) Checking backed-up snapshots

> rustic -P ~/.config/rustic.toml snapshots
[INFO] using config /home/anurag/.config/rustic.toml
[INFO] using warm-up command bash -c 'id=%id; rclone backend restore s3:rustic-demo-cold/data/${id:0:2}/ --include "$id" -o priority=Standard -o lifetime=7' with batch size 1
[INFO] repository rclone:s3:rustic-demo-cold#rclone:b2:rustic-demo-hot: password is correct.
[INFO] using cache at /home/anurag/.cache/rustic/2877a54b3c9d7d6ba36a7e4cf4b201c4e10b6e7fdf4473db956a48bde8776c7e
[00:00:00] getting snapshots... ████████████████████████████████████████ 2/2
snapshots for (host [desktop], label [], paths [file1.bin,file2.bin])
| ID | Time | Host | Label | Tags | Paths | Files | Dirs | Size |
|----------|---------------------|---------|-------|------|-----------|-------|------|----------|
| eab96cac | 2026-08-19 03:15:35 | desktop | | | file1.bin | 2 | 1 | 10.0 MiB |
| | | | | | file2.bin | | | |
1 snapshot(s)
snapshots for (host [desktop], label [], paths [file2.bin])
| ID | Time | Host | Label | Tags | Paths | Files | Dirs | Size |
|----------|---------------------|---------|-------|------|-----------|-------|------|---------|
| 44d9e8aa | 2026-08-19 03:15:51 | desktop | | | file2.bin | 1 | 1 | 5.0 MiB |
1 snapshot(s)
total: 2 snapshot(s)

8) Checking hot & cold repo storage size

If we check the raw storage via rclone again, we should see the data in cold storage but only a tiny amount of metadata in hot storage:

> rclone size s3:rustic-demo-cold
Total objects: 9
Total size: 10.004 MiB (10490114 Byte)
> rclone size b2:rustic-demo-hot
Total objects: 8
Total size: 3.444 KiB (3527 Byte)
> rclone lsf s3:rustic-demo-cold/data/cf/cf5ac179f17f6c13f041845665a94b7bc90cd51bfc929ae633b2033e33f69c5a --format "pT"
cf5ac179f17f6c13f041845665a94b7bc90cd51bfc929ae633b2033e33f69c5a;GLACIER_IR

This confirms that the file is indeed in the Glacier class (Instant Retrieval in this case).

9) Data restore

> rustic -P ~/.config/rustic.toml restore 44d9e8aa ~/Downloads/rustic-demo-restore
[INFO] using config /home/anurag/.config/rustic.toml
[INFO] using warm-up command bash -c 'id=%id; rclone backend restore s3:rustic-demo-cold/data/${id:0:2}/ --include "$id" -o priority=Standard -o lifetime=7' with batch size 1
[INFO] repository rclone:s3:rustic-demo-cold#rclone:b2:rustic-demo-hot: password is correct.
[INFO] getting snapshot ...
[00:00:00] collecting file information... Files: 1 to restore, 0 unchanged, 0 verified, 0 to modify, 0 additional
Dirs: 0 to restore, 0 to modify, 0 additional
[INFO] total restore size: 5.0 MiB
[00:00:00] warming up Pack(s)... ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░ 0/1 [
{
"Status": "Not GLACIER or DEEP_ARCHIVE or INTELLIGENT_TIERING storage class",
"Remote": "cf5ac179f17f6c13f041845665a94b7bc90cd51bfc929ae633b2033e33f69c5a"
}
]
[00:00:00] setting metadata... restore done.
> ls ~/Downloads/rustic-demo-restore
file2.bin
> md5sum file2.bin
da0b706cacb5787a8b43ac636ccd397b file2.bin
> md5sum ~/Downloads/rustic-demo-restore/file2.bin
da0b706cacb5787a8b43ac636ccd397b /home/anurag/Downloads/rustic-demo-restore/file2.bin

Use case

With this, I can migrate a copy of my self-hosted Immich backups from Backblaze B2 to Glacier. I hold multiple backups in different locations/media, and this third backup acts like a backup of last resort.

Typical cost difference for 600GB data:

Storage Provider	Monthly per GB cost (USD/INR)	Total monthly cost for 600GB data (USD/INR)	Restore cost for full data
Backblaze B2	$0.00695 / 0.67 INR	$4.17 / 402 INR	Free (as up to 3x i.e. 1.8T B is free egress bandwidth)
AWS Glacier Deep-archive	$0.00099 / 0.09 INR	$0.59 / 54 INR	(Bulk Retrieval: $2.20 + data transfer outside of 100GB quota: $45.00) $47.20 / 4522 INR

Thus the break-even here is around 396 days. If not restored within this period, it becomes cheaper on Glacier compared to B2. Never use Glacier Deep Archive as the only backup copy for data that you intend to restore regularly.

Limitations

One cannot run restic check --read-data as it would be very expensive in such a setup. I like to do that to validate the consistency of data. However, if this is the 2nd or 3rd copy of the data, I am comfortable relying on AWS’s own integrity checks, as they run their own checksums on the data. One cheap DIY way out here can be to retrieve all the data once a year or so using bulk retrieval cost & then have an EC2 machine in the same region to run the check (and avoid massive egress toll).
AWS Glacier Deep Archive has a minimum storage duration of 180 days. Thus, one cannot delete data before 180 days without incurring an early deletion charge. Depending on what one is backing up, this may or may not be an issue.

Fascinating history of transit-free tier 1 networks

Thu, 06 Aug 2026 03:18:07 +0530

I find the concept of transit-free / Tier 1 networks fascinating. These are networks which, by definition, are transit-free. They do not have any transit and can reach the entire routing table just through peering relationships. The importance of these networks has reduced over time as more and more peering happens directly between eyeball networks and content networks. But regardless, I always find it fascinating that these networks effectively define the full autonomy of the internet routing table. Together with the trust in 13 magical root DNS servers and these transit-free networks define what we call “the internet” today.

I was once chatting with my friend and guru Martin Levy (now happily retired) about which networks were possibly transit-free when the internet started. He suggested that Wikipedia history on Tier 1 networks has many of them. The challenge with the Wikipedia page on Tier 1 networks is that the list of transit-free networks appears on 14 Nov 2005. This leaves a gap around which networks were transit-free before Nov 2005.

Before Nov 2005, Nov 2005 and till now

I had an email exchange with Mr Randy Bush and he kindly replied with some hints. According to him AS701/702/703/704 (UUnet / now Verizon), AS174 (PSI / now Cogent), AS1 (BBN) and AS1239 (Sprint) were transit-free networks before this period. The oldest routing dump on RIPE RIS is from 1999 but those dumps are very small and contain too few routes to establish anything. RIPE RRC00 from September 1999 has “view” files, which are essentially a “sh ip bgp dump” and the following month they seem to have moved to the MRT format. On Oregon Route Views, the oldest I can find is this dump from 08 Nov 1997.

This brings me to a possible, although not perfect, method to document the history of transit-free networks.

Timeline	Method
Before-Nov 1997	Assume AS1, AS1239, AS701 and AS174 to be transit-free and find who else was adjacent to all of them along with BGP routing table dumps
Pre - Nov 2005	Use Wikipedia’s first documented list of tier 1 networks
Nov 2005 - Aug 2006 & onwards	With some scripting, dump Wikipedia page history & parse who became tier 1 when along with the support of BGP routing table dumps to cross-validate it via public route collectors

Limitations

There are some key limitations in my method:

Ignore paid peering: Because it’s impossible to know about all paid peering agreements, I think, from a technical perspective, it makes sense to focus on “adjacencies” and finding whether a network is transiting via a transit-free network to another transit-free network or not. Take AS174 (earlier PSI and now Cogent) for example publicly states in this release on this release of possible paid peering outside of settlement-free terms with Level 3 in 2005. The Wikipedia article uses this as a reference, as it didn’t classify AS174 as transit-free until 2014. On technical grounds I am just ignoring that, as we never know who else in the list also had similar arrangements. If AS X transits via Y to reach Z", then X is not tier 1. If X has direct adjacency to Y and Z, I assume it is to be “transit-free” (with a mix of settlement-free and/or paid peerings).
Unknown relation with last transit: If X transits via Y to reach Z and later establishes settlement-free peering with Z, X will appear to be directly adjacent to Y & Z, while X-Y is still a transit relation. Unless X negotiates with its last transit to become a peer as well. Take the case of AS7922 (Comcast) - which is adjacent to all networks but was transiting via AS6453 until recently. AS6453 still tags them as “6453:50 - Customer route”. It would be hard to state AS7922 as transit-free or not. Technically AS7922 is not transiting AS6453 to reach other transit-free networks but at the same time they are transiting AS6453 to reach AS6453 downstreams.
Lack of data: The Internet itself was new back in the 1990s, BGP was new and so was the concept of route collectors. So what is visible did exist but there could be more that was not visible at the public collectors.

History of transit-free / tier 1 networks

Based on BGP routing table data + Wikipedia Tier 1 history page + other publicly available data

Month/Year	Data Source	Known transit-free networks	Comments
Before-Nov 1997	Randy Bush	AS701 (UUnet), AS174 (PSI), AS1 (BBN) & AS1239 (Sprint)	Assumed to have been transit-free ASNs based on Randy’s information for further mapping. Full direct AS_PATHs between them are visible in the November 1997 Route Views dump.
Nov 1997	rviews dump	AS701 (UUnet), AS174 (PSI), AS1 (BBN), AS1239 (Sprint) + (AS3561) C&W America	AT&T AS7018/6478 was likely not tier 1 at this point as routes visible with path “701 1 6478 7018” while some direct paths indicate that AT&T was transiting from BBN AS1 to UUnet AS701. Global Crossing was not a tier 1 at this point - AS3549 had transit via AS6196 (ISI) to reach AS1239 (Sprint) while direct adjacency to PSI AS174, AS701 (UUNET) etc. Unable to establish the situation with AS3356 as it had adjancency to AS701 but no (direct/indirect) routes visible to AS174/AS1/AS1239. Verio AS2914 was not tier 1 at this point as it had transit to via ISC AS1280 to reach UUNET AS701 but does have direct adjacency to AS1/174/1239. Qwest AS209 was not Tier 1, as it had transit to reach AS701 & others. Teleglobe AS6453 was not transit-free and was using UUNET AS701 to reach PSI AS174. Tiscali / Tinet AS3257 was not a tier 1 and was using AS6453 to reach UUNET AS701. TeliaSonera International Carrier AS1299 was not a tier 1 as it was many ASNs away from AS1239/AS3541 etc using AS4200 as transit. The situation with AboveNet AS6461 is unclear as it has adjacency with 701/1239/3561 but unclear with others.
June 1998	rviews dump	AS701 (MCI UUNET), AS174 (PSI), AS1 (BBN), AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (Verio)	AS7018 (AT&T ) got adjacency with AS701, AS1238, AS1, AS3561 etc and thus likely AS7018 became transit-free around this time
Sep 1998	Wikipedia page on MCI	AS701 (MCI UUNET), AS174 (PSI), AS1 (BBN), AS1239 (Sprint) + (AS3561) C&W America	WorldCom (which acquired UUNET earlier) merged with MCI creating MCI WorldCom & thus AS701 becomes MCI UUNET
May 2000	Lightreading	AS701 (MCI UUNET), AS174 (PSI), AS1 (BBN), AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (NTT)	NTT buys Verio, and thus AS2914 becomes NTT from this point onwards
Jun 2000	Verizon	AS701 (MCI UUNET), AS174 (PSI), AS1 (Genuity), AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (NTT)	GTE Corp. merged with Bell Atlantic Corp to form Verizon. They barred Verizon from offering long-distance voice/data services in certain regions due to regulation & thus BBN IP assets including AS1 went to Genuity.
Aug 2000	RIPE RIS RRC00 dump	AS701 (MCI UUNET), AS174 (PSI), AS1 (Genuity), AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (NTT) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS6461 (Metromedia)	AS3549 (Global Crossing) establishes direct adjacency to AS701 and AS3356 (Level 3) also establishes adjacency to AS1239 and seems to be directly adjacent to all known transit-free networks. Thus Both AS3549 and AS3356 likely became tier 1 around this point. Also, AS6461 (Metromedia) got adjacency to AS1239 and had the rest already by now. So AS6461 (Metromedia) became transit-free around this point
Sept 2001	route-views dump + RIPE RIS RRC03	AS701 (MCI UUNET), AS174 (PSI), AS1 (Genuity), AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (NTT) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS6461 (Metromedia) + AS6453 (Teleglobe)	Teleglobe AS6453 connects to Savvis AS3561. Got other adjacencies also within this year and thus likely AS6453 becomes transit-free network
April 2002	Cogent press release	AS701 (MCI UUNET), AS174 (Cogent), AS1 (Genuity), AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (NTT) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS6461 (Metromedia) + AS6453 (Teleglobe)	Cogent acquires PSI AS174 and thus AS174 becomes Cogent from this point onwards. Besides getting the network they also get C-root DNS server
Jan 2003	Budapest University of Technology and Economics	AS701 (MCI UUNET), AS174 (Cogent), AS1 (Level 3), AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (NTT) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS6461 (Metromedia) + AS6453 (Teleglobe)	Level 3 acquires Genuity which had assets from previous spun off in 2000s when BBN merged with Bell Atlantic to form Verizon. AS1 was included in the assets and thus AS1 goes to Level 3
May 2003	RIPE RIS & RRC03 dumps	AS701 (MCI UUNET), AS174 (Cogent), AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (NTT) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS6461 (Metromedia) + AS6453 (Teleglobe)	AS1 loses adjacency with most of transit-free ASNs including AS701, AS6453 etc & losses its transit-free network status
Sep 2003	The New York times	AS701 (MCI UUNET) + AS174 (Cogent) + AS1239 (Sprint) + AS3561 (C&W America) + AS7018 (AT&T) + AS2914 (NTT) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS6461 (AboveNet) + AS6453 (Teleglobe)	AS6461 emerged from bankruptcy and becomes AboveNet
Jan 2004	The New York times	AS701 (MCI UUNET) + AS174 (Cogent), AS1239 (Sprint) + AS3561 (Savvis) + AS7018 (AT&T) + AS2914 (NTT) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS6461 (AboveNet) + AS6453 (Teleglobe)	Savvis buys C&W America, and thus AS3561 becomes Savvis
June 2004	rviews dump	AS701 (MCI UUNET), AS174 (Cogent), AS1239 (Sprint) + AS3561 (Savvis) + AS7018 (AT&T) + AS2914 (NTT) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS6461 (AboveNet) + AS6453 (Teleglobe)	AS1299 (TeliaSonera International Carrier) has adjacency to all known transit-free networks by this point although it was added to the Wikipedia list a few years later
Nov 2005	Wikipedia	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (MCI WorldCom) + AS2914 (NTT) + AS209 (Qwest) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Teleglobe)	First list out on Wikipedia + existing known transit-free networks via BGP table guess work
Jan 2006	Wikipedia page on MCI	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (Qwest) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Teleglobe) + AS1299 (Telia)	Verizon acquires MCI and thus AS701 becomes Verizon along with other AS702, 703, 704 etc
Feb 2006	SEC filing	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (Qwest) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (VSNL)	VSNL acquires AS6453 (Teleglobe) and thus from this point onwards AS6453 becomes VSNL
May 2006	Wikipedia	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (Qwest) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (VSNL) + AS174 (Cogent) + AS1668 (AOL/ATDN)	AS174 (Cogent) reappears in here as it’s there in the Wikipedia list + AS1668 (AOL / ATDN) appears in the list
Dec 2007	Economic Times, India	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (Qwest) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Tata Comm) + AS174 (Cogent)	Tata Group renames VSNL to Tata Communications and thus AS6453 becomes Tata Communications from this point onwards. This single entity gets assets of VSNL + Teleglobe + Tyco etc under the brand name Tata Comm
May 2008	Wikipedia	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (Qwest) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia)	AS1299 (Telia) added to the list as per Wikipedia
Apr 2010	Wikipedia	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (Qwest) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet)	AS3257 (Tinet) added to Wikipedia list
Nov 2010	CenturyLink press release	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet)	CenturyLink merges with Qwest & AS209 becomes CenturyLink
Feb 2011	Wikipedia	AS7018 (AT&T) + AS3539 (Global Crossing) + AS3356 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet) + AS3320 (DTAG)	AS3320 (DTAG) added to the Wikipedia list
Apr 2011	Techcrunch	AS7018 (AT&T) + AS3356/AS3539 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS3561 (Savvis) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet) + AS3320 (DTAG)	Level 3 acquires Global Crossing and thus AS3549 starts fading behind AS3356 from this point onwards
Jul 2011	CenturyLink press release	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (AboveNet) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet) + AS3320 (DTAG)	CenturyLink merges with Savvis & thus AS3561 moves to CenturyLink
Mar 2012	SEC filing	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet) + AS3320 (DTAG)	Zayo acquires Abovenet and thus AS6461 moves to Zayo
May 2012	Wikipedia	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet) + AS3320 (DTAG) + AS6762 (Telecom Italia)	AS6762 (Telecom Italia added to Wikipedia list)
Jun 2012	Wikipedia	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm)	AS2828 (XO Communications) is listed as tier 1 network this time onwards
Aug 2012	Wikipedia	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm) + AS5511 (France Telecom)	Wikipedia reports AS5511 (France Telecom) became transit-free around this time based on Renesys & CAIDA
Sept 2012	RIPE RIS RRC00 + Wikipedia	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (Tinet) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm) + AS5511 (France Telecom) + AS12956 (Telefonia)	AS12956 (Telefonia) got adjacency to all known transit-free networks earlier in the year around Feb 2012 & by Sep 2012 it is listed in Wikipedia list as transit-free network
Apr 2013	Telecom Ramblings	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm) + AS5511 (France Telecom) + AS12956 (Telefonia)	GTT Buys Inteliquent’s Data Network and thus AS3257 moves to GTT
Jul 2013	Lightreading	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1668 (AOL/ATDN) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm) + AS5511 (Orange) + AS12956 (Telefonia)	France telecom renames to Orange and thus AS5511 becomes Orange from this point onwards
Feb 2014	Wikipedia	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm) + AS5511 (Orange) + AS12956 (Telefonia)	AS1668 (AOL/ATDN) was removed from the list as it started taking transit via AS3356 (Level 3) and AS1273 (Sprint)
Apr 2016	Wikipedia	AS7018 (AT&T) + AS3356/AS3539/AS3561 (Level 3) + AS701 (Verizon) + AS2914 (NTT) + AS209 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm) + AS5511 (Orange) + AS12956 (Telefonia) + AS286 (KPN)	AS286 (KPN) appears in the Wikipedia list as tier 1 network
October 2016	CenturyLink press release	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS209/AS3356/AS3539/AS3561 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm) + AS5511 (Orange) + AS12956 (Telefonia) + AS286 (KPN)	CenturyLink acquires Level 3 and thus AS3356 moves to CenturyLink. While CenturyLink acquired it, the Level 3 IP network was huge and over time AS3356 remained, while the other ASNs went behind it
Nov 2016	Wikipedia	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS209/AS3356/AS3539/AS3561 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS2828 (XO Comm) + AS5511 (Orange) + AS12956 (Telefonia) + AS286 (KPN) + AS6830 (Liberty Global)	AS6830 (Liberty Global) gets added to the list on Wikipedia
Feb 2017	Lightreading	AS7018 (AT&T) + AS701/AS2828 (Verizon) + AS2914 (NTT) + AS209/AS3356/AS3539/AS3561 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS174 (Cogent) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia) + AS286 (KPN) + AS6830 (Liberty Global)	AS701 (Verizon) buys XO Communications and slowly integrates it in it’s backbone. Thus AS2828 is no longer transit-free.
April 2017	Wikipedia	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS209/AS3356/AS3539/AS3561 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia) + AS286 (KPN) + AS6830 (Liberty Global)	AS174 (Cogent) is removed from the Wikipedia tier 1 list due to a lack of IPv6 routes with AS6939 (HE)
Sep 2017	Wikipedia + route-views2 + rrc03 dump	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS209/AS3356/AS3539/AS3561 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia) + AS286 (KPN) + AS6830 (Liberty Global) + AS3491 (PCCW)	AS3491 (PCCW) established direct adjacency to AS1299 (TeliaSonera International) earlier in the year. Was already connected to others by now. By Sep 2017 it was added to the list of transit-free networks by Wikipedia
Dec 2019	KPN AS286 website	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS209/AS3356/AS3539/AS3561 (CenturyLink) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia) + AS6830 (Liberty Global) + AS3491 (PCCW)	(AS286) KPN was taken over by AS3257 (GTT), and thus AS286 was integrated into it and is removed from the list of tier 1 networks
Sep 2020	CenturyLink press release	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS3356 (Lumen) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS1299 (Telia) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia) + AS6830 (Liberty Global) + AS3491 (PCCW)	CenturyLink rebranded as Lumen and thus AS3356 becomes Lumen from this point onwards and most of its other ASNs had slowly moved behind AS3356 by this time.
Jan 2022	Telecom TV	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS3356 (Lumen) + AS1239 (Sprint) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS1299 (Arelion) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia) + AS6830 (Liberty Global) + AS3491 (PCCW)	Telia becomes Arelion and thus AS1299 becomes Arelion
May 2023	SEC filing	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS3356 (Lumen) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS1299 (Arelion) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia) + AS6830 (Liberty Global) + AS3491 (PCCW)	Cogent acquires Sprint’s IP network as Sprint’s mobile network goes to T-Mobile and and shortly afterwards AS1239 goes behind AS174
Nov 2023	Colt press release	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT) + AS3356 (Lumen + Colt jointly maintaining it) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS1299 (Arelion) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia) + AS6830 (Liberty Global) + AS3491 (PCCW)	Colt buys Lumen’s EMEA network and from this point onwards AS3356 becomes a split network jointly owned by Lumen (in North America, Singapore etc) and Colt in Europe, the Middle East and Africa
Aug 2026	Wikipedia page on Tier 1 networks	AS7018 (AT&T) + AS701 (Verizon) + AS2914 (NTT), AS3356 (Lumen + Colt) + AS6461 (Zayo) + AS6453 (Tata Comm) + AS1299 (Arelion) + AS3257 (GTT) + AS3320 (DTAG) + AS6762 (Telecom Italia) + AS5511 (Orange) + AS12956 (Telefonia/Telxius) + AS6830 (Liberty Global) + AS3491 (PCCW)	Current situation! :-)

Notes on AS174 / Cogent / PSI

While Randy’s email listed PSI (AS174) which later became Cogent as transit-free from the early days, I do not see that from Wikipedia’s first list in 2005.
It was added as Tier 1 in the May 2006 list and was repeatedly added and removed due to their declared paid peering agreement with AS3356 (Level 3). I considered it transit-free until the 2017 change because I am excluding “paid peering” logic for any given ASN. (Just looking at whether transit exists or not).
There is a genuine possibility of AS174 being transit-free earlier and not later for a few years as they expanded to Asia. This slide from Gaurab Upadhaya (PCH) shows AS_PATH: 1239 2914 174 on slide 7 showing that there was a transit via AS2914 (NTT).
Since its status is not clear, I will just follow the Wikipedia list. It might be worth exploring through each available collector, including those in Asia, if this situation changed at some other time.
It was removed from the list in April 2017 due to a lack of IPv6 peering with AS6939 (Hurricane Electric). My table is reflecting the same. Since I am not a neutral person on this, I will avoid any further comment and simply follow what others have documented on the Wikipedia page.

Interesting history Verio -> NTT

Verio had an interesting history of being a web hosting provider in 1996 & was acquired by NTT in 2000. Wikipedia has a fascinating history of the transaction and it is worth reading:

By the year 2000, Verio had purchased 55 ISP/Hosting companies, most in the U.S. but some in Europe. During this time Verio went public on the NASDAQ, trading under the symbol VRIO, with a market value exceeding $1 billion. Shortly after the IPO, in early 2000, Verio was sold to NTT at a per-share price of $73, a total cost slightly exceeding $5 billion. Because NTT was a 53% Japanese government-owned company, foreigners were not allowed to own NTT stock, according to Japanese law at the time,[2] and therefore the buy-out was a 100% cash deal, making it one of the highest grossing deals of the dotcom era. The United States Congress held hearings over the transaction to ensure it did not violate national security concerns. The Justice Department and the Federal Bureau of Investigation expressed concern that the Japanese government, which owned 53 percent of NTT at the time, could gain access to classified information should the U.S. government use Verio’s network to tap Internet communications during an investigation. To placate these concerns, NTT agreed to form a separate division within the company staffed only by U.S. citizens to handle any work in support of government investigations. As a result, the Committee on Foreign Investment in the United States recommended that President Clinton allow the $5.5 billion purchase to proceed. The deal also prompted scrutiny of Japan’s openness to foreign telecom competitors.

Shortly after the announced deal, the NASDAQ stock market crashed in the spring of 2000 in the dot-com bubble burst. The agreed price of $73 remained and NTT and Verio completed the transaction by the fall of 2000.

Eventually, it became NTT AS2914 as we know it today and the hosting part was sold off to Endurance in 2015.

Transit but cannot transit!

Another important thing to mention here is that - there are cases of large networks that have transit but are not completely open to receiving any traffic from transit. When Wikipedia states that “X buys transit from Y” - one may or may not be able to reach X via Y due to the way they control the announcements.

Here’s an example:

Let’s look for a random IP (1.71.103.0/24) from AS4809 (China Telecom) in AS1299 (Arelion) looking glass:

Notice the use of:

1299:2009 (Do NOT announce to ANY peer in Europe)
1299:5009 (Do NOT announce to ANY peer in North America)
1299:7009 (Do NOT announce to ANY peer in Asia)

Thus even though it’s tagged as 1299:37000 (APAC customer) - most of the peers of AS1299 will not get this route because AS4809 is maintaining a direct relationship with those networks & seems to be using AS1299 for very selective announcements. It’s hard to locate all these historical cases but in the present scenario consider these networks to not have full transit. They have their own peerings with many other transit-free networks, along with AS_PATH filters dropping those tier1s from other paths (i.e peer lock).

Feedback & updates

I have very likely missed some data here. Feedback via the comments below or through the direct contact page is welcome. I will update the page accordingly.

Use of VRF in dual WAN setup

Tue, 04 Aug 2026 02:28:23 +0530

Over the weekend I migrated my home router from multiple routing tables to a VRF-based design, placing each WAN uplink into its own VRF. While multiple routing tables worked for basic policy routing, they have some limitations and that led to several edge cases that became increasingly difficult to work around.

Issues with the setup:

If the active default route pointed to ISP1, traffic arriving on ISP1 naturally returned via ISP1. However, traffic terminating on the router and arriving via ISP2 would also be replied to through ISP1, since both uplinks still shared the same routing domain. This did not cause issues for traffic on devices below the router but was bad for traffic terminating on the router interface itself.
Due to the above reason, I recently lost access to my home router while I was out of the country because ISP1 had a partial outage (their transit went down, peering stayed up) & due to distributed tooling, the auto switch trigger could not happen either to take care of it. Packets from ISP 2 were being returned via the ISP1 route & thus blackholed.
I have a special case where I want most of the devices on a redundant setup but some devices (containers) on specific ISP only. These are measurement containers running blackbox exporter behind a specific ISP as well as RIPE Atlas. I don’t want these to switch over for accuracy of measurement. Without VRF it was ugly config-wise, as ISP1 failure will lead to ISP2 routing even when the specific routing table did not have that route.

Old Setup

My old setup was running multiple routing table pairs:

ISP1 only & ISP1 as primary (ISP2 as secondary)
ISP2 only & ISP2 as primary (ISP1 as secondary)

All WAN interfaces, LAN interfaces and routing tables were in the same single “Main” VRF. This by default takes care of full outage events, fibre cuts, etc. For packet loss-based switchover, I have set up Prometheus + Semaphore as covered in this post last year.

Understanding VRF

VRF is Virtual Routing and Forwarding. It allows “virtual partitioning” of the router in L3 terms. Different physical & VLAN interfaces can be part of different VRFs, and they have no connectivity with each other at all unless routes are leaking between the VRFs. Thus a router with VRF - Main, ISP1 & ISP2 acts like three separate routers. It’s like a VLAN for layer3. In Mikrotik it can get confusing as one can create multiple isolated routing tables in the same Main VRF however since WAN interfaces are also in the same main VRF, the default route from the main table takes out if a next-hop becomes unavailable on an interface.

New Setup

With three VRFs in place (default + 2 other), I have isolated each uplink in its own VRF. Also, removed the ISP1-only & ISP2-only routing tables, as VRF creates its own isolated table. Most of the policy-routing logic remains unchanged. Mangle does the marking of routing to steer traffic & mangle rules for an address list where IPs are enabled or disabled from external tooling based on how I want to route the traffic. Every WAN-facing route now belongs to the corresponding VRF instead of the Main routing domain. And since these act like separate routers, I had to add a path for the return route as well. So e.g if 172.16.0.0/24 on say VLAN10 (in Main VRF) communicates via VRF-ISP2 via mangle-marked routing, the VRF-ISP2 routing table needed an interface route to return the traffic. For VLANs specific to these ISPs, I added them in the respective ISP VRF. These VLANs are also trunked to my home server, where they map directly onto Docker networks, allowing selected containers to live natively inside a specific ISP VRF

In hindsight, I should have switched to VRFs much earlier. Multiple routing tables worked for a long time, but they started showing their limits as the network became more complex. VRFs make the separation explicit, and the router behaves much closer to how I actually expect it to.

Beyond 5G: The Two Biggest Challenges for India's Internet

Fri, 03 Jul 2026 01:20:52 +0530

The recent debate around Airtel’s priority 5G generated plenty of noise. Personally, I’m not a fan of it either but Airtel and Jio are private companies. Their job is to maximise shareholder value, not to maintain market competition. The real question is why competition remains so limited. And by competition, I don’t mean another mobile operator. I mean fixed-line broadband.

Two long-standing problems continue to hold India’s internet ecosystem back.

1. Every ISP builds its own last mile

In the early days of FTTH this was understandable. In 2026, it feels increasingly wasteful.

At my home in Haryana, around a dozen FTTH providers are available. The pole outside carries 14 fibre cables from different operators — all solving the same problem independently. The last mile isn’t where ISPs differentiate. Fibre from ISP A isn’t inherently “faster” than fibre from ISP B. The real differentiation is in the network behind that fibre capacity, routing, peering, resilience and customer support. Yet every provider continues deploying duplicate infrastructure. Even Airtel and Jio increasingly push Fixed Wireless Access because expanding FTTH remains expensive.

Instead of parallel deployments, we should encourage neutral last-mile infrastructure. A small number of regulated fibre providers could deploy high core count cables, while ISPs lease strands and compete on service quality rather than digging the same streets repeatedly. This will also help in fixing streets which look quite ugly with excess fibre routing in all directions.

Earlier this year, Stefan Schüller published an excellent article, The Free Market Lie: Why Switzerland Has 25 Gbit Internet and America Doesn’t comparing broadband infrastructure in Switzerland, the US and Germany. It’s well-researched and provides useful context for many of the points discussed here.I do disagree with one aspect of the article - its comparison of Passive Optical Networks (PON) and Active Optical Networks (AON). Contention exists throughout the internet, from access networks to the application layer. Networks are engineered with reasonable headroom, but oversubscription is a fundamental part of how the internet operates. So I don’t believe PON is inherently the limitation it’s sometimes portrayed to be. That said, it’s still an insightful read about the topic.

India has fibre in many of the urban areas from homes to offices, but we don’t make full use of it. The biggest reason isn’t technology, it’s regulation and liability. Because ISPs must maintain subscriber and CGNAT logs for Law Enforcement Agencies (LEAs), sharing a normal retail broadband connection publicly becomes legally risky. In practice, the compliant solution is managed Wi-Fi provided by a licensed ISP.

That’s why places like gyms, cafes or small businesses often avoid offering Wi-Fi even when a fibre connection is sitting right there for their own usage.

PM-WANI attempted to address this, but its focus evolved around commercial hotspot operators and it got projected more as a “make side money” scheme instead. It has many issues, some of which were covered by Mr Parag Kar in this video in Hindi last month. The bigger opportunity is making secure Wi-Fi sharing simple, not selling.

Centralised wifi authentication & logging

Imagine a nationwide authentication platform:

One-time user verification (both for the end user and for the person offering the Wi-Fi hotspot)
Certified Wi-Fi hardware that securely streams required logs to central infra (host gets a unique token when setting it up & the same is used to stream logs)
Seamless automatic connection across compliant hotspots with the end-user device holding a certificate to prove identity
No traffic tunnelling or unnecessary complexity

Homes, offices, hotels, cafes, gyms and other venues could legally offer Wi-Fi while preserving the audit trail required by law enforcement by just getting the compliant hardware and signing up with it.

The next leap probably isn’t faster 5G or 6G. It’s building shared infrastructure where duplication adds little value and removing regulatory friction that prevents existing connectivity from being used more effectively.

RFC8950: Announce IPv4 with an IPv6 next-hop

Mon, 22 Jun 2026 14:04:09 +0530

For the last few days I have been playing with RFC8950 setup, which allows routing of IPv4 on top of IPv6. While logically it’s quite simple, it has a very powerful application towards making “IPv4 an optional” feature riding on top of an IPv6 network (but without any tunnels!)

Some of the interesting use cases:

A large network can run IPv6 only everywhere on its core and leave IPv4 only on edge devices. This can work without customers doing any tweaking at their end. They still do things the old way but Backbone can run IPv6 only at its core.
An IP transit backbone can run IPv6 only while exchanging IPv4 routes with its customers/peers/upstream over IPv6 sessions. So IPv4 as a “feature” stays but the network in itself is IPv6 only without the overhead of IPv4 interface config, BGP session config, firewall config, etc.
An IXP can offer services without IPv4 essentually by using a (unique) IPv6 at the IX LAN & running route server to exchange both IPv4 & IPv6 routes over the IPv6 only session.

Thus what 464XLAT did to the mobile networks - this has the potential to do that for the backbone networks.

How does it work?

Multi-protocol BGP (MP-BGP) supports announcement of IPv4 routes over IPv6 sessions with IPv4 next-hops and IPv6 routes over IPv4 sessions with IPv6 next-hops. RFC8950 extends this feature for IPv4 to have IPv6 next-hops. So for IPv4 address, next hop is IPv6, IPv6 reach works via usual NDP and IPv4 rides on ethernet frames to the same MAC as discovered by the NDP. Technically RFC8950 only impacts control plane decisions by using IPv6 as the control plane but keeps the data plane as it is. IPv4 stays IPv4 & goes as IPv4 over the wire unlike say some tunnel or 464xLAT etc.

Hardware support

As of now it’s quite widely supported by a lot of routing platforms like Juniper JunOS, Cisco IOS XR, Nokia SR OS, Mikrotik, Huawei, BIRD, FRR etc. So, it is very likely supported if you are running a recent version of the OS.

Demo lab

Here’s a fun lab with two customer routers: C1 & C2 connected via provider routers R1-R2-R3 in Container Lab. All routers here are running open source FRR.

C1 is IPv6 only on the WAN side facing R1. It originates 10.0.0.0/24 to R1 over an IPv6 session.
R1 & R2 are IPv6 only with no IPv4 anywhere on their interfaces (but carry IPv4 routes in their table)
R3 has IPv6 with R2 but just IPv4 only with C2. So C2 is like a typical enterprise user who lacks IPv6 deployment and still wants IPv4 only to work.
R3 has 100.64.0.0/24 and originates it in the IPv4 table (to IPv6 adjacency - R2)
C2 sits on 100.64.0.2/30 given by R3 and communicate in IPv4 only.
C1 and C2 can reach each other over this network with parts being IPv6 only without any tunnel.

topology.yml

name: rfc8950
topology:
nodes:
R1:
kind: linux
image: frr-with-bgpd
binds:
- configs/R1.conf:/etc/frr/frr.conf
R2:
kind: linux
image: frr-with-bgpd
binds:
- configs/R2.conf:/etc/frr/frr.conf
R3:
kind: linux
image: frr-with-bgpd
binds:
- configs/R3.conf:/etc/frr/frr.conf
C1:
kind: linux
image: frr-with-bgpd
binds:
- configs/C1.conf:/etc/frr/frr.conf
C2:
kind: linux
image: frr-with-bgpd
binds:
- configs/C2.conf:/etc/frr/frr.conf
links:
- endpoints: ["C1:eth1", "R1:eth1"]
- endpoints: ["R1:eth2", "R2:eth1"]
- endpoints: ["R2:eth2", "R3:eth1"]
- endpoints: ["R3:eth2", "C2:eth1"]

> clab deploy -t topology.yml
16:37:08 INFO Containerlab started version=0.76.1
16:37:08 INFO Parsing & checking topology file=topology.yml
16:37:09 INFO Creating lab directory path=~/ipv4-over-ipv6/clab-rfc8950
16:37:09 INFO Creating container name=C2
16:37:09 INFO Creating container name=C1
16:37:09 INFO Creating container name=R1
16:37:09 INFO Creating container name=R2
16:37:09 INFO Creating container name=R3
16:37:09 INFO Created link: R3:eth2 ▪┄┄▪ C2:eth1
16:37:09 INFO Created link: C1:eth1 ▪┄┄▪ R1:eth1
16:37:09 INFO Created link: R1:eth2 ▪┄┄▪ R2:eth1
16:37:09 INFO Created link: R2:eth2 ▪┄┄▪ R3:eth1
16:37:09 INFO Adding host entries path=/etc/hosts
16:37:09 INFO Adding SSH config for nodes path=/etc/ssh/ssh_config.d/clab-rfc8950.conf
╭─────────────────┬───────────────┬─────────┬───────────────────╮
│ Name │ Kind/Image │ State │ IPv4/6 Address │
├─────────────────┼───────────────┼─────────┼───────────────────┤
│ clab-rfc8950-C1 │ linux │ running │ 172.20.20.6 │
│ │ frr-with-bgpd │ │ 3fff:172:20:20::6 │
├─────────────────┼───────────────┼─────────┼───────────────────┤
│ clab-rfc8950-C2 │ linux │ running │ 172.20.20.4 │
│ │ frr-with-bgpd │ │ 3fff:172:20:20::4 │
├─────────────────┼───────────────┼─────────┼───────────────────┤
│ clab-rfc8950-R1 │ linux │ running │ 172.20.20.2 │
│ │ frr-with-bgpd │ │ 3fff:172:20:20::2 │
├─────────────────┼───────────────┼─────────┼───────────────────┤
│ clab-rfc8950-R2 │ linux │ running │ 172.20.20.8 │
│ │ frr-with-bgpd │ │ 3fff:172:20:20::8 │
├─────────────────┼───────────────┼─────────┼───────────────────┤
│ clab-rfc8950-R3 │ linux │ running │ 172.20.20.5 │
│ │ frr-with-bgpd │ │ 3fff:172:20:20::5 │
╰─────────────────┴───────────────┴─────────┴───────────────────╯

Checking C1 for route to C2

C1# show ip bgp 100.64.0.2
BGP routing table entry for 100.64.0.0/24, version 2
Paths: (1 available, best #1, table default)
Advertised to non peer-group peers:
2001:db8:0:1::1
65501 65502 65503
2001:db8:0:1::1 from 2001:db8:0:1::1 (172.20.20.2)
(fe80::a8c1:abff:fe60:5467) (used)
Origin IGP, valid, external, best (First path received)
Last update: Mon Jun 22 18:57:38 2026
C1#

C1 -> C2 trace

C1# traceroute 100.64.0.2
traceroute to 100.64.0.2 (100.64.0.2), 30 hops max, 46 byte packets
1 clab-rfc8950-R1.clab (172.20.20.2) 0.004 ms 0.004 ms 0.002 ms
2 clab-rfc8950-R2.clab (172.20.20.8) 0.002 ms 0.003 ms 0.003 ms
3 clab-rfc8950-R3.clab (172.20.20.5) 0.002 ms 0.003 ms 0.003 ms
4 100.64.0.2 (100.64.0.2) 0.002 ms 0.006 ms 0.003 ms
C1#

Note: It took IPv4 from the eth0 management interfaces as deployed by Container Lab for TTL exceeded used in the trace.

If I check R1, it has only IPv6 sessions but no IPv6 prefixes

R1# sh bgp ipv6 summary
IPv6 Unicast Summary (VRF default):
BGP router identifier 172.20.20.2, local AS number 65501 vrf-id 0
BGP table version 0
RIB entries 0, using 0 bytes of memory
Peers 2, using 1434 KiB of memory
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd PfxSnt Desc
2001:db8:0:1::2 4 64512 151 141 0 0 0 00:39:19 0 0 N/A
2001:db8:0:2::2 4 65502 65 64 0 0 0 00:36:46 0 0 N/A
Total number of neighbors 2
R1#

Checking for IPv4 routes on R1

R1# sh bgp ipv4
BGP table version is 4, local router ID is 172.20.20.2, vrf id 0
Default local pref 100, local AS 65501
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
i internal, r RIB-failure, S Stale, R Removed
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found
Network Next Hop Metric LocPrf Weight Path
*> 10.0.0.0/24 fe80::a8c1:abff:fe44:afb8
0 0 64512 i
*> 100.64.0.0/24 fe80::a8c1:abff:fed8:5eb5
0 65502 65503 i
Displayed 2 routes and 2 total paths
R1#

Router Configs

Router	Link
R1	Click here
R2	Click here
R3	Click here
C1	Click here
C2	Click here

Telegram BGP hijack due to weird blackholing config

Thu, 18 Jun 2026 05:22:47 +0530

Since the Telegram’s prefix hijack (4 days ago on 17-Jun-2026) by RCom, there is visible noise in the BGP routing table from multiple other Indian ISPs, including Tata Teleservices (AS45820) / Lightstorm (AS135709/AS152144), etc. mostly in IPv6.

Let’s look at 2a0a:f280::/48 (Live lookup here)

Notice all the ASNs here largely seem to be downstreams or peers of AS45820 in India, no major large peer or upstream that would take this announcement outside of India. Similarly, take the case of aggregate - 2a0a:f280::/32. This has TTSL (AS45820) as well as Lightstrom (AS152144/135709) originating these prefixes to smaller peers.

Analysis: Possible reason and impact of this behaviour

Today I tested some config in lab to see why this could be happening. Here’s what I strongly feel is happening (quite sure, unless someone has a better explanation):

Indian Govt. has asked ISPs to block Telegram and unlike past blocks mostly at the DNS layer, ISPs have been asked to drop IPs as well.
Technically in these cases ISPs could simply add a blackhole route and that would drop traffic going towards these prefixes (from their customers) and that would not be visible at BGP (control plane) but only in traceroutes (data plane). If any of these customers had a full routing feed from those respective ISPs, they would keep on learning Telegram’s route with the correct/expected AS_PATH and expected origin AS.
Very likely these players had blackhole config setup for the DDoS protection and ended up using the same i.e they are treating Telegram’s IPv6 prefixes like they would treat their own when under attack.
Also, blackholing is a common practice during DDoS attacks. Imagine a network with a 100G uplink gets hit by a 400G volumetric attack. It will take up all the link bandwidth and thus in these cases ISPs blackhole their own (or downstream) IPs (often small - single /32s or a few) and they signal this to their BGP adjacencies as well. There is a standard BGP blackhole community: 65535:666 as defined in RFC7999.
To make things worse - RPKI RoV usually excludes RFC7999, so this would not be easy to catch with RPKI as well for networks that support blackholing with these ASNs.
Normally ISPs are not supposed to blackhole and BGP announces the blackhole unless it’s their own or downstream prefix where they have an understanding with downstream to do it (or downstream triggers it) to protect other IPs which are not under the attack. It’s incorrect for some Indian ISPs to be blackholing and BGP originating the blackhole to their adjacencies. The effect of this so far has been harmless because these are just going to the customers, however it’s not ideal.
It can still create unexpected complications. Lightstorm for instance also has customers outside of India (Nepal) where Telegram is not blocked. By “BGP originating” blackhole, they would end up in creating a short AS_PATH (and hence usually preferred unless their customer de-pref it) by their multi-homed users steering traffic towards them and then drop it (as blackhole exists). Also, while they are originating /32, but if they originate a smaller more specific one that would also end up in steering traffic which is not needed otherwise.

Note

Check out this discussion on X with my friend Bryton Herdes here
Also, Bryton / Cloudflare talk at Chicago NOG last month on how remote-triggered blackhole can bypass RoV - False immunity: long prefixes that bypass ROV: Slide and Video

Disclaimer: This is my personal blog, and hence, posts made here are in my personal capacity. These do not represent the views of my employer.

Telegram prefixes hijack by Rcom AS18101

Wed, 17 Jun 2026 01:14:03 +0530

The last few hours were quite noisy with a lot of discussion around Telegram block and Rcom’s hijack of Telegram prefixes. For those who may not know, Telegram has been blocked in India till 22 June 2026 (news here). The justification has been to avoid paper leak over the telegram. Anyways, I am not going into whether the block is good or bad as it can be part of an endless discussion depending on how one views it. Maybe some separate time but let’s look at the BGP routing side of things.

ISPs went for blocking it in all possible ways - from the usual DNS layer to block resolution of Telegram to also blackhole its prefixes. Telegram is a special case as they have their own ASN, IP prefixes etc making it easy to block them at the routing layer directly. I saw this on Airtel, where traffic seems to be dropping at their routers.

mtr -wby0 -4 web.telegram.org
Start: 2026-06-17T01:20:46+0530
HOST: desktop Loss% Snt Last Avg Best Wrst StDev
1. AS??? _gateway (172.16.0.1) 0.0% 10 0.2 0.2 0.2 0.2 0.0
2. AS??? 10.240.9.204 0.0% 10 4.6 7.4 4.2 25.7 6.5
3. AS??? 172.31.0.154 0.0% 10 8.1 13.5 6.4 26.4 6.2
4. AS9498 169.168.18.125.dhcp.anaronline.net (125.18.168.169) 0.0% 10 5.2 6.8 4.9 13.6 2.6
5. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0

At 11:58 PM - Telegram CEO Pavel Durov tweeted and accused Reliance (technically Rcom and not Jio) for outage of Telegram outside of India by BGP hijacking.

Indian telecom Reliance is sabotaging access to Telegram for millions of users OUTSIDE India (including the UAE) via a rogue method called BGP hijacking.

The sabotage seems intentional, as Reliance has ignored multiple reports.

This may be part of a competitive war, as…
— Pavel Durov (@durov) June 16, 2026

Technically it’s true that Rcom AS18101 has hijacked Telegram’s prefixes. Take e.g 91.105.192.0/23 - AS18101 started originating this prefix at 16:14:19 GMT / 21:44:19 IST on 16 June as visible from many RIPE RIS collectors including RIPE RIS RRC01 in London. It’s bad and they should not have done it. With that being said I strongly feel it’s a “fat finger mistake”.

https://bgp.he.net/AS18101#_prefixes

Likely was not intentional and here’s why:

Rcom would have a bunch of their own super-set prefixes in their router with BGP communities acting as the pull-up route. It’s quite common to blackhole one’s own superset prefix and then use slices of it on interfaces, static routes, etc. Likely they would have copy-pasted the existing prefix list and added Telegram prefixes to the list and boom!

The impact here varies as Telegram itself has multiple ASNs - AS62041, AS62014, AS59930, AS44907 and AS211157. These prefixes have different set of upstreams. The ones which are learning hijacked prefixes directly from Telegram, they normally won’t have an impact.

If it was actually intentional, one would have kept the origin ASN the same and faked Telegram AS211157 behind AS18101 or so. That would not trigger RPKI RoV filtering across a larger transit-free tier-1 layer as well as various backbones and IXPs.

Technically what failed?

Well, besides the fat finger mistake by Rcom AS18101 here, what failed was the lack of RPKI RoV by their upstreams - FLAG AS15412 and Tata Comm AS4755. Since these prefixes were signed, an RPKI RoV deployment would have easily filtered these and restricted routes within AS18101 and any of its downstream that were not filtering.

Update: 17 June 2026 - 01:47 IST

AS I am writing this, I see the prefix has gone behind FLAG AS15412. Either they have filtered it or Rcom has stopped announcing it and it’s slowly fading away from the global table. Write now, the prefix is visible only behind AS4755 and that too only in India. That’s for some of the prefixes, some are still visible.

Source

Disclaimer: This is my personal blog, and hence, posts made here are in my personal capacity. These do not represent the views of my employer.

Mapping major CDNs across the globe

Sat, 06 Jun 2026 18:47:04 +0530

In Dec 2022 I mapped popular CDNs like Google GGC, Facebook FNA, Akamai, etc. across Indian networks (post here). Since it’s been close to 4 years now, I wanted to do that again. This time, I decided to expand the scope to the entire globe instead of just the Indian networks.

Mapping logic

The logic here is simple: get unique prefixes from the global routing table, break them into /24s, uniquely sort out /24s, and this gives all routed IP addresses in the world in batches of /24s (256 IPs). Next, identify the ones with an open port 443 and then connect to each of them to figure out the TLS certificate common name. Next, map those IP addresses to AS numbers, locations, etc., using Maxmind’s GeoIP Lite databases. Essentially, I am using the BGP routing table instead of scanning the entire 0.0.0.0/0 to avoid scanning for prefixes which are not routed in the global table as well as reserved private IPs, etc.

Global stats

CDN Provider	No. unique ASNs	TLS domain	Details list URL
Google GGC	4247	googlevideo.com	csv & json
Netflix OCA	2903	oca.nflxvideo.net, assets.nflxext.com	csv & json
Facebook FNA	2548	‘fbcdn.net’, fna.whatsapp.net	csv & json
Akamai	1094	edgesuite.net, edgekey.net, akamaized.net, akamaihd.net, akamaized.net, akamai.net	csv & json
Microsoft	1655	microsoft.com	csv & json
Apple	974	apple.com & itunes.apple.com	csv & json
AWS Cloudfront	217	.amazonaws.com & .awsstatic.com	csv & json

ASN-CDN Mapping table

I have mapped ASNs with the presence of each of these CDNs like last time. This makes it easy to read the data instead of going through each of the files separately.

Full data in Google Sheet link here and raw CSV is also published here.

Notes:

Facebook FNA serving Facebook with *.fbcdn.net is almost the same if I look for *.fna.whatsapp.net.
Apple seems to have presence in over 236 locations with PCH/Woodynet for hosting their DNS over HTTPS endpoint: doh.dns.apple.com (detailed list here)
GGC has a global coverage across 215 countries across the world, followed by Facebook FNA/MNA at 204, Netflix at 181 and Akamai at 146. Out of these 172 are common countries across all these.

Map

Indian summary

GGC increased from 228 Indian ASNs to 252 between Dec 2022 and June 2026
FNA increased from 125 Indian ASNs to 199 between May 2025 and June 2026
Akamai has increased from 70 Indians to 74 Indian ASNs between Dec 2022 and June 2026

Notes on the accuracy of data

As explained in the logic - the data is based on the common name in the TLS certificates. If a server does not present the certificate or if these CDNs are using a domain which is not well known yet, I would miss that in this data. Thus as usual - what you see here does exist for sure but there could be more outside of this data.

Disclaimer: This is my personal blog, and hence, posts made here are in my personal capacity. These do not represent the views of my employer.

Measuring IPv6 deployment on Indian Govt websites

Fri, 05 Jun 2026 01:49:53 +0530

India has done well on IPv6 deployment thanks to Jio taking the lead in 2016, along with Airtel & VI aggressively catching up. Many of us in the industry are very proud of the work done by these players, resulting in over 78% deployment of IPv6 when measured by APNIC / Google, etc. Jio and Airtel numbers are as high as 95% IPv6 preferred at this point in time.

But that’s one part of the Indian metrics. IPv6 deployment, DNSSEC usage or even Internet Exchange peering - one can find very different metrics depending on how one is measuring. Both APNIC & Google rely on Google’s network and judge how (largely mobile) end users across India are connecting to their (IPv6-enabled) services, and this gives the “eyeballs” side of specs.

One key element missing in the picture is IPv6 support on the Government of India websites. I am using “Govt. of India” loosely here. I mean the Central Govt, state governments, as well as UTs and any of their departments. I have always felt that deployment is very poor, as randomly looking across the critical government websites. The service either shows no DNS AAAA record or may have a AAAA record but does not respond to port 80/443, giving an idea of a broken deployment.

Measuring deployment is hard since (as far as I know) there is no authoritative source of government domains. Likely NIXI or ERNET, etc. may hold them but then getting them may be a long process and may not work at all. Then I thought of certificate transparency logs. Most of the CAs would log TLS certificates issued in public logs. One can read these logs for *.gov.in and *.nic.in to get a list of all government. used domains under those well-known TLDs and the ones which took a TLS certificate. I did some extra checks, like verifying the A record, working website on IPv4, before looking for IPv6. That way I remove all non-existent/old/expired websites. Also, looked for and www. in some cases as TLS may be issued for both. In case of wildcard TLS certificates like *..gov.in, I looked at .gov.in again to see if the website exists on IPv4 and if yes, explored IPv6 deployment on it.

Result

There are 3068 visible domains under *.gov.in (with TLS) with a working website IPv4.
Out of these 3068, only 140 have working IPv6 on the web, i.e., an IPv6 endpoint is successfully serving the website.
There are 1856 domains under *.nic.in (with TLS) with a working website on IPv4.
Out of these 1856, only 34 have working IPv6 on the web
Majority of websites with working IPv6 are on either Cloudflare, Akamai or Amazon AWS.

Thus, out of the total websites (4924), only 174 i.e 3.5% seems to have IPv6 deployed. There are also 13 websites which have DNS AAAA record pointing these domains to an IPv6 address but they don’t reply on port 443 when queried over IPv6.

Sites’s with broken IPv6 deployment:

meity.gov.in
accounting.sthreenidhi.ap.gov.in
dgma.gov.in
digitalgujarat.gov.in
lms.istart.rajasthan.gov.in
sampada-mofpi.gov.in
serp.ap.gov.in
sthreenidhi.ap.gov.in
wdra.gov.in
www.aistic.gov.in
www.statedrugs.gov.in
mpforest.gov.in
wdra.gov.in

Raw data: Raw CSV for the lookup is here for *.gov.in and here for *.nic.in.

Airtel-Tata Comm routing is cold potato in India?

Wed, 20 May 2026 18:50:53 +0530

Over the years, I have seen a number of traces between Airtel (AS9498) and Tata Comm (AS4755) in India, which strongly suggests that routing tends to be ‘cold potato’ between them. By default, routing between large backbones that interconnect across different locations is typically ‘hot potato’ - meaning traffic exits to other ASNs as close to the source as possible

Hot potato routing is common

Hot potato routing is quite common across large backbones. Let’s analyse this concept by tracing between Arelion AS1299 and Cogent AS174 from their respective looking glasses.

Trace Arelion AS1299 (Frankfurt) -> Cogent AS174 (San Francisco) - 66.250.250.146 (ism01.sfo01.atlas.cogentco.com):

Router: ffm-b11 / Frankfurt (Equinix FR5, Kleyerstrasse)
Command: traceroute ipv4 66.250.250.146 timeout 1 source Loopback0
Tracing the route to 66.250.250.146
1 * * *
2 be7948.ccr42.fra05.atlas.cogentco.com (154.54.72.126) 1 msec 1 msec
be5484.ccr41.fra05.atlas.cogentco.com (130.117.1.1) 1 msec
3 be3343.ccr41.ams03.atlas.cogentco.com (154.54.62.142) 9 msec
be2950.ccr42.ams03.atlas.cogentco.com (154.54.72.41) 9 msec
be3343.ccr41.ams03.atlas.cogentco.com (154.54.62.142) 9 msec
4 be9036.ccr82.lon05.atlas.cogentco.com (154.54.72.185) 104 msec
be2182.ccr21.lpl01.atlas.cogentco.com (154.54.77.246) 104 msec 104 msec
5 port-channel8444.ccr92.lhr01.atlas.cogentco.com (154.54.72.182) 16 msec
port-channel3464.ccr91.lhr01.atlas.cogentco.com (154.54.75.150) 17 msec 17 msec
6 be8668.ccr31.bos01.atlas.cogentco.com (154.54.167.29) 99 msec
be3501.ccr42.jfk02.atlas.cogentco.com (154.54.95.101) 101 msec
be8668.ccr31.bos01.atlas.cogentco.com (154.54.167.29) 104 msec
7 be8030.ccr22.alb02.atlas.cogentco.com (154.54.169.226) 104 msec
port-channel2994.ccr92.cle04.atlas.cogentco.com (154.54.31.233) 99 msec
be8029.ccr21.alb02.atlas.cogentco.com (154.54.169.222) 104 msec
8 be2718.ccr42.ord01.atlas.cogentco.com (154.54.7.129) 101 msec
port-channel8023.ccr92.cle04.atlas.cogentco.com (154.54.169.197) 99 msec 99 msec
9 be5214.ccr31.oma02.atlas.cogentco.com (154.54.165.133) 126 msec
be2717.ccr41.ord01.atlas.cogentco.com (154.54.6.221) 103 msec 103 msec
10 be5214.ccr31.oma02.atlas.cogentco.com (154.54.165.133) 115 msec 115 msec
be5068.ccr32.oma02.atlas.cogentco.com (154.54.166.73) 116 msec
11 be6904.ccr81.den01.atlas.cogentco.com (154.54.95.97) 124 msec
be8568.ccr82.den01.atlas.cogentco.com (154.54.95.109) 126 msec
be2353.ccr81.slc03.atlas.cogentco.com (154.54.5.102) 132 msec
12 be3906.ccr22.sfo01.atlas.cogentco.com (154.54.5.154) 152 msec
be9563.ccr82.slc03.atlas.cogentco.com (154.54.163.206) 135 msec
be2353.ccr81.slc03.atlas.cogentco.com (154.54.5.102) 159 msec
13 be3906.ccr22.sfo01.atlas.cogentco.com (154.54.5.154) 150 msec
be2902.agr21.sfo01.atlas.cogentco.com (154.54.3.50) 149 msec
be3905.ccr21.sfo01.atlas.cogentco.com (154.54.1.210) 150 msec
14 be2903.agr21.sfo01.atlas.cogentco.com (154.54.6.202) 151 msec
be2902.agr21.sfo01.atlas.cogentco.com (154.54.3.50) 153 msec
be2903.agr21.sfo01.atlas.cogentco.com (154.54.6.202) 151 msec
15 ism01.sfo01.atlas.cogentco.com (66.250.250.146) !A !A !A

The trace shows the first hop timed out, and the second hop indicates a handover to Cogent, which subsequently routes the traffic to the US. This suggests that Cogent is responsible for carrying traffic between Germany and the US when the destination is also on the Cogent network.

Let’s check the reverse:

Cogent San Francisco -> Arelion Frankfurt - 62.115.129.161 (ffm-b11.ip.twelve99.net):

 traceroute to ffm-b11.ip.twelve99.net. (62.115.129.161), 30 hops max, 60 byte packets
1 gi100-0-0-40.agr21.sfo01.atlas.cogentco.com (66.250.250.145) 0.673 ms 0.641 ms
2 be2903.ccr22.sfo01.atlas.cogentco.com (154.54.6.201) 0.917 ms 1.010 ms
3 be2379.ccr31.sjc04.atlas.cogentco.com (154.54.42.158) 1.431 ms be2430.ccr31.sjc04.atlas.cogentco.com (154.54.88.186) 1.518 ms
4 palo-b24-link.ip.twelve99.net (62.115.174.64) 1.185 ms 1.190 ms
5 palo-bb4-link.ip.twelve99.net (62.115.139.110) 4.864 ms 4.172 ms
6 den-bb2-link.ip.twelve99.net (62.115.139.113) 354.428 ms 26.677 ms
7 kanc-bb2-link.ip.twelve99.net (62.115.137.115) 39.102 ms chi-bb1-link.ip.twelve99.net (62.115.115.77) 48.806 ms
8 chi-bb2-link.ip.twelve99.net (62.115.136.102) 46.750 ms 46.933 ms
9 ldn-bb1-link.ip.twelve99.net (62.115.139.245) 136.341 ms ewr-bb2-link.ip.twelve99.net (62.115.132.134) 63.423 ms
10 ldn-bb2-link.ip.twelve99.net (62.115.139.247) 134.406 ms 134.345 ms
11 prs-bb2-link.ip.twelve99.net (62.115.133.239) 135.638 ms 135.602 ms
12 * ffm-b11-link.ip.twelve99.net (62.115.124.117) 148.809 ms

As is visible, traffic is handed over to Arelion on the 4th hop to Arelion within the Bay area (San Jose) and then Arelion carries it to Europe. It’s a standard practice and almost all major backbones have it as long as they are peering on multiple continents.

Airtel - Tata Comm routing in India

Coming to the Airtel - Tata Comm routing case, I don’t have access to full tables with BGP tables with BGP communities, but can certainly trace & do some tricks with trace to establish whether it’s hot potato or cold potato routing.

Trace from Airtel Haryana -> Tata Comm Chennai speedtest node:

mtr ookla.cn.as6453.net -wby0 -4
Start: 2026-05-20T18:51:59+0530
HOST: rtk01.anuragbhatia.com Loss% Snt Last Avg Best Wrst StDev
1. AS??? 172.16.16.1 0.0% 10 0.3 0.3 0.3 0.4 0.0
2. AS??? 10.240.9.204 0.0% 10 6.2 6.3 4.2 12.8 2.5
3. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
4. AS9498 169.168.18.125.dhcp.anaronline.net (125.18.168.169) 0.0% 10 5.0 8.8 4.9 22.0 5.9
5. AS??? 116.119.164.99 0.0% 10 28.5 30.1 28.5 31.9 1.1
6. AS4755 115.110.234.141.static.Mumbai.vsnl.net.in (115.110.234.141) 0.0% 10 31.9 33.5 31.9 35.8 1.3
7. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
8. AS4755 115.110.96.156.static-Ahmedabad.vsnl.net.in (115.110.96.156) 0.0% 10 50.4 50.9 49.2 52.1 0.9

Ignore AS6453 in hostname (it’s AS4755), ignore Ahmedabad in reverse DNS as it’s Chennai and also ignore “anaronline.net” in hope 4 - it’s just an Airtel router with this outdated PTR in Saha, Ambala, Haryana. 😀 With those three things out of the way, the trace shows Hop 5 with 28ms latency, indicating a Mumbai-based hop, followed by Hop 6, which is the Tata Comm IP.

So there are two possibilities here:

Hop 5 is Tata Comm AS4755 router sitting on Airtet’s IPv4. The chance of it is very low because a) Hop 4 is a known non-peering location (Ambala) for Airtel. It would be very unusual for them to have a circuit from Ambala to Mumbai with Airtel side in Ambala peering with the Tata Comm side in Mumbai. b) Usually networks interconnect over a /30 or /31. If it were a /30 CIDR block, 116.119.164.99 would fall within 116.119.164.96/30, requiring the use of .97 and .98, which is not the case. If it were a /31, it would be 116.119.164.98/31, with one side at .98 (Ambala) and the other at .99 (Mumbai). 116.119.164.99 is in Mumbai for sure as I have servers in Mumbai directly connected to Airtel and it’s 1.7ms away but then .98 from my home in Haryana should be 5ms, but it’s 85ms.
Hop 5 is Airtel, hop 6 is Tata Comm and interconnection (PNI) is between hop 5 and hop 6. In this case likely 115.110.234.140/30 is used with .141 on the Tata Comm side and .142 on the Airtel side. It cannot be /31 as in that case it would be .141 and .140 and .140 is not reachable. From my server in Mumbai connected to Tata Comm, I see 4 hops to .141 and 5 hops to .142. So it’s safe to establish that this is indeed the case.

So for traffic from Airtel Haryana to Tata Comm Chennai, traffic handover happened far off in Mumbai with Airtel carrying traffic till Mumbai. Let’s see how it is in reverse: OVH Mumbai (behind Tata Comm) to speedtestsaha1.airtel.in (Airtel Ambala Speedtest):

> Mumbai, IN, AS, OVH (AS16276)
Host Loss% Drop Rcv Avg StDev Javg
1. AS16276 _gateway (148.113.1.252) 0.0% 0 3 0.5 0.0 0.3
2. AS??? 10.164.249.124 (10.164.249.124) 0.0% 0 3 0.3 0.1 0.1
3. AS??? 10.133.49.24 (10.133.49.24) 0.0% 0 3 0.4 0.1 0.3
4. AS??? 10.75.24.26 (10.75.24.26) 0.0% 0 3 0.2 0.0 0.1
5. AS??? 10.75.248.194 (10.75.248.194) 0.0% 0 3 2.3 0.5 1.9
6. AS??? 10.75.248.200 (10.75.248.200) 0.0% 0 3 2.4 0.6 2.0
7. AS16276 bom-ynm1-sbb1-8k.ind.asia (103.5.12.8) 0.0% 0 3 2.2 0.6 1.7
8. AS??? 10.200.4.7 (10.200.4.7) 0.0% 0 3 0.3 0.0 0.2
9. AS??? (waiting for reply)
10. AS??? (waiting for reply)
11. AS4755 115.110.210.66.static-Delhi.vsnl.net.in (115.110.210.66) 0.0% 0 3 52.6 0.1 26.4
12. AS9498 116.119.50.26 (116.119.50.26) 0.0% 0 3 57.2 0.1 28.7
13. AS9498 nsg-corporate-130.219.185.122.airtel.in (122.185.219.130) 0.0% 0 3 57.0 0.1 28.6

Hops 8, 9 & 10 keep timing out no matter how many times I try. Hop 11 - that’s a Tata Comm IP as visible from reverse DNS and ASN mapping. This could actually be Tata Comm IP on Tata Comm router OR Tata Comm IP on Airtel router (part of /30 peering). 115.110.210.66 is from 115.110.210.64/30 with .65 and .66 as usable IPs.

Let’s trace from my home:

HOST: rtk01.anuragbhatia.com Loss% Snt Last Avg Best Wrst StDev
1. AS??? 172.16.16.1 0.0% 10 0.3 0.3 0.2 0.4 0.1
2. AS??? 10.240.9.204 0.0% 10 27.8 17.6 4.2 64.0 18.9
3. AS??? 172.31.0.195 0.0% 10 1171. 1352. 1068. 1656. 231.4
4. AS9498 173.168.18.125.dhcp.anaronline.net (125.18.168.173) 0.0% 10 12.6 8.1 4.9 13.1 2.8
5. AS4755 115.110.210.66.static-Delhi.vsnl.net.in (115.110.210.66) 0.0% 10 11.2 11.7 9.5 13.8 1.5

This strongly hints that 115.110.210.66 is a Tata Comm IP on an Airtel router because hop 4 (Ambala) is not known to be connecting to outside networks. So, it is highly likely that 115.110.210.66 is IPv4 on an Airtel router in Delhi NCR. Let’s check for .65 to see if it adds exactly one hop extra:

HOST: rtk01.anuragbhatia.com Loss% Snt Last Avg Best Wrst StDev
1. AS??? 172.16.16.1 0.0% 10 0.3 0.4 0.2 0.7 0.1
2. AS??? 10.240.9.204 0.0% 10 5.0 7.6 4.5 22.5 5.4
3. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
4. AS9498 169.168.18.125.dhcp.anaronline.net (125.18.168.169) 0.0% 10 4.4 6.4 4.4 8.6 1.4
5. AS??? 116.119.161.28 0.0% 10 15.8 19.0 10.6 32.7 6.4
6. AS4755 115.110.210.65.static-Delhi.vsnl.net.in (115.110.210.65) 0.0% 10 13.1 12.4 10.5 14.4 1.3

With this it’s safe to say that in OVH trace, hop 11 is an Airtel router on Tata Comm IP & both .65 (Tata Comm) - .66 (Airtel) are in Delhi NCR. This means routing was as follows: OVH Mumbai > Tata Comm Mumbai > Tata Comm Delhi (one of those hops timing out) > Airtel Delhi > Airtel Ambala. So again, handover happened near destination & not the source - cold potato routing.

One more trace: Tata Comm AS4755 Bangalore -> speedtestggn1.airtel.in (Airtel speeddtest, Gurgaon) from active RIPE Atlas probe (#50525):

RESOLVED speedtestggn1.airtel.in TO 223.224.79.2
STARTED QUERY AT 2026/05/20 14:44:28 UTC
Fetching Measurement: 172850592
Traceroute from 193.32.247.8 to 223.224.79.2 (223.224.79.2):
1 121.242.164.33.static-Bangalore.vsnl.net.in (121.242.164.33) 1.239ms 0.764ms 0.716ms
2 219.65.110.33.static-bangalore.vsnl.net.in (219.65.110.33) 1.837ms 2.054ms 1.606ms
3 172.31.155.74 44.018ms 33.515ms 33.377ms
4 115.110.232.174.static.Delhi.vsnl.net.in (115.110.232.174) 39.724ms 39.559ms 39.534ms
5 116.119.109.16 35.586ms 35.628ms 35.488ms
6 dsl-tn-dynamic-138.223.22.125.airtelbroadband.in (125.22.223.138) 37.125ms 37.273ms 37.094ms
7 223.224.79.2 37.375ms 37.318ms 37.437ms

Again, hop 4 - 115.110.232.174 is Tata Comm’s IP on the Airtel router and thus interconnection happens before that. Hop 3 has to be Tata Comm and since latency is already 33ms, it confirms Tata Comm carried traffic till Delhi NCR and handover happened far from the origin.

Another test: RIPE Atlas probe (#1009203) on Excitel Delhi (behind Tata Comm) > Airtel Mumbai speedtest:

Fetching Measurement: 172853300
Traceroute from 172.21.0.2 to 182.78.248.10 (182.78.248.10):
1 172.21.0.1 0.101ms 0.073ms 0.111ms
2 192.168.1.1 1.222ms 0.678ms 1.024ms
3 103.212.157.10 1.496ms 1.94ms 2.639ms
4 103.212.157.1 4.718ms 4.759ms 9.593ms
5 14.140.113.29.static-Delhi-vsnl.net.in (14.140.113.29) 3.302ms 3.987ms 3.292ms
6 172.23.78.234 25.475ms 25.656ms 25.618ms
7 115.110.234.142.static.Mumbai.vsnl.net.in (115.110.234.142) 29.061ms 53.108ms 29.121ms
8 116.119.121.198 26.656ms 26.289ms 26.564ms
9 speedtestmum1.airtel.in (182.78.248.10) 29.857ms 28.881ms 28.397ms

Excitel hands over to Tata Comm within Delhi (3ms latency). Hop 6 is Mumbai-based on latency (likely Tata Comm’s private IP) and hop 7 is Tata’s IP on Airtel. Again, hop 7 - 115.110.234.142 is part of 115.110.234.140/30 with .141 and .142. Likely .142 is Airtel and .141 is Tata Comm. Let’s verify with trace to see if there’s an exact 1 hop difference:

HOST: bom01.anuragbhatia.com Loss% Snt Last Avg Best Wrst StDev
1. AS31898 140.91.204.7 0.0% 10 0.3 0.3 0.2 0.3 0.0
2. AS4755 121.241.5.170.mumbai-static.vsnl.net.in (121.241.5.170) 0.0% 10 1.0 1.1 0.9 1.6 0.2
3. AS4755 121.241.5.169.mumbai-static.vsnl.net.in (121.241.5.169) 0.0% 10 1.4 1.3 1.1 1.5 0.1
4. AS4755 115.110.234.141.static.Mumbai.vsnl.net.in (115.110.234.141) 0.0% 10 1.9 2.3 1.6 7.1 1.7

HOST: bom01.anuragbhatia.com Loss% Snt Last Avg Best Wrst StDev
1. AS31898 140.91.204.33 0.0% 10 0.2 0.3 0.2 0.4 0.1
2. AS4755 121.241.5.170.mumbai-static.vsnl.net.in (121.241.5.170) 0.0% 10 1.0 1.0 0.8 1.2 0.1
3. AS4755 121.241.5.169.mumbai-static.vsnl.net.in (121.241.5.169) 0.0% 10 1.3 1.4 1.2 1.6 0.1
4. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
5. AS4755 115.110.234.142.static.Mumbai.vsnl.net.in (115.110.234.142) 0.0% 10 2.3 3.9 2.0 18.5 5.1

This confirms hop 7 in the Excitel trace is Tata Comm’s IP on the Airtel router and hence handover happened between 6 & 7 with 6 in Mumbai itself. So again, Tata Comm carried over traffic over long distance Delhi -> Mumbai when the destination was Airtel.

I think it’s safe to conclude with all these traces that Airtel & Tata Comm follow cold potato routing in India. Each carries traffic over its own backbone over long distances and hands it over only close to the destination.

Why it’s cold potato routing?

Hard to know for sure but I guess the same set of people would have handled Airtel - Tata Comm PNI who also handled NIXI peering. Operators were used to regional route announcement (only) at NIXI and hence the same continued in private peering as well. Likely Airtel is announcing North Indian routes to Tata Comm in Delhi PNI (only), Western routes in Mumbai PNI etc with some overlap between Mumbai & Chennai for Bangalore/Kolkata routes to balance off PNIs. Same with the Tata Comm side as well. Changing it at later stage would need coordination for re-balancing traffic over PNIs which would add to the friction.

This can always lead to some missed routes, resulting in them not being announced at any peering & thus would take a scenic route via transit-free settlement backbones like AS6453 (technical upstream of AS4755, partially of AS9498 as well) and for AS9498 - AS1299/174/AS2914 etc.

Disclaimer: This is my personal blog, and hence, posts made here are in my personal capacity. These do not represent the views of my employer.

Strange route with Jio and GTT as adjacency

Fri, 01 May 2026 01:12:16 +0530

Last month (on 18th April 2026), I saw an alert for a strange route announcement:

Prefix	AS Path	Origin ASN
117.120.58.0/23	262427 262761 263444 6453 3257 55836 9498 134863	134863

Background

While this specific alert triggered because it had three large Indian backbones - Tata Comm (AS6453), Jio (AS55836) and Airtel (AS9498) and normally this would appear just as a leak by Jio for routes learnt from Airtel (a known peer of Jio), but AS3257 before Jio makes it strange and interesting. AS3257 is a known transit-free network GTT. Jio is not known to be connected to them at all. Jio does have a mix of transit & peering internationally via AS64049 but not via AS55836. When they started, they had some transit via Tata Comm (AS4755) for a few early days of the launch. After that AS55836 only has peering with Tata Comm (AS4755) and Airtel (AS9498), besides technically transit via their own AS64049 which takes transit from Arelion (AS1299), Cogent (AS174) and NTT (AS2914) and a mix of transit/peering relations with Lumen (AS3356).

Who possibly caused it?

The AS_PATH here: 262427 262761 263444 6453 3257 55836 9498 134863 can be read backwards to understand the announcement:

SP Internet (134863) > Airtel (9498) > Jio (55836) > GTT (AS3257) > Tata Comm (AS6453) > Open X Tecnologia (AS263444) > Sinal Br Telecom (AS262761) > Invista Net Provedor (AS262427). As mentioned in a post last year, I hold recent routes in a Clickhouse table from various route collectors. This makes it easy to query a large number of routes quickly.

SELECT count(*)
FROM bgp.table
Query id: df33f954-7fbd-41a3-8de2-184990ca06d4
┌─────count()─┐
1. │ 68882920533 │ -- 68.88 billion
└─────────────┘
1 row in set. Elapsed: 0.034 sec.

Let’s ask this table of 68.88 billion routes for routes matching AS_PATH: 3257 55836:

SELECT timestamp, prefix, as_path, collector
FROM bgp.table
WHERE arrayExists(i -> ((i < length(as_path)) AND ((as_path[i]) = 3257) AND ((as_path[i + 1]) = 55836)), arrayEnumerate(as_path))
Query id: 6805de6a-c1c5-47e8-ab83-9e1549527d65
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector─────────────┐
1. │ 2026-04-19 02:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.rio │
2. │ 2026-04-19 02:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.fortaleza │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴───────────────────────┘
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector───────┐
3. │ 2026-04-19 04:00:02 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.rio │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴─────────────────┘
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector─────────────┐
4. │ 2026-04-19 04:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.fortaleza │
5. │ 2026-04-19 06:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.fortaleza │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴───────────────────────┘
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector───────┐
6. │ 2026-04-19 06:00:04 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.rio │
7. │ 2026-04-18 16:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.rio │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴─────────────────┘
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector─────────────┐
8. │ 2026-04-18 16:00:04 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.fortaleza │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴───────────────────────┘
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector─────────────┐
9. │ 2026-04-18 18:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.rio │
10. │ 2026-04-18 18:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.fortaleza │
11. │ 2026-04-18 20:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.fortaleza │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴───────────────────────┘
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector───────┐
12. │ 2026-04-18 20:00:04 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.rio │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴─────────────────┘
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector─────────────┐
13. │ 2026-04-18 22:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.rio │
14. │ 2026-04-18 22:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.fortaleza │
15. │ 2026-04-19 00:00:03 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.fortaleza │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴───────────────────────┘
┌───────────timestamp─┬─prefix──────────┬─as_path────────────────────────────────────────────┬─collector───────┐
16. │ 2026-04-19 00:00:04 │ 117.120.58.0/23 │ [262427,262761,263444,6453,3257,55836,9498,134863] │ route-views.rio │
└─────────────────────┴─────────────────┴────────────────────────────────────────────────────┴─────────────────┘

So routes were only visible at route-views in Rio de Janeiro (Brazil) and Fortaleza (Brazil), not any other collector or by any other ASN other than AS262427 which is feeding these routes in both the collectors. Both GTT (AS3257) and Tata Comm (AS6453) feed a few collectors but this route is not visible in their feeds. So it’s safe to say that this route is “generated” by the ASNs on the left side of AS6453 in the above AS_PATH. It’s either AS262427, AS262761 or AS263444 that “generated” this route.

I cannot find exactly which one of these three because:

Route is historical (no point of live lookup now)
AS262761 and AS263444 are not feeding any of the known public collectors and hence routes cannot be compared between these three.

It could be some sort of BGP route optimiser’s work in (AS262427, AS262761 or AS263444) but I fail to understand why they would show this fake adjacency between 3257 & 55836. For all this time the prefix 117.120.58.0/23 in reality was not in the downstream of Airtel (AS9498) but Vodafone IDEA (AS55410).

Netflow for home router & Linux servers

Fri, 10 Apr 2026 01:50:00 +0530

For the last few weeks, I have been running a NetFlow collector for the home router. This is something I wanted to do for a long time, but I was missing the time to invest. There are a few open source options, and I guess many commercial solutions offering NetFlow, often bundled with other products.

My personal monitoring is 100% open source and presently running Prometheus + Thanos + Node Exporter + Blackbox Exporter + SNMP Exporter + Grafana + Grafana Loki + a few more exporters. So I started looking for open source options which are still under active development and supported.

Two of them are quite popular - Pmacct and Akvorado. Pmacct is extremely advanced, flexible, but at the same time overall complicated to set up (and maintain). Even their quickstart file 3100-line file is full of setup options. On the other hand, Akvorado seems simpler to maintain. Some complications, anyway, are expected with NetFlow because the goal is not just to collect the data but also to have a system to store it, analyse it, map IPs to location/AS numbers, etc., dashboards, etc. It’s a tool developed by French ISP Free, which is part of the Iliad group, which also owns Scaleway.

Setting up Akvorado is easy if you are familiar with Docker. They have a simple 4 command quick-start which deploys all their containers as part of the stack. They deploy a few containers as part of the overall stack.

The big picture page on their demo site documentation covers the overall architecture. I started feeding it data from the home (Mikrotik) router and later also added various Linux servers/VMs I manage for R&D, DNS, as well as to host this blog and other infrastructure. For Linux, I am using Pmacct as an exporter, as Akvorado’s documentation suggests using it in exporter mode and has a sample config which works.

Some data

Now that it’s been running for a few weeks, I can see major sources of data coming to the home. Before jumping to data, it’s important to note that I have regular camera feeds uploading traffic to my server in Mumbai, and some backups, route collector data pull and automated speedtests at home. All these kinds of “pollute” data, and thus, it is more fun to exclude and look for remaining data coming from regular usage by end devices at home. In the setup, I can easily filter data based on srcAS, dstAS, srcIP, dstIP, port, in and out interfaces, etc.

Here’s the data for the last 10 days towards end devices (excluding home server and camera traffic)

While most of ASNs here are expected, but just for the context, Esto AS135817 is one of the upstreams at home (friendly ISP that I reach over a GRE tunnel over an IX from an underlying ISP), and most of the traffic hitting it is actually CDN traffic for Google GGC, Facebook FNA, etc., sitting on their IPs.

What is more fun here are the Sankey graphs, where for a given interval, I can map ANY attribute -> ANY other attribute. e.g., srcAS -> Home Out interfaces or SrcPort -> SrcAS -> Destination IPs, etc.

Here’s the Sankey Graph for SrcPort -> SrcAS -> In Interface at home

On this connection, I get most of Google traffic from Google’s AS directly; however, for a few days, I was on Airtel as primary due to an outage on the underlying ISP and thus could not reach Esto over the tunnel. For those days, traffic patterns were very different. Both Google and Akamai terminated most of the traffic on caching nodes within Airtel.

Traffic profile when running home traffic via Airtel:

Akvorado has support for Grafana, and while their dashboard seems good for usual lookups, I miss a pie chart. Let’s see what the pie chart of the last 10 days’ traffic looks like:

Another fun thing is to look for traffic by Etype for IPv4 Vs IPv6 comparison. Let’s see how much IPv4 Vs IPv6 traffic there is in the last week:

The same logic can be used to have a pie chart, but it has to be in Grafana.

Thus for now 89% of traffic is IPv6 at home. On that note, time to end this post. 😀

Who can light fibre in India?

Sun, 22 Feb 2026 02:19:05 +0530

Friend and his two homes…

Recently, a friend of mine mentioned his plan to procure dark fibre to connect two of his homes. His idea was straightforward - get LCO to give a dark strand between both homes and run Bidi optics over it. I ended up telling him that while his idea is technically fine, as modern optics can easily cover that sort of distance, it would be illegal as per Indian telecom laws!

It wasn’t just him, but numerous content players, hosting companies, and Cloud players find it a bit odd when they find this out. In this post, I will document what is allowed and what is not allowed.

Advance warning: I am a technical person & do not know much about regulatory. Whatever I am sharing here is what I have learned from my friends who have expertise in the subject. Indian telecom rules are overly complex and can have a different set of interpretations, and often go into the grey area. I welcome readers to share feedback if they are aware of something which isn’t correct as per the regulation. With that warning out of the way, I will proceed with the post.

Understanding licensing

In India, when fibre is going outside of a given building, one needs to have a license to light it. In fact, not just “lighting up a dark fibre” but to deploy and resell dark fibre, one needs a license (IP-1). Infrastructure Provider Category 1 (or IP-1) is considered an easy license which can be used to sell passive assets such as dark fibre, ducts, towers, etc. Active infra i.e lighting up fibre, is not allowed under this license, and it’s expected that an IP-1 holder leases their dark fibre only to a license holder who is authorised to light the fibre. Within a building, campus, datacenter, home etc one can deploy a light fibre but as soon as it goes outside of that it becomes a bit of a grey area depending on the use case.

An Internet Service Provider (ISP) holding an ISP license or Unified License with ISP authorisation can light fibre but only for two specific purposes:

Offering internet drop to the end user (quite obvious)
Their own backhaul connectivity between routers, switches, OLTs, etc located anywhere within their authorisation area

While this looks quite straightforward, it specifically excludes point-to-point connectivity. An ISP cannot offer point-to-point connectivity under its license. So for my friend’s use case - he cannot buy a point-to-point link using any technology (MPLS, layer 2 VLAN, DWDM wavelength or dark fibre strands) from the ISP as the ISP under the ISP license is not authorised to sell those. But funny enough if the same ISP sells him “internet drop” under ISP license and two routers (outside of core ISP business) and “helps” in setting VPN over the public internet, it’s largely legal because from ISP and DoT’s point of view this is an internet product. About #2 - For a long time ISPs could not procure dark fibre and make use of it because they could not make full use of the capacity. They could not light it with say DWDM & resell wavelengths. However recently it was clarified that an ISP can sell capacity to other licensed ISPs only (not end non-licensed customers). When researching I found some friends not being aware of it as well as some calling it a grey area rule.

Access License & NLD

An Access license in India for selling circuits. It allows many things including selling p2p circuits, voice, mobile, etc. Many networks use an access license along with NLD (National Long Distance) which allows these services across India. It’s a vast license due to involvement of voice and mobile permissions but for the context of this post - it’s the ultimate license needed by players to do things without going into the grey area.

Debate on captive

I have heard of different opinions around captive traffic. An ISP friend of mine who has expertise in regulation (and doesn’t want to be named in this post) recently bid for a government contract. A project where the goal was to offer dark fibre to one department of the government. He had an ISP license while the government department that was taking fibre had no license. Their end goal was to connect to their own infra. This raised some concerns from other bidders. For now it’s unclear and I guess subject to interpretation on whether someone can light fibre for their captive use or not when the end product itself does not resell the capacity. Most large projects end up taking NLD and access licenses anyway because they intend to sell excess capacity. Thus networks like Railtel (by Indian Railway), Oil India, Powergrid Corp, etc all take NLD license to have legal ability to light fibre for their own use as well as to resell excess capacity. Industry-wide I hear that private players taking dark fibre and lighting it themselves is absolutely not allowed while for the government. bodies it remains a grey area.

Cloud and content players

Modern cloud and content players are major users of dark fibres, especially within metros. Hyperscalars like Amazon AWS, Google Cloud and Microsoft Azure deploy their infra in three datacenters in a given region & these have to be technically connected with dark fibre. There’s massive East-West traffic due to replication, high availability & many other reasons. In the majority of markets these companies can procure dark fibre between their datacenters and light it themselves but in India they cannot. In India, it’s a known case that they get external license-holding players to light fibre for them. In most of these cases hyperscalars work very closely with those players, they define network topology, hardware, configs etc but ultimately someone else executes it for them. Most of the Cloud players will never take a telecom license (ISP, access, NLD, ILD etc) in India simply because then they would be subject to many of the telecom-related rules and regulations. That brings business, security and many concerns. Take e.g the long over-discussed AGR case. If any of the Cloud players had a license, under older terms their compute, storage (and just anything) revenue would be subject to AGR / license fees. In a way, it’s good that Cloud players work well without a license as that keeps things simple. The fact that they cannot light fibre themselves is a drawback of our Indian licensing system and should be fixed. In most other markets including the US, UK, Germany, Singapore etc - lighting of fibre does not need a license.

Status of datacenters

Over time datacenters have taken an NLD license. List includes but is not limited to NTT Netmagic, Yotta, Ctrls etc. This is mostly because these players have multiple datacenters in a given area, and they want to connect and resell lit services. They can in theory do without NLD by using IP-1 and then reselling dark fibre strands only but that does not scale up, plus their customers who are buying would need to have a license to light the fibre.

Old and outdated ideas around regulation

Most of these ideas around licensing in the fixed-line market do not make sense anymore, simply because everything runs on fibre and fibre is not a scarce resource. ISP license somewhat makes sense (excluding the AGR part) because there is a requirement of URL filtering, CGNAT logging, etc., but outside of that, it’s largely an inflated set of rules. On the access side, the requirement of a license to light fibre simply promotes more rent-seeking behaviour instead of cleaner, competitive and optimised deployments. AGR on fixedline (and probably mobile telephony) also doesn’t make sense. In case of fixed line, the network runs on optical fibre which is laid by vendors who pay Rights of Way (RoW) to municipalities, State and central government (whichever road they use). There is taxation in the form of GST in the system on top of RoW. So the AGR on top doesn’t make sense as bits are not flowing over any scarce resource. It’s largely over privately built infrastructure, for which RoW and GST are already being paid.

With those thoughts I think my friend needs an access license to connect his homes and his LCO (or ISP who actually runs it) needs a IP-1 license to legally re-sell him dark fibre. 😀

Disclaimer: This is my personal blog, and hence, posts made here are in my personal capacity. These do not represent the views of my employer.

Update - 18 Mar 2026

I cannot count how many people by now have told me that new rules are just about to be notified and changes are inevitable. New rules will allow ISP license holders (or folks with UL license with ISP authorization) to be able to sell point to point services. That won’t fix the problem completely but greatly reduces it.

Another point to mention here is that in Nov 2022 DoT tweaked IP-1 license rules to allow them to give dark fibre to non-licence holders which are specified by DoT.

Airtel India - Europe routing issues

Thu, 29 Jan 2026 03:45:04 +0530

Since 20:08 IST / 14:38 GMT on 28 Jan 2026, the latency between Airtel India and Europe has gone up along with significant packet loss.

Here’s an example of shortlist 300+ endpoints from Airtel India with European endpoints:

Traceroutes

Airtel Haryana -> Contabo Europe

Start: 2026-01-29T03:52:33+0530
HOST: desktop Loss% Snt Last Avg Best Wrst StDev
1. AS??? _gateway (172.16.0.1) 0.0% 10 0.2 0.2 0.2 0.2 0.0
2. AS??? 10.240.9.204 0.0% 10 6.2 7.7 4.6 17.7 3.7
3. AS??? 172.31.0.155 0.0% 10 12.1 8.4 5.0 12.8 3.2
4. AS9498 173.168.18.125.dhcp.anaronline.net (125.18.168.173) 0.0% 10 5.1 6.5 5.0 8.0 1.2
5. AS??? 182.79.243.34 0.0% 10 266.6 262.7 259.1 276.3 5.3
6. AS1299 lax-b22-link.ip.twelve99.net (62.115.162.62) 0.0% 10 259.4 258.1 256.6 259.9 1.1
7. AS1299 lax-bb2-link.ip.twelve99.net (62.115.140.156) 10.0% 10 256.7 257.9 256.6 259.6 1.2
8. AS1299 dls-b7-link.ip.twelve99.net (62.115.140.247) 20.0% 10 323.4 324.2 322.6 326.2 1.2
9. AS1299 atl-b24-link.ip.twelve99.net (62.115.140.32) 0.0% 10 336.4 336.5 335.4 338.0 0.9
10. AS1299 atl-bb2-link.ip.twelve99.net (62.115.143.236) 0.0% 10 323.1 321.7 320.2 323.4 1.0
11. AS1299 ash-bb2-link.ip.twelve99.net (62.115.137.132) 40.0% 10 320.9 321.1 320.6 321.7 0.4
12. AS1299 prs-bb2-link.ip.twelve99.net (62.115.140.106) 0.0% 10 318.8 318.3 317.2 319.5 0.9
13. AS1299 laut-b2-link.ip.twelve99.net (62.115.136.185) 0.0% 10 325.7 326.0 324.3 327.7 1.3
14. AS1299 contabohubeurope-ic-385701.ip.twelve99-cust.net (213.248.67.11) 0.0% 10 325.8 324.4 322.8 326.4 1.2
15. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
16. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
17. AS51167 eu01.anuragbhatia.com (213.199.54.67) 0.0% 10 323.1 323.5 322.4 324.9 0.9

Reading this trace: This is Ambala » Airtel MPLS » Los Angeles Arelion AS1299 handover > Arelion Dallas > Arelion Atlanta > Arelion Ashburn > Arelion Paris > Europe

Contabo Europe -> Airtel Haryana (CGNAT IP terminating in Saha (Ambala) Haryana):

Start: 2026-01-29T03:53:51+0530
HOST: eu01.anuragbhatia.com Loss% Snt Last Avg Best Wrst StDev
1. AS??? 10.243.1.101 0.0% 10 0.2 0.3 0.2 0.4 0.1
2. AS??? 10.0.66.2 0.0% 10 0.3 0.3 0.2 0.3 0.0
3. AS1299 laut-b2-link.ip.twelve99.net (213.248.70.176) 0.0% 10 2.7 2.8 1.4 4.7 1.0
4. AS1299 laut-b1-link.ip.twelve99.net (62.115.136.194) 0.0% 10 3.6 2.7 1.9 3.6 0.6
5. AS1299 ffm-bb1-link.ip.twelve99.net (62.115.136.146) 0.0% 10 2.9 3.0 2.9 3.5 0.2
6. AS1299 mei-b5-link.ip.twelve99.net (62.115.124.59) 0.0% 10 18.4 18.3 18.2 18.7 0.1
7. AS1299 sng-b7-link.ip.twelve99.net (62.115.134.229) 0.0% 10 155.1 155.2 154.9 155.6 0.3
8. AS1299 sng-b5-link.ip.twelve99.net (62.115.139.1) 80.0% 10 162.5 162.5 162.5 162.6 0.1
9. AS1299 bhartiairtel-ic-375944.ip.twelve99-cust.net (62.115.49.157) 0.0% 10 238.1 236.2 233.5 252.4 5.9
10. AS9498 116.119.44.230 0.0% 10 320.7 321.0 320.5 323.2 0.8
11. AS9498 170.168.18.125.dhcp.anaronline.net (125.18.168.170) 0.0% 10 327.1 328.9 319.4 355.2 13.6
12. AS??? ???

This one is handover to Arelion AS1299 Europe locally > Arelion Frankfurt > Arelion Marseille » Arelion Singapore > Bharti Airtel AS9498 Singapore handover »> Airtel MPLS > Haryana

How to know that it’s not just me on both ends - my home connection in Haryana and my server in EU being impacted?

Let’s look for average latency to a few dozen Airtel endpoints in India from Contabo, Hetzner, hosting.de:

Impact from BGP routing table

Let’s compare Airtel AS9498 routes inside Arelion AS1299 table with BGP community tags: 1299:30000 (EU Customers) Vs 1299:37000 (APAC customers) based on routes visible from various downstreams of AS1299 across public route collectors (since Arelion’s own direct feed does not include the BGP community tag):

Within a few hours Arelion’s learning of Airtel AS9498 routes in Europe (1299:30000) went down by 7000 routes and APAC (1299:37000) went up by over 6000 routes.

What about other Indian telcos?

There seems some minor impact on Tata Comm but it is largely contained. Average latency from EU - Tata Comm Indian endpoint went up from 164ms to 180ms. In the case of Jio, it also shows the same behaviour as usual. However, on Jio, there’s a visible jump in latency every late evening for months, and this jump seems to follow the same pattern.

Trace from EU to Tata Comm AS4755 Delhi confirms largely normal routing.

Start: 2026-01-29T04:16:29+0530
HOST: eu01.anuragbhatia.com Loss% Snt Last Avg Best Wrst StDev
1. AS??? 10.243.1.101 0.0% 10 0.2 0.2 0.2 0.3 0.0
2. AS??? 10.0.66.2 0.0% 10 0.2 0.2 0.2 0.3 0.0
3. AS1299 laut-b1-link.ip.twelve99.net (62.115.45.148) 0.0% 10 2.5 3.1 1.6 4.7 1.0
4. AS1299 ffm-bb1-link.ip.twelve99.net (62.115.136.146) 0.0% 10 3.2 3.1 2.9 3.4 0.2
5. AS1299 ffm-b5-link.ip.twelve99.net (62.115.136.213) 0.0% 10 3.5 3.2 3.0 3.5 0.1
6. AS1299 tata-ic-378325.ip.twelve99-cust.net (213.248.78.41) 60.0% 10 4.3 4.3 3.9 4.7 0.3
7. AS6453 if-bundle-56-2.qcore2.pvu-paris.as6453.net (80.231.245.40) 60.0% 10 16.1 16.0 15.6 16.4 0.4
8. AS6453 if-bundle-12-2.qcore1.pvu-paris.as6453.net (80.231.245.12) 0.0% 10 15.4 15.8 15.4 16.8 0.4
9. AS6453 if-bundle-22-3.qcore1.pye-paris.as6453.net (80.231.154.200) 50.0% 10 15.7 15.9 15.6 16.3 0.3
10. AS6453 if-bundle-2-2.qcore2.pye-paris.as6453.net (80.231.154.27) 0.0% 10 15.5 15.9 15.4 16.4 0.4
11. AS6453 if-bundle-13-2.qcore1.ldn-london.as6453.net (80.231.196.37) 90.0% 10 20.5 20.5 20.5 20.5 0.0
12. AS6453 195.219.213.129 70.0% 10 15.4 15.5 15.4 15.7 0.2
13. AS6453 80.231.131.77 0.0% 10 123.0 125.2 123.0 144.3 6.7
14. AS??? ??? 100.0 10 0.0 0.0 0.0 0.0 0.0
15. AS4755 115.114.73.212.static-delhi.vsnl.net.in (115.114.73.212) 0.0% 10 158.5 158.6 158.5 158.7 0.1

With hope that your packets won’t be spread at the bottom of the ocean, it’s time for me to end this post and get back to work. 😀

Update - 29 Jan 2026 / 17:14 IST

A friend from the industry has confirmed that MENA cable has went down around the same time.

Update - 04 Mar 2026

Significant traffic routing between India and EU seems direct now.

Update - 13 Mar 2026

MENA Cable has been fixed and connectivity between India and EU is back online. Since evening there’s a visible further drop in average latency between India and EU over the Airtel.

When BGP lies, the internet believes!

Wed, 21 Jan 2026 00:02:42 +0530

Earlier in the day, I came across Liberty Global (AS6830) seemingly originating several Vodafone Romania (AS12302) prefixes.

Source: https://bgp.he.net/AS6830#_prefixes

This is highly unusual because AS6830 is a large transit-free network. I have seen some transit-free networks leaking routes, but originating a large number of prefixes is not common. It has massive stakes from Belgium-based Telenet to Virgin Media, etc. To verify if they are actually originating these or not, let’s check from their looking glass for one of the prefixes here: 46.97.104.0/24 via their PoP at Interxion FRA6 Frankfurt:

This clearly shows that their own router is learning it from Vodafone C&W AS1273 and not holding the fake route. The origin is AS12302 (Vodafone Romania). So why does that prefix appear in bgp.he.net?

Let’s look at real-time lookup from super-lg:

Reading this AS_PATH: 35505 44682 8751 6830

So RIPE RIS RRC22 in Bucharest, Romania “learns” this from AS35505 (Pronet Solutii IT SRL) which learns it from AS44682 (SIL-MIRO COM SRL) which learns it from AS8751 (MEDIA SAT SRL) which “claims” to have learnt it from AS6830. This very much smells like a “fake route” generated by someone here likely from a route optimiser. From the Liberty Global looking glass, it’s clear that AS6830 does not have it. So it has to be either AS8751 or AS44682 or AS35505 having this route in their table. It’s hard to verify and be 100% sure who since those three ASNs don’t have a looking glass or a RIPE Atlas probe for me to see their routing. Thus when BGP lies, the internet believes!

Disclaimer: This is my personal blog, and hence, posts made here are in my personal capacity. These do not represent the views of my employer.

Opentofu + Tailscale = Bootstrapped DevOps environment with VPN

Sun, 18 Jan 2026 02:58:46 +0530

For the last few days, I have been playing with OpenTofu. For those who may not know, it’s a fork of last Terraform as Terraform’s license was changed due to IBM’s acquisition of Hashicorp and is a Cloud Native Foundation project. It can be used to quickly deploy (and remove) resources from various cloud players.

Yesterday came across this tweet from Tailscale about tailscale’s module for deployment.

Cloud-init can be tough, and we've heard all about it. So we built an open-source Terraform module, one that helps provide a more consistent Tailscale experience across AWS, Azure, GCP, and everywhere. No more surprises, OS quirks, or other mysteries: https://t.co/vn1ITU7Fgz pic.twitter.com/stcF6WuiD7
— Tailscale (@Tailscale) January 15, 2026

It’s cool and useful, though I find Cloud init way more useful since for me these tools are more useful for quick testing rather than a permanent production server. The idea of tailscale on devops VMs as they come up is pretty powerful, as it takes care of the private network between these machines, plus underlay doesn’t matter, and hence one can use (cheaper) IPv6-only VMs.

An example of OpenTofu config (which is similar to Terraform) to deploy three IPv6 only servers on players like Hetzner across Nuremberg, Helsinki, and Ashburn:

main.tf

terraform {
required_providers {
hcloud = {
source = "hetznercloud/hcloud"
version = "~> 1.59.0"
}
}
}
provider "hcloud" {
token = var.hcloud_token
}
data "hcloud_ssh_key" "desktop" {
name = "desktop"
}
data "hcloud_image" "ubuntu" {
name = "debian-13"
most_recent = true
}
resource "hcloud_server" "vms" {
for_each = var.vms
name = each.key
image = data.hcloud_image.ubuntu.id
server_type = each.value.server_type
location = each.value.location
ssh_keys = [data.hcloud_ssh_key.desktop.id]
user_data = <<-EOF
#cloud-config
package_upgrade: true
packages:
- curl
runcmd:
- curl -fsSL https://tailscale.com/install.sh | sh
- tailscale up --authkey=${var.tailscale_authkey} --accept-routes
EOF
public_net {
ipv4_enabled = false
ipv6_enabled = true
}
labels = {
managed-by = "opentofu"
}
lifecycle {
ignore_changes = [user_data]
}
}
output "vm_ips" {
value = { for k, vm in hcloud_server.vms : k => vm.ipv6_address }
}

variables.tf (to hold non-secret variables)

variable "hcloud_token" {
description = "Hetzner Cloud API Token"
type = string
sensitive = true
}
variable "vms" {
description = "Map of VMs with locations"
type = map(object({
location = string
server_type = optional(string, "cx23")
}))
default = {
devops01 = { location = "nbg1" }
devops02 = { location = "ash", server_type = "cpx11" }
devops03 = { location = "hel1" }
}
}
variable "tailscale_authkey" {
description = "Tailscale Auth key"
type = string
sensitive = true
}

terraform.tfvars (to hold secret variables)

hcloud_token = "XXX"
tailscale_authkey = "tskey-auth-XXXXXX"

Triggering machines (with VPN built in)

Initiating (to install providers, initiate backend etc)

anurag@desktop ~/R/P/c/a/test (main)> tofu init
...
anurag@desktop ~/R/P/c/a/test (main)>

And the deployment!

anurag@desktop ~/R/P/c/a/test (main)> tofu apply
data.hcloud_image.ubuntu: Reading...
data.hcloud_ssh_key.desktop: Reading...
data.hcloud_ssh_key.desktop: Read complete after 0s [name=desktop]
data.hcloud_image.ubuntu: Read complete after 0s [name=debian-13]
OpenTofu used the selected providers to generate the following execution plan. Resource actions are indicated with
the following symbols:
+ create
OpenTofu will perform the following actions:
# hcloud_server.vms["devops01"] will be created
+ resource "hcloud_server" "vms" {
+ allow_deprecated_images = false
+ backup_window = (known after apply)
+ backups = false
+ datacenter = (known after apply)
+ delete_protection = false
+ firewall_ids = (known after apply)
+ id = (known after apply)
+ ignore_remote_firewall_ids = false
+ image = "310554929"
...
...
Plan: 3 to add, 0 to change, 0 to destroy.
Changes to Outputs:
+ vm_ips = {
+ devops01 = (known after apply)
+ devops02 = (known after apply)
+ devops03 = (known after apply)
}
Do you want to perform these actions?
OpenTofu will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
hcloud_server.vms["devops02"]: Creating...
hcloud_server.vms["devops01"]: Creating...
hcloud_server.vms["devops03"]: Creating...
hcloud_server.vms["devops01"]: Still creating... [10s elapsed]
hcloud_server.vms["devops03"]: Still creating... [10s elapsed]
hcloud_server.vms["devops02"]: Still creating... [10s elapsed]
hcloud_server.vms["devops01"]: Creation complete after 19s [id=117722922]
hcloud_server.vms["devops02"]: Still creating... [20s elapsed]
hcloud_server.vms["devops03"]: Still creating... [20s elapsed]
hcloud_server.vms["devops02"]: Creation complete after 26s [id=117722920]
hcloud_server.vms["devops03"]: Still creating... [30s elapsed]
hcloud_server.vms["devops03"]: Creation complete after 35s [id=117722921]
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
Outputs:
vm_ips = {
"devops01" = "2a01:4f8:1c1f:88df::1"
"devops02" = "2a01:4ff:f0:7df7::1"
"devops03" = "2a01:4f9:c013:f6e0::1"
}
anurag@desktop ~/R/P/c/a/test (main)>

In the Hetzner console for this project, three VMs appear:

Reaching devops01 -> devops02

root@devops01:~# tailscale ping devops02
pong from devops02 (100.68.146.42) via [2a01:4ff:f0:7df7::1]:41641 in 111ms
root@devops01:~#
root@devops01:~# ping -c 5 devops02
PING devops02.tail362a2.ts.net (100.116.50.72) 56(84) bytes of data.
64 bytes from devops02.tail362a2.ts.net (100.116.50.72): icmp_seq=1 ttl=64 time=106 ms
64 bytes from devops02.tail362a2.ts.net (100.116.50.72): icmp_seq=2 ttl=64 time=106 ms
64 bytes from devops02.tail362a2.ts.net (100.116.50.72): icmp_seq=3 ttl=64 time=106 ms
64 bytes from devops02.tail362a2.ts.net (100.116.50.72): icmp_seq=4 ttl=64 time=106 ms
64 bytes from devops02.tail362a2.ts.net (100.116.50.72): icmp_seq=5 ttl=64 time=106 ms
--- devops02.tail362a2.ts.net ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 105.735/106.012/106.243/0.197 ms
root@devops01:~#

It can reach devops02 over IPv6 transport.

NCMC design and technical limitations

Sun, 04 Jan 2026 00:16:00 +0530

I was in Mumbai a few months ago for the Equinix India Peering forum. The event happened to be very near the airport & I found that the airport, conference venue, hotel, and a few other locations I was visiting were all near the newly built Mumbai Line 3 (Aqua Line). To save on hassle with regular tickets, I went for an NCMC (National Common Mobility Card). Most people travelling in the Delhi metro would be aware of it by now, thanks to a bit of marketing push inside the metro train with announcements. Mumbai line 3 has a tie-up with SBI for the card, and thus, I got the SBI card. Ended up using it in Mumbai, Delhi and Chennai over the last few months.

Lately, I have been reading about way NCMC works, the limitations of the Indian NCMC design compared to other mass transit systems. In many countries, mass transit systems simply use MasterCard/Visa/Apple Pay tap and charge directly from the respective card (credit/debit card, etc.), like the London tube, Stockholm metro, HK airport express, Singapore, etc. Indian deployment is bit different.

Indian NCMC offline wallet concept

The whole idea of NCMC is to have a single card which can make payment for public transport, including metro, buses etc across the country digitally. It has to be fast and reliable, and this is where the whole tech differs from systems like UPI or Visa/MasterCard/Rupay POS payments. In case of UPI as well as credit/debit card payments on POS machines - entire transaction runs online. Money goes from the person paying the amount to the receiver as part of a settlement facilitated by NPCI. I don’t need to go in detailed steps involved there, but the key idea is that there’s an active involvement of 1) Payer’s APP 2) Payer’s bank 3) Receiver’s PSP 4) Receiver’s bank 5) NPCI UPI switch, etc. All these have to work and most important - internet has to work. 😀

All this largely works well for casual payments across the vendors, but not for transit payments, especially at metro stations. It easily takes 2-3 seconds in best case scenario for these online payments, which can result in a massive queue at the metro station gate, besides the risk of it not working at all if “the internet goes down” or the app freezes and whatnot. To deal with these issues, most of the mass transit systems work in hybrid mode, where the money is collected locally offline, and details of such transactions are uploaded in bulk transactions (which, if delayed, do not cause any major issue). A regular Delhi (or any other) metro card follows that. When money is added to the card, it is literally added to the card via an NFC operation with cryptography in place. One can add money to that card online but it won’t reflect unless a sync operation is performed at the metro station where online balance is literally written to the card. While there is of course UPI support where tickets can be bought online using UPI & processed by the gate via QR code but anyone who has used that would know it’s very slow. There has been some development at the UPI front with UPI Lite X supporting local on-device wallet with the option of offline transaction, but yet to see it in action. NCMC is much ahead compared to UPI Lite X for now.

NPCI seems to have taken these same concepts of offline wallet into a rupay card. So one can have a prepaid/debit/credit Rupay card from any (supported) bank and can request the creation of a “service area” in the card, and then an offline wallet balance can be stored in that service area. So while this ensures there remains a single card for transit across India (which I personally find very useful) but it’s not a simple “you can use your Rupay card to tap and pay” concept that exists in some countries, etc. To pay for transit, existing debit/credit card one must have support on the card itself, the bank, metro station (to create a service area). That’s quite a lot of moving parts and feels a bit fragile. Adding insulting to injury - NCMC is often markted as card which can pay outside of transit systems as well on the POS machines. Credit/Debit could anyway do that but this projection is often made for the prepaid cards & all this adds to chaos KYC & extra steps.

Direct use of Visa/MasterCard/Apple Pay

While the Indian system doesn’t support Visa/MasterCard/Apple Pay for transit payments and rightfully so as the early days of Russia-Ukriane conflict have taught us, with Visa/Mastercard suspending their operations in Russia, besides the high MDR rates. But concept-wise, these implementation are interesting. No special service account, no special permissions - just directly tap the card & go. And ofcourse they are not processing these payments in real time either (same latency/uptime/performance challenge). What they do instead is that their system records each tap, and at the end of the day they do a batch settlement with the card issuer. The fun part implementation comes in how they detect fraud. Imagine someone tapping a debit card with no balance in a bank account or a credit card with no credit limit - they would let such a card pass initially, but once the system detects it during the batch settlement, they will simply block it and upload the blocked card data in batch operation across all terminals. Thus frauds can happen, but is largely contained. Unsure why NPCI didn’t go for a similar design with Rupay cards.

NCMC is the default new card

Due to the “service account creation” process, use of Rupay debit/credit didn’t really picked up at metro stations, but prepaid NCMC with pre-created service area has become de-facto for the newly issued cards. Logically, that makes sense as that removes the metro operator from payment handling & puts an RBI authorized full fledged bank or payments bank to deal with the balance money. Delhi Metro went with Paytm initially and later moved to Airtel NCMC as RBI imposed restrictions on Paytm. Mumbai metro went for SBI, and many state transport systems did their own partership for issuing these cards.

Offline balance limitations

Since these NCMCs are holding an offline balance, there are some interesting and weird limitations in the way they work.

Similar to older non-NCMC metro cards, these can be recharged online but a “sync” operation has to be performed either by metro station staff or newly installed machines or with the limited support in the Android app (Apple’s NFC is still closed & yet not supported). This adds a bottleneck at busy stations.
If a card is lost, the bank either cannot refund the balance at all or can only do so after the card actually expires. This is because the card has an offline balance & there’s no way for the bank to claim & credit it back unless the card cryptographically expires.
For a credit/debit card, this is an entirely different account unrelated to their primary “online” balance. Money can be moved from an online balance to an offline balance, but again would require a sync operation by an NFC terminal.
The apps/websites connected to an NCMC gets updated with a delay of days as the payments are processed.
While the app/websites take time to update, card themselves hold recent transactions locally. I noticed this when I paid via NCMC in Chennai for a bus to the airport, that same transaction was visible at the DMRC terminal in Delhi metro, but not one the bank portal for the next few days.

This talk from Sandesh Kunder (NCPI) at Global Fintech Fest 23 covers the concept as well as ideas from NPCI’s point of view.

Fragile system

While there is a conceptual idea of full compatibility across NCMC - cards, terminals, etc., it’s not smooth. E.g initially I got “ECH_0007” error on DMRC terminals when doing a sync operation on the Mumbai metro-issued SBI card. It’s hard to know what went wrong since so many people were involved. Thus I could not move “online balance” which I paid via an UPI app to the card’s offline balance.

I think at some point, NPCI will ditch this overly complicated system and will bring NCMC version 2 with a direct charge option for the Rupay card. That, along with a mix of tech like UPI Lite X with NFC, will act as a reliable, fast option for transit payments. Remember there are over 600 million rupay debit cards alone. Making them directly compatible while keeping single “online” balance which can be settled in batch operations.

Personal blog of Anurag Bhatia

IP Telephony in India: From regulation to reality

History

Ringo Vs telcos

Present situation

Regulation

Market

My setup

My son & his fixedline phone

Grey areas

Technical limitations

References

Airtel (AS9498) to take the lead on the RPKI RoV front

How many of these invalids are single-homed vs multi-homed?

Indian backbones need AS_PATH filters

BGP route leak

AS_PATH filters / Peer lock

Incremental backup on AWS Glacier with rustic

Rustic and cold storage

Demo

1) Create repo

2) Create rustic config profile

3) Generate password

4) Initial rustic repo

5) Generating random files

6) Trigger a backup

7) Checking backed-up snapshots

8) Checking hot & cold repo storage size

9) Data restore

Use case

Limitations

Fascinating history of transit-free tier 1 networks

Before Nov 2005, Nov 2005 and till now

Limitations

History of transit-free / tier 1 networks

Notes on AS174 / Cogent / PSI

Interesting history Verio -> NTT

Transit but cannot transit!

Feedback & updates

Use of VRF in dual WAN setup

Old Setup

Understanding VRF

New Setup

Beyond 5G: The Two Biggest Challenges for India's Internet

1. Every ISP builds its own last mile

2. Wi-Fi is surprisingly difficult to share legally

Centralised wifi authentication & logging

RFC8950: Announce IPv4 with an IPv6 next-hop

How does it work?

Hardware support

Demo lab

Router Configs

Telegram BGP hijack due to weird blackholing config

Analysis: Possible reason and impact of this behaviour

Note

Telegram prefixes hijack by Rcom AS18101

Likely was not intentional and here’s why:

Technically what failed?

Update: 17 June 2026 - 01:47 IST

Mapping major CDNs across the globe

Mapping logic

Global stats

ASN-CDN Mapping table

Map

Indian summary

Notes on the accuracy of data

Measuring IPv6 deployment on Indian Govt websites

Result

Airtel-Tata Comm routing is cold potato in India?

Hot potato routing is common

Airtel - Tata Comm routing in India

Why it’s cold potato routing?

Strange route with Jio and GTT as adjacency

Background

Who possibly caused it?

Netflow for home router & Linux servers

Some data

Who can light fibre in India?

Friend and his two homes…

Understanding licensing