Cloudflare DNS outage & BGP route hijack
Yesterday was quite an eventful day on the internet with an outage on Cloudflare’s public DNS - 1.1.1.1. One can say impact is high when even Indian media starts reporting it (like this and this). 😀
Source: Cloudflare community
Status doesn’t tell much and there has been speculation on BGP hijack by Tata Comm Indian network AS4755 causing it. Cloudflare’s own tool (unrelated to their DNS) reported that Tata Comm AS4755 originated 1.1.1.0/24 for a period of 7 minutes. The timeline here is interesting. Tata Comm’s hijack ended at 22:01 while Cloudflare reports issues with 1.1.1.1 at 22:13 but routing-wise it doesn’t make any sense. Hijack of Cloudflare prefix by Tata Comm cannot cause such a widespread outage.
For prefix hijack it always makes sense to look at the AS_PATH before coming to a conclusion as in theory one can hijack a prefix by using AS4755 in origin in the AS_PATH and if BGP adjacencies do not filter, they will pick it up. Let’s look for 1.1.1.0/24 via RIPE BGPPlay for this duration. This link should load BGPplay with an exact time filter.
RIPE RIS Collector RRC03 - Amsterdam shows 1.1.1.0/24 with path 6453 4755. This is a real route as AS6453 directly feeds RRC03.
Limited route propagation
While the announcement was bad but the impact would be extremely low and cannot cause a globally visible outage. Here’s why:
- Cloudflare AS13335 is directly connected to all major backbones and announcing 1.1.1.0/24 directly. The only exception to this is German Deutsche Telekom - DTAG AS3320. Most of them consider Cloudflare as a customer (some as a peer). All of them consider Tata Comm a peer. Thus they would either have high local pref on Cloudflare (where Cloudflare is downstream) compared to Tata Comm (a peer) or the same localpref on Cloudflare (a peer) vs Tata Comm (also a peer).
- RPKI validation would drop invalid hijacked announcement by many large networks.
- The majority of small to mid-sized networks are peered to Cloudflare directly & hence would pick that route over the route of Cloudflare coming via AS6453 > AS4755 path except a) Non-Cloudflare peer downstream networks of Tata Comm AS4755 in India and b) DTAG AS3320.
- Tata Comm would normally treat both - their domestic Indian network AS4755 and Cloudflare AS13335 as downstream of AS6453 and hence with the same local preference. Thus if AS13335 announcement is up, likely AS4755 route will not show up outside of India.
Domestic Indian routing with AS4755
This is a classic case of hijack of the route of a well-peered network (Cloudflare) by a transit-free network (selective peering) and hence quite tricky to detect. As long as the Cloudflare announcement is up, the majority of public collectors outside of India will not get it.
Since it’s from AS4755, route announcement at NIXI Chennai by AS4755 as picked by PCH collector on 14 July 2025 - route-collector.maa.pch.net:
Timestamp (GMT) | Type | BGP Adjacency | Prefix | AS_PATH | BGP Communities |
---|---|---|---|---|---|
9:50:07 PM | Announcement | 24029 | 1.1.1.0/24 | 24029 4755 13335 | 4755:25 4755:44 4755:199 4755:888 4755:2000 4755:40018 4755:47556 6453:9636 |
9:53:06 PM | Withdrawal | 24029 | 1.1.1.0/24 | ||
9:53:38 PM | Announcement | 24029 | 1.1.1.0/24 | 24029 4755 | 4755:44 4755:99 |
10:19:53 PM | Announcement | 24029 | 1.1.1.0/24 | 24029 4755 13335 | 4755:25 4755:44 4755:199 4755:888 4755:2000 4755:40018 4755:47556 6453:9636 |
10:19:55 PM | Announcement | 24029 | 1.1.1.0/24 | 24029 4755 13335 | 4755:25 4755:44 4755:199 4755:888 4755:2000 4755:40018 4755:47556 6453:9636 |
BGP community 4755:44 - 44 here is likely Chennai (STD code of Chennai - Tata Comm & Airtel both use STD codes in their BGP communities). This is a bit strange as there is clearly visible withdrawal of Cloudflare’s announcement at 9:53:06 PM and within 22 seconds there was origination by AS4755. This cannot be manual and as already guessed my friend Doug Madory in his tweet - I also think that AS4755 announcement was already there and was normally suppressed. It showed up when AS13335 pulled its announcement for whatever internal issue they had. Possibly Tata Comm AS4755 has high localpref on eBGP routes compared to locally originated routes. The majority of networks set high localpref on downstream routes but sometimes do not match localpref on locally originated routes.
Likely that announcement is still there inside AS4755 but just suppressed for now. In past, I have seen 1.1.1.1 Anycast broken on Tata Comm. It always routes it via Chennai. Wonder if it’s related. Hard to know for sure since the AS4755 looking glass has been broken for years.
Run your own DNS resolvers!
This is also a reminder to various network operators, datacenter and Cloud players - run your own DNS resolvers. There’s no good reason to use 1.1.1.1 or 8.8.8.8 fr your customers since there’s no SLA, no contact point involved with Cloudflare or Google. They may go down and worst - they can rate limit you anytime. A couple of unbound instances with anycast within your network can take you very far.
Disclaimer: This is my personal blog and hence posts made here are in my personal capacity. These do not represent the views of my employer.