Basic traffic engineering for maximising peering traffic
Hello world from Gujarat! This is my 3rd visit to Gujarat. :)
Coming to today’s post: I have noticed ISPs doing really crazy things to maximise traffic on peerings and IXPs. Some of those are bad and some are very bad. Additionally I came across this comment and thought to put this quick post.
Example of some bad ways to increase IXP traffic:
-
Using upstream’s ASN to keep AS path shorter (yes, believe me I have seen that!)
-
Prepending a few hundred times on the transit
-
Deaggregate prefixes all the way upto /24s in IPv4 and /48s in IPv6.
and more!
Some quick fundamental ways to maximise traffic on the peering (both direct PNIs as well as IXP peering):
Inbound traffic
-
Aggregate your prefixes. That is if your allocation is a /22, ensure you originate /22 to transit only. If there are multiple transits, announce /22s on both and then announce a /23 or /24 as needed to increase traffic on one or other link. DO NOT announce /24s on all!
-
Announce a second level aggregate at peerings. That is if allocation is /22, announce two corresponding /23s on peerings. If allocation is /20, announce /20 on transit and /21s on peerings.
-
Avoid prepends. They do not help much because for other side (which is sending you traffic), it will simply pick routes based on it’s own set of localpref and localpref wins over AS path length.
-
Setup bilateral BGP sessions over the IX fabric. Do not rely only on route servers because many large content networks do not announce their critical heavy traffic routes on the route server (due to risk of leak as well as lack of control) but do that over the bilateral sessions.
Outbound traffic
Plan 4 set of local preferences. Any number will do but for example:
| Local Preference | Purpose |
|---|---|
| 400 | Downstream customer routes |
| 300 | Routes learnt from PNI interconnects |
| 200 | Routes learnt from the IX peering |
| 100 | Routes learnt from the IP transit |
Additionally, it’s a good idea to use BGP communities internally. Logic there is to simply tag all the routes you are learning from anywhere whether it’s peer or transit or customer. Based on tag, one can have an export policy and easily announce those routes. That also helps in preventing route leaks specially in multi-homed customer setups significantly.