20 Sep

IPv6 allocations to downwards machine with just one /64

One of my friend went for a VM with a German hosting provider. He got single IPv4 (quite common) and a /64 IPv6. Overall /64 per VM/end server used to be ok till few years back but now these days running applications inside LXC containers (OS level virtualization) make more sense. This gives option to maintain separate hosting environment for each application. I personally do that a lot and infect blog which you are reading right now itself is on a LXC container.

anurag@server7:~$ sudo lxc-ls -f |grep websrv1
[sudo] password for anurag:
websrv1.server7.core.anuragbhatia.com RUNNING 1 - 2402:b580:1:4:1:1:1:1, 2402:b580:1:4::abcd

So my friend tried to do similar setup but it went tricky for him because of just one single /64 from upstream. For me I have a /32 and I originate a /48 from this location giving me over 65k /64s of IPv6 for any testing and random fun application deployments.
The challenge in his setup was following: 

  1. One can use available 18 quintillion IPv6 address in the /64 by bridging the internal container interface with it. That’s ok for IPv6 but fails terribly for IPv4 as many people do not need dedicated IPv4 per container while it’s fun to have that for IPv6 and gives so much flexibility. For IPv4 a custom setup makes more sense with specific DST NAT and reverse proxy for port 80 and port 443 traffic.
  2. For NATing IPv4 a separate virtual interface (veth) makes sense so that one can run private IPv4 addressing. Now here firstly subnetting of /64 sounds stupid and weird but even if one does that it won’t work because main /64 allocation is via layer 2 and not a routed pool. This doesn’t works, read further on why.

So after our discussion my friend decided to use a /112 for container (ugly I know but datacenter provider quoted 4-5Euro/month for additional /64!). A /112 out of 128 IPv6 addressing gives one 2^16 i.e 65k IPv6 addresses to use on containers which is good number of IPv6 with few limitations like:

  1. Many things support /64 only like for instance use of IPv6 in OpenVPN sticks with that due to Linux Kernel implentation.
  2. IPv6 auto conf heavily depends on it. In my own personal setup I have a dedicated /64 for the container interfaces and radvd takes care of autoconfig via router advertisements. With anything less then /64 that’s not possible.

So we broke the allocated /64 into a /112 and allocated first IP our of that on veth based interface and next used 2nd IP on a container. IPv4 was working fine on container with SRC NAT in this case but IPv6 connectivity wasn’t. Container was able to reach host machine but nothing beyond that. I realised issue was of layer 2 based allocation which was relying on IPv6 NDP. So the container’s IPv6 had internal reachability with host machine but whenever any packet came from internet, the L3 device of VM provider wasn’t able to send packets further because of missing entry of that IP in their NDP table. Problem wasn’t just with IPv6 of container but with just any IPv6 used on any interface of the VM (whether that virtual veth or even loopback). Adding IPv6 on eth0 (which was connected to upstream device) was making IPv6 to work but not possible to use it further on a downstream device like a container. The datacenter provider offered to split /64 into /65s and route 2nd /65 for a monthly charge (ugly!!!). So we ended up with a nasty workaround – use of proxy NDP. This is very similar concept to proxy arp as in case of  IPv4. So that required enabling proxy arp by enabling in sysctl.conf and next doing proxy NDP for specific IPv6 using: ip neigh add proxy xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64 dev eth0
Hurricane Electric datacenter
This works and thus with an extra step of adding proxy NDP entry for each IPv6 in use. In general routed pool is way better and if I had a make a choice on behalf of his datacenter provider, I would have gone for use of /64 for point to point connectivity and a routed /48. At Hurricane Electric (company I work for) we offer IPv6 free of charge so that networks can grow without having to worry about address space or do nasty things like one I described above. 😉
Haven’t deployed IPv6 on your application yet? Do it now!
Time to get back to work and do more IPv6 🙂

12 Sep

Three days without laptop / Repair of damaged Macbook Pro screen

In Hong Kong and right on the day when I was leaving for Hong Kong – I noticed a bug in my Macbook Pro’s screen. A real insect yes!
This was irritating. Hard to know how it actually got inside but may be from bottom side of laptop which was missing two screws for a while which I recently bought from UK and added. Apple support couldn’t answer on how it got there. Furthermore I was facing a problem of screen showing pink/green lines after turning on for almost 1.8 years now. They used to show up when laptop was turned on and used to go away after 15-20mins. Over time the issue increased and now it was taking as much as 40-50mins for lines to go away. Since I was visiting Hong Kong for HKNOG 3.0, I booked an appointment with Apple store. Fixing time went quick (over phone call with Apple support) and I was at Apple store in HK right on next day after an overnight flight to Hong Kong.
They dropped bill bomb by quoting me $5600 HKD / $720 USD / 48k INR !!!
While they told me of the amount they kept on running built in diagnostic tests to see if something else had any issue. I was there trying to count whether it makes sense to spend $720 USD on repair or rather just buy a new one. New one was 3x price of that and in current config I quite existing one. There wasn’t any other major reason to buy new one except screen issue. Furthermore re-sell value would be just zero for a laptop with insect inside screen. This is my three years old laptop and I used it almost for 12-16hrs every day on each day in last 3 years.
In meantime Apple store advisor told me (as part of regular diagnostic) they would check my screen for any issues with top coating and if they found any issue they would replace screen for free as Apple covers screen replacement for damaged coating even for devices outside warranty program and I was lucky it was. 🙂
So the good news for me was free replacement of screen and that would get me rid of bug inside screen beside green/pink lines but sad part was 3-5 days time they quoted for replacement. In real it took just 3 days and they were read with device by 3rd evening (a few hours ago – when I got my device back). You can read more about free screen replacement / Apple screen coating issues in this post and I can very much confirm that this is applicable. That saved me a hell lot of money and the hard choice of putting $720 on three year old device Vs new one.
Also I realised this was first time in three years when I was without laptop.
Day 0 – This was day when I deposited laptop. This day just went as I was deadly sleepy after a 9hrs non-sleepy flight and bit of travel inside HK.
Day 1 – This was HKNOG 3.0 conference day. I was on way in HK MTR (metro rail) to conference venue. I checked and edited my slides on my Nexus 5 smartphone in Google Sheets. It was OKish experience. Worked but definitely not fun. I very much missed laptop at that time. Did not miss laptop in daytime as was busy with conference. I could surely have worked during event and email replies could have been bit faster! But not much beyond that. Conference day usually goes busy in meeting people, and discussing about various things.
Day 2 – This was day when I checked out of hotel and checkin to hostel and terribly missed laptop. I did strained my eyes a bit while waiting for rain to stop outside by watching Suits Season 3 on my Android phone on Netflix. I even thought to download wordpress app on Android and post on my blog but realised I have done bit of fencing on blog to make it hard to connect! I did roamed around nearby in electronics market and even explored option of cheap laptop to run Linux and have it as backup laptop but I usually prefer single device and that stays very well customised and personalised.
Day 3 – It was Sunday and I roamed around a bit and had some serious plans while I cancelled as soon as I read email from Apple store with subject: “Your product is ready for pickup” and I just rushed to Apple store (which happens to be open on Sunday as well here). Collected laptop and started working on restoring OS (an hour of task via Time Machine backup).
For now laptop seems fixed and has got a new screen.
Here’s what bill says:
With that being said time to look at BGP routing table 🙂