01 Oct

IPv6 Only Web Hosting

Saw this excellent presentation in UKNOF 34 by¬†Peter Stevens from Mythic Beasts. Really enjoyed the challenges and fixes he shared in running an IPv6 only web hosting. A must watch for geeks ūüôā

 

Also, UKNOF & NLNOG both seem to have excellent content in their conferences along with professional video recording which they make available over YouTube channels.

 

20 Sep

IPv6 allocations to downwards machine with just one /64

IPv6

One of my friend went for a VM with a German hosting provider. He got single IPv4 (quite common) and a /64 IPv6. Overall /64 per VM/end server used to be ok till few years back but now these days running applications inside LXC containers (OS level virtualization) make more sense. This gives option to maintain separate hosting environment for each application. I personally do that a lot and infect blog which you are reading right now itself is on a LXC container.

anurag@server7:~$ sudo lxc-ls -f |grep websrv1
[sudo] password for anurag:
websrv1.server7.core.anuragbhatia.com RUNNING 1 - 10.20.70.3 2402:b580:1:4:1:1:1:1, 2402:b580:1:4::abcd
anurag@server7:~$

 

So my friend tried to do similar setup but it went tricky for him because of just one single /64 from upstream. For me I have a /32 and I originate a /48 from this location giving me over 65k /64s of IPv6 for any testing and random fun application deployments.

 

The challenge in his setup was following: 

  1. One can use available¬†18 quintillion IPv6 address in the /64 by bridging the internal container interface with it. That’s ok¬†for IPv6 but fails terribly for IPv4 as many people do not need dedicated IPv4 per container while it’s fun to have that for IPv6 and gives so much flexibility. For IPv4 a custom setup makes more sense with specific DST NAT and reverse proxy for port 80 and port 443 traffic.
  2. For NATing IPv4 a separate virtual interface (veth) makes sense so that one can run private IPv4 addressing. Now here firstly subnetting of /64 sounds stupid and weird but even if one does that it won’t work because main /64 allocation is via layer 2 and not a routed pool. This doesn’t works, read further on why.

 

workaround

So after our discussion my friend decided to use a /112 for container (ugly I know but datacenter provider quoted 4-5Euro/month for additional /64!). A /112 out of 128 IPv6 addressing gives one 2^16 i.e 65k IPv6 addresses to use on containers which is good number of IPv6 with few limitations like:

  1. Many things support /64 only like for instance use of IPv6 in OpenVPN sticks with that due to Linux Kernel implentation.
  2. IPv6 auto conf heavily depends on it. In my own personal setup I have a dedicated /64 for the container interfaces and radvd takes care of autoconfig via router advertisements. With anything less then /64 that’s not possible.

 

So we broke the allocated /64 into a /112 and allocated first IP our of that on veth based interface and next used 2nd IP on a container. IPv4 was working fine on container with SRC NAT in this case but IPv6 connectivity wasn’t. Container was able to reach host machine but nothing beyond that. I realised issue was of layer 2 based allocation which was relying on IPv6¬†NDP. So the container’s IPv6 had internal reachability with host machine but whenever any packet came from internet, the L3 device of VM provider wasn’t able to send packets further because of missing entry of that IP in their NDP table. Problem wasn’t just with IPv6 of container but with just any IPv6 used on any interface of the VM (whether that virtual veth or even loopback). Adding IPv6 on eth0 (which was connected to upstream device) was making IPv6 to work but not possible to use it further on a downstream device like a container. The datacenter provider offered to split /64 into /65s and route 2nd /65 for a monthly charge (ugly!!!).¬†So we ended up with a nasty workaround – use of proxy NDP. This is very similar concept to proxy arp as in case of ¬†IPv4. So that required enabling proxy arp by enabling in sysctl.conf and next doing proxy NDP for specific IPv6 using: ip neigh add proxy xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx:xxxx/64 dev eth0

 

Hurricane Electric datacenter

 

This works and thus with an extra step of adding proxy NDP entry for each IPv6 in use. In general routed pool is way better and if I had a make a choice on behalf of his datacenter provider, I would have gone for use of /64 for point to point connectivity and a routed /48. At Hurricane Electric (company I work for) we offer IPv6 free of charge so that networks can grow without having to worry about address space or do nasty things like one I described above. ūüėČ

Haven’t deployed IPv6 on your application yet? Do it now!

Time to get back to work and do more IPv6 ūüôā

27 Mar

Letsencrypt – Free signed automated SSL

Last year a really good project Letsencrypt came up. They key objective of this project is to help in securing web by pushing SSL everywhere.

 

Two key cool features

  1. It offer free signed SSL certs!
  2. It helps in setting up SSL via an agent seamlessly without having to deal with CSR, getting it signed & updating web server configuration.

 

At this stage Letsencrypt is itself a Certificate Authority¬†and but it’s root certs are yet not in the browser. It’s probably going to take a while till all major browsers get their certificate.

To help on that one of it’s sponsors IdenTrust has signed their intermediate certs. Hence certs signed by Letsencrypt are accepted by all browsers right away. All certs signed by Letsencypt are signed by Letencrypt Authority X1 which have signature from DST Root CA X3 which is accepted by pretty much all popular¬†browsers. You can read more about How it works here.

 

Here’s an example of SSL setup for say “demo.anuragbhatia.com” test domain which is already up and working without SSL. http://demo.anuragbhatia.com shows a plain text page. This is Apache running on Ubuntu server.

The Apache web config is pretty straightforward.

<VirtualHost *:80>
ServerName demo.anuragbhatia.com
DocumentRoot /var/www/demo.anuragbhatia.com
ErrorLog /var/log/apache2/demo.anuragbhatia.com
LogLevel notice
</VirtualHost>

 

 

Step 1 – Grab the Letscrypt agent

git clone https://github.com/letsencrypt/letsencrypt

 

Step 2 – Execute the auto script

./letsencrypt-auto –help

 

This will grab all needed dependencies and will get the agent working.

 

Step 3 – Execute Letsencrypt auto script with it’s Apache plugin

./letsencrypt-auto –apache -d demo.anuragbhatia.com

 

It takes with a quick wizard and in the end I get:

Congratulations! You have successfully enabled
https://demo.anuragbhatia.com

You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=demo.anuragbhatia.com

 

And it’s done!

Wizard got me a signed SSL and installed it in the apache config as well.

Screen Shot 2016-03-27 at 7.22.21 PM

 

Screen Shot 2016-03-27 at 7.22.37 PM

 

The agent created an addional Apache config with name demo.anuragbhatia.com-le-ssl.conf with following content

<IfModule mod_ssl.c>
<VirtualHost *:443>
ServerName demo.anuragbhatia.com
DocumentRoot /var/www/demo.anuragbhatia.com
ErrorLog /var/log/apache2/demo.anuragbhatia.com
LogLevel notice
SSLCertificateFile /etc/letsencrypt/live/demo.anuragbhatia.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/demo.anuragbhatia.com/privkey.pem
Include /etc/letsencrypt/options-ssl-apache.conf
SSLCertificateChainFile /etc/letsencrypt/live/demo.anuragbhatia.com/chain.pem
</VirtualHost>
</IfModule>

 

Here¬†options-ssl-apache.conf plays an important role by using better security options. It’s config:

# Baseline setting to Include for SSL sites

SSLEngine on

# Intermediate configuration, tweak to your needs
SSLProtocol             all -SSLv2 -SSLv3
SSLCipherSuite          ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:AES:CAMELLIA:DES-CBC3-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!MD5:!PSK:!aECDH:!EDH-DSS-DES-CBC3-SHA:!EDH-RSA-DES-CBC3-SHA:!KRB5-DES-CBC3-SHA
SSLHonorCipherOrder     on
SSLCompression          off

SSLOptions +StrictRequire

# Add vhost name to log entries:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\"" vhost_combined
LogFormat "%v %h %l %u %t \"%r\" %>s %b" vhost_common

CustomLog /var/log/apache2/access.log vhost_combined
LogLevel warn
ErrorLog /var/log/apache2/error.log

# Always ensure Cookies have "Secure" set (JAH 2012/1)
#Header edit Set-Cookie (?i)^(.*)(;\s*secure)??((\s*;)?(.*)) "$1; Secure$3$4"

 

Some of the limitations 

  1. Signed SSL certs are valid only for 90 days and have to be renewed.
  2. Wildcard SSL certs are not supported yet.
  3. IPv6 is not supported in the autoconfig setup via client. One can always get certificate manually and use with IPv6 but agent is yet to support IPv6 (which I guess is from next month).

 

You can read more on their excellent documentation here and can also consider checking Presentation by Ashley Jones from PCH at SANOG on All TLS, all the time.

 

Have fun!