Cloudpipe is a method for connecting end users to their project instances in vlan mode.
The support code for cloudpipe implements admin commands (via nova-manage) to automatically create a vm for a project that allows users to vpn into the private network of their project. Access to this vpn is provided through a public port on the network host for the project. This allows users to have free access to the virtual machines in their project without exposing those machines to the public internet.
The cloudpipe image is basically just a linux instance with openvpn installed. It needs a simple script to grab user data from the metadata server, b64 decode it into a zip file, and run the autorun.sh script from inside the zip. The autorun script will configure and run openvpn to run using the data from nova.
It is also useful to have a cron script that will periodically redownload the metadata and copy the new crl. This will keep revoked users from connecting and will disconnect any users that are connected with revoked certificates when their connection is renegotiated (every hour).
Making a cloudpipe image is relatively easy.
# install openvpn on a base ubuntu image. # set up a server.conf.template in /etc/openvpn/
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
port 1194 proto udp dev tap0 up "/etc/openvpn/up.sh br0" down "/etc/openvpn/down.sh br0" persist-key persist-tun ca ca.crt cert server.crt key server.key # This file should be kept secret dh dh1024.pem ifconfig-pool-persist ipp.txt server-bridge VPN_IP DHCP_SUBNET DHCP_LOWER DHCP_UPPER client-to-client keepalive 10 120 comp-lzo max-clients 1 user nobody group nogroup persist-key persist-tun status openvpn-status.log verb 3 mute 20
# set up.sh in /etc/openvpn/
1 2 3 4 5 6 7
#!/bin/sh BR=$1 DEV=$2 MTU=$3 /sbin/ifconfig $DEV mtu $MTU promisc up /usr/sbin/brctl addif $BR $DEV
# set down.sh in /etc/openvpn/
1 2 3 4 5 6 7
#!/bin/sh BR=$1 DEV=$2 /usr/sbin/brctl delif $BR $DEV /sbin/ifconfig $DEV down
# download and run the payload on boot from /etc/rc.local
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36
#!/bin/sh -e # # rc.local # # This script is executed at the end of each multiuser runlevel. # Make sure that the script will "exit 0" on success or any other # value on error. # # In order to enable or disable this script just change the execution # bits. # # By default this script does nothing. ####### These lines go at the end of /etc/rc.local ####### . /lib/lsb/init-functions echo Downloading payload from userdata wget http://169.254.169.254/latest/user-data -O /tmp/payload.b64 echo Decrypting base64 payload openssl enc -d -base64 -in /tmp/payload.b64 -out /tmp/payload.zip mkdir -p /tmp/payload echo Unzipping payload file unzip -o /tmp/payload.zip -d /tmp/payload/ # if the autorun.sh script exists, run it if [ -e /tmp/payload/autorun.sh ]; then echo Running autorun.sh cd /tmp/payload sh /tmp/payload/autorun.sh else echo rc.local : No autorun script to run fi exit 0
# setup /etc/network/interfaces
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
# This file describes the network interfaces available on your system # and how to activate them. For more information, see interfaces(5). # The loopback network interface auto lo iface lo inet loopback # The primary network interface auto eth0 iface eth0 inet manual up ifconfig $IFACE 0.0.0.0 up down ifconfig $IFACE down auto br0 iface br0 inet dhcp bridge_ports eth0
# register the image and set the image id in your flagfile:
# you should set a few other flags to make vpns work properly:
When you use nova-manage to launch a cloudpipe for a user, it goes through the following process:
In vlan networking mode, the second ip in each private network is reserved for the cloudpipe instance. This gives a consistent ip to the instance so that nova-network can create forwarding rules for access from the outside world. The network for each project is given a specific high-numbered port on the public ip of the network host. This port is automatically forwarded to 1194 on the vpn instance.
If specific high numbered ports do not work for your users, you can always allocate and associate a public ip to the instance, and then change the vpn_public_ip and vpn_public_port in the database. This will be turned into a nova-manage command or a flag soon.
If the use_project_ca flag is set (required to for cloudpipes to work securely), then each project has its own ca. This ca is used to sign the certificate for the vpn, and is also passed to the user for bundling images. When a certificate is revoked using nova-manage, a new Certificate Revocation List (crl) is generated. As long as cloudpipe has an updated crl, it will block revoked users from connecting to the vpn.
The userdata for cloudpipe isn’t currently updated when certs are revoked, so it is necessary to restart the cloudpipe instance if a user’s credentials are revoked.
You can reboot a cloudpipe vpn through the api if something goes wrong (using euca-reboot-instances for example), but if you generate a new crl, you will have to terminate it and start it again using nova-manage vpn run. The cloudpipe instance always gets the first ip in the subnet and it can take up to 10 minutes for the ip to be recovered. If you try to start the new vpn instance too soon, the instance will fail to start because of a NoMoreAddresses error. If you can’t wait 10 minutes, you can manually update the ip with something like the following (use the right ip for the project):
euca-terminate-instances <instance_id> mysql nova -e "update fixed_ips set allocated=0, leased=0, instance_id=NULL where fixed_ip='10.0.0.2'"
You also will need to terminate the dnsmasq running for the user (make sure you use the right pid file):
sudo kill `cat /var/lib/nova/br100.pid`
Now you should be able to re-run the vpn:
nova-manage vpn run <project_id>
The keypair that was used to launch the cloudpipe instance should be in the keys/<project_id> folder. You can use this key to log into the cloudpipe instance for debugging purposes.
CloudPipe - Build a user-data payload zip file, and launch an instance with it.
Wrappers around standard crypto data elements.
Includes root and intermediate CAs, SSH key_pairs and x509 certificates.
Compute an md5 hash.
|Parameters:||fp (file) – File pointer to the file to MD5 hash. The file pointer will be reset to the beginning of the file before the method returns.|
|Returns:||the hex digest version of the MD5 hash|
Convert a ssh public key to openssl format Equivalent to the ssh-keygen’s -m option
Ensure the CA filesystem exists.
Get crl file for project.
Generate and sign a cert for user in project.
Revoke a cert by file name.
Revoke all project certs.
Revoke all user certs.
Revoke certs for user in project.
Encrypt text with an ssh public key.