Controller Node

 Introduction

The Controller node will provide :

  • Databases (with MySQL)

  • Queues (with RabbitMQ)

  • Keystone

  • Glance

  • Nova (without nova-compute)

  • Cinder

  • Quantum Server (with Open-vSwitch plugin)

  • Dashboard (with Horizon)

 Common services

 Operating System

  1. Install Debian Wheezy Ubuntu 12.04 or 13.04. The exact installation procedure is outside the scope of this document, but please note the following configurations:

    • Time zone: UTC

    • Hostname: cloud

    • Packages: OpenSSH-Server, wget

    Once installation has finished, the server will reboot.

  2. Since the default OpenStack release in Ubuntu 12.04 LTS is older, we are going to use the Ubuntu Cloud Archive for Grizzly:

    # apt-get install ubuntu-cloud-keyring

    Edit /etc/apt/sources.list.d/cloud-archive.list:

    deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/grizzly main

    Upgrade the system (and reboot if you need):

    # sudo apt-get update && apt-get upgrade

    Since all OpenStack packages have not been introduced into Debian official repositories, we can use a repository which supports Grizzly release. Edit /etc/apt/sources.list.d/grizzly.list:

    deb http://archive.gplhost.com/debian grizzly main
    deb http://archive.gplhost.com/debian grizzly-backports main

    Upgrade the system (and reboot if you need):

    # apt-get update
    # apt-get install gplhost-archive-keyring
    # apt-get upgrade

  3. Configure the network:

    • Edit /etc/network/interfaces:

      # Internal Network
      auto eth0
      iface eth0 inet static
          address 10.10.10.10
          netmask 255.255.255.0
      
      # External Network
      auto eth1
      iface eth1 inet static
          address 10.0.0.10
          netmask 255.255.255.0
          gateway 10.0.0.1
          dns-nameservers 8.8.8.8

    • Edit /etc/sysctl.conf:

      net.ipv4.conf.all.rp_filter = 0
      net.ipv4.conf.default.rp_filter = 0

      Then, restart the network service:

      # service networking restart

      And apply the sysctl settings:

      # sysctl -e -p /etc/sysctl.conf

    • Edit the /etc/hosts file and add cloud, network, and c01 hostnames with correct IP.

      127.0.0.1       localhost
      10.10.10.10     cloud
      10.10.10.9      network
      10.10.10.11     c01

      [Note]Note

      While manually specifying host entries is acceptable for a simple or testing environment, it is highly recommended to use proper DNS entries, or at a minimum a configuration management system such as Puppet, to maintain your IP to host mappings.

  4. Install NTP. NTP will ensure that the server has the correct time. This is important because if an OpenStack server's time is not correct, it will be removed from the rest of the cloud.

    • # apt-get install -y ntp

 MySQL Database Service

The various OpenStack components store persistent data in a relational database. MySQL is the most popular choice.

  1. Install the packages:

    # apt-get install -y python-mysqldb mysql-server

    [Note]Note

    apt-get will prompt you to set the MySQL root password.

  2. By default, MySQL will only accept connections from localhost. This needs changed so that the compute nodes can access the OpenStack Networking service. Database requests for the OpenStack Compute service are proxied through the nova-conductor service.

    # sed -i 's/127.0.0.1/0.0.0.0/g' /etc/mysql/my.cnf

  3. Restart the service:

    # service mysql restart

  4. The various databases that the OpenStack services require need created. Additionally, MySQL accounts to access those databases need created, too:

    # mysql -u root -p <<EOF
    CREATE DATABASE nova;
    GRANT ALL PRIVILEGES ON nova.* TO 'nova'@'localhost' \
    IDENTIFIED BY 'password';
    CREATE DATABASE cinder;
    GRANT ALL PRIVILEGES ON cinder.* TO 'cinder'@'localhost' \
    IDENTIFIED BY 'password';
    CREATE DATABASE glance;
    GRANT ALL PRIVILEGES ON glance.* TO 'glance'@'localhost' \
    IDENTIFIED BY 'password';
    CREATE DATABASE keystone;
    GRANT ALL PRIVILEGES ON keystone.* TO 'keystone'@'localhost' \
    IDENTIFIED BY 'password';
    CREATE DATABASE quantum;
    GRANT ALL PRIVILEGES ON quantum.* TO 'quantum'@'localhost' \
    IDENTIFIED BY 'password';
    GRANT ALL PRIVILEGES ON quantum.* TO 'quantum'@'10.10.10.9' \
    IDENTIFIED BY 'password';
    GRANT ALL PRIVILEGES ON quantum.* TO 'quantum'@'10.10.10.11' \
    IDENTIFIED BY 'password';
    FLUSH PRIVILEGES;
    EOF

  RabbitMQ Messaging Service

The OpenStack components also communicate through a queuing service. For example, the Cloud Controller places a request to launch an instance on the queue. The Compute Node then picks this request up and launches the instance. OpenStack can work with several different queuing services.

  1. Install the packages:

    # apt-get install -y rabbitmq-server

  2. Change the default password:

    # rabbitmqctl change_password guest password

    [Note]Note

    In addition to choosing another password in a production environment, you should also disable the guest account and use a proper RabbitMQ account. Please see the RabbitMQ documentation for further details.

 OpenStack Identity Service

The OpenStack Identity Service provides the cloud environment with an authentication and authorization system. In this system, users are a part of one or more projects. In each of these projects, they hold a specific role.

  1. Install the packages:

    # apt-get install -y keystone python-keystone python-keystoneclient

  2. Edit /etc/keystone/keystone.conf:

    [DEFAULT]
    admin_token = password
    debug = True
    verbose = True
    
    [sql]
    connection = mysql://keystone:password@localhost/keystone
  3. Restart Keystone and create the tables in the database:

    # service keystone restart
    # keystone-manage db_sync
                        

    [Note]Note

    Check the /var/log/keystone/keystone.log file for errors that would prevent the Identity Service from successfully starting.

  4. Create an openrc file:

    • Create a file called ~/openrc. This file contains the OpenStack admin credentials that are used when interacting with the OpenStack environment on the command line.

      export OS_TENANT_NAME=admin
      export OS_USERNAME=admin
      export OS_PASSWORD=password
      export OS_AUTH_URL="http://localhost:5000/v2.0/"
      export OS_SERVICE_ENDPOINT="http://localhost:35357/v2.0"
      export OS_SERVICE_TOKEN=password

    [Note]Note

    Best practice for bootstrapping the first administrative user is to use the OS_SERVICE_ENDPOINT and OS_SERVICE_TOKEN together as environment variables, then set up a separate RC file just for Identity administration that uses port 35357 for the OS_AUTH_URL. This example is meant to provide a quick setup, not an audit-able environment.

    • Source the credentials into your environment:

      source ~/openrc
    • Configure the Bash shell to load these credentials upon each login:

      echo "source ~/openrc" >> ~/.bashrc

  5. The following bash script will populate Keystone with some initial data:

    • Projects: admin and services

    • Roles: admin, Member

    • Users: admin, demo, nova, glance, quantum, and cinder

    • Services: compute, volume, image, identity, ec2, and network

    #!/bin/bash
    
    # Modify these variables as needed
    ADMIN_PASSWORD=${ADMIN_PASSWORD:-password}
    SERVICE_PASSWORD=${SERVICE_PASSWORD:-$ADMIN_PASSWORD}
    DEMO_PASSWORD=${DEMO_PASSWORD:-$ADMIN_PASSWORD}
    export OS_SERVICE_TOKEN="password"
    export OS_SERVICE_ENDPOINT="http://localhost:35357/v2.0"
    SERVICE_TENANT_NAME=${SERVICE_TENANT_NAME:-service}
    #
    MYSQL_USER=keystone
    MYSQL_DATABASE=keystone
    MYSQL_HOST=localhost
    MYSQL_PASSWORD=password
    #
    KEYSTONE_REGION=RegionOne
    KEYSTONE_HOST=10.10.10.10
    
    # Shortcut function to get a newly generated ID
    function get_field() {
        while read data; do
            if [ "$1" -lt 0 ]; then
                field="(\$(NF$1))"
            else
                field="\$$(($1 + 1))"
            fi
            echo "$data" | awk -F'[ \t]*\\|[ \t]*' "{print $field}"
        done
    }
    
    # Tenants
    ADMIN_TENANT=$(keystone tenant-create --name=admin | grep " id " | get_field 2)
    DEMO_TENANT=$(keystone tenant-create --name=demo | grep " id " | get_field 2)
    SERVICE_TENANT=$(keystone tenant-create --name=$SERVICE_TENANT_NAME | grep " id " | get_field 2)
    
    # Users
    ADMIN_USER=$(keystone user-create --name=admin --pass="$ADMIN_PASSWORD" --email=admin@domain.com | grep " id " | get_field 2)
    DEMO_USER=$(keystone user-create --name=demo --pass="$DEMO_PASSWORD" --email=demo@domain.com --tenant-id=$DEMO_TENANT | grep " id " | get_field 2)
    NOVA_USER=$(keystone user-create --name=nova --pass="$SERVICE_PASSWORD" --tenant-id $SERVICE_TENANT --email=nova@domain.com | grep " id " | get_field 2)
    GLANCE_USER=$(keystone user-create --name=glance --pass="$SERVICE_PASSWORD" --tenant-id $SERVICE_TENANT --email=glance@domain.com | grep " id " | get_field 2)
    QUANTUM_USER=$(keystone user-create --name=quantum --pass="$SERVICE_PASSWORD" --tenant-id $SERVICE_TENANT --email=quantum@domain.com | grep " id " | get_field 2)
    CINDER_USER=$(keystone user-create --name=cinder --pass="$SERVICE_PASSWORD" --tenant-id $SERVICE_TENANT --email=cinder@domain.com | grep " id " | get_field 2)
    
    # Roles
    ADMIN_ROLE=$(keystone role-create --name=admin | grep " id " | get_field 2)
    MEMBER_ROLE=$(keystone role-create --name=Member | grep " id " | get_field 2)
    
    # Add Roles to Users in Tenants
    keystone user-role-add --user-id $ADMIN_USER --role-id $ADMIN_ROLE --tenant-id $ADMIN_TENANT
    keystone user-role-add --tenant-id $SERVICE_TENANT --user-id $NOVA_USER --role-id $ADMIN_ROLE
    keystone user-role-add --tenant-id $SERVICE_TENANT --user-id $GLANCE_USER --role-id $ADMIN_ROLE
    keystone user-role-add --tenant-id $SERVICE_TENANT --user-id $QUANTUM_USER --role-id $ADMIN_ROLE
    keystone user-role-add --tenant-id $SERVICE_TENANT --user-id $CINDER_USER --role-id $ADMIN_ROLE
    keystone user-role-add --tenant-id $DEMO_TENANT --user-id $DEMO_USER --role-id $MEMBER_ROLE
    
    # Create services
    COMPUTE_SERVICE=$(keystone service-create --name nova --type compute --description 'OpenStack Compute Service' | grep " id " | get_field 2)
    VOLUME_SERVICE=$(keystone service-create --name cinder --type volume --description 'OpenStack Volume Service' | grep " id " | get_field 2)
    IMAGE_SERVICE=$(keystone service-create --name glance --type image --description 'OpenStack Image Service' | grep " id " | get_field 2)
    IDENTITY_SERVICE=$(keystone service-create --name keystone --type identity --description 'OpenStack Identity' | grep " id " | get_field 2)
    EC2_SERVICE=$(keystone service-create --name ec2 --type ec2 --description 'OpenStack EC2 service' | grep " id " | get_field 2)
    NETWORK_SERVICE=$(keystone service-create --name quantum --type network --description 'OpenStack Networking service' | grep " id " | get_field 2)
    
    # Create endpoints
    keystone endpoint-create --region $KEYSTONE_REGION --service-id $COMPUTE_SERVICE --publicurl 'http://'"$KEYSTONE_HOST"':8774/v2/$(tenant_id)s' --adminurl 'http://'"$KEYSTONE_HOST"':8774/v2/$(tenant_id)s' --internalurl 'http://'"$KEYSTONE_HOST"':8774/v2/$(tenant_id)s'
    keystone endpoint-create --region $KEYSTONE_REGION --service-id $VOLUME_SERVICE --publicurl 'http://'"$KEYSTONE_HOST"':8776/v1/$(tenant_id)s' --adminurl 'http://'"$KEYSTONE_HOST"':8776/v1/$(tenant_id)s' --internalurl 'http://'"$KEYSTONE_HOST"':8776/v1/$(tenant_id)s'
    keystone endpoint-create --region $KEYSTONE_REGION --service-id $IMAGE_SERVICE --publicurl 'http://'"$KEYSTONE_HOST"':9292' --adminurl 'http://'"$KEYSTONE_HOST"':9292' --internalurl 'http://'"$KEYSTONE_HOST"':9292'
    keystone endpoint-create --region $KEYSTONE_REGION --service-id $IDENTITY_SERVICE --publicurl 'http://'"$KEYSTONE_HOST"':5000/v2.0' --adminurl 'http://'"$KEYSTONE_HOST"':35357/v2.0' --internalurl 'http://'"$KEYSTONE_HOST"':5000/v2.0'
    keystone endpoint-create --region $KEYSTONE_REGION --service-id $EC2_SERVICE --publicurl 'http://'"$KEYSTONE_HOST"':8773/services/Cloud' --adminurl 'http://'"$KEYSTONE_HOST"':8773/services/Admin' --internalurl 'http://'"$KEYSTONE_HOST"':8773/services/Cloud'
    keystone endpoint-create --region $KEYSTONE_REGION --service-id $NETWORK_SERVICE --publicurl 'http://'"$KEYSTONE_HOST"':9696/' --adminurl 'http://'"$KEYSTONE_HOST"':9696/' --internalurl 'http://'"$KEYSTONE_HOST"':9696/'
                        

[Note]Note

If you make a mistake during this guide, you can reset the Keystone database by performing the following steps:

# mysql -u root -p -e "drop database keystone"
# mysql -u root -p -e "create database keystone"
# mysql -u root -p -e "grant all privileges on keystone.* TO 'keystone'@'localhost' identified by 'password'"
# keystone-manage db_sync

And finally, re-run the above bash script.

 OpenStack Image Service

The Image Service provides a catalog of virtual machine images from which you can launch instances.

For example, if a Ubuntu 12.04 image exists, you can use it to launch a Ubuntu 12.04 instance.

 

Procedure 2.1. To install and configure the OpenStack Image Service

  1. Install the OpenStack Image packages, as follows:

    # apt-get install glance

  2. Configure the OpenStack Image Service, as follows:

    1. The OpenStack Image Service provides the glance-api and glance-registry services. You configure these services identically. However, be aware that each provides a distinct service.

      Edit /etc/glance/glance-api.conf and /etc/glance/glance-registry.conf, as follows:

      [DEFAULT]
      sql_connection = mysql://glance:password@localhost/glance
      [keystone_authtoken]
      admin_tenant_name = service
      admin_user = glance
      admin_password = password
      [paste_deploy]
      flavor=keystone 
    2. Restart both services, as follows:

      # service glance-api restart && service glance-registry restart

      [Note]Note

      Check the /var/log/glance/*.log files for errors that cause the Image Service to fail to start.

    3. Create the OpenStack Image tables in the database, as follows:

      # glance-manage db_sync
    4. Download and import Ubuntu 12.04 LTS UEC image:

      $ wget http://uec-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img
      $ glance image-create --is-public true --disk-format qcow2 --container-format bare --name "Ubuntu" < precise-server-cloudimg-amd64-disk1.img                      

      Download and import the CirrOS QCOW2 Image:

      $ wget http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-disk.img
      $ glance image-create --is-public true --disk-format qcow2 --container-format bare --name "CirrOS 0.3.1" < cirros-0.3.1-x86_64-disk.img
    5. Check if the images have been introduced in the index:

      $ glance image-list
      +--------------------------------------+--------------+-------------+------------------+-----------+--------+
      | ID                                   | Name         | Disk Format | Container Format | Size      | Status |
      +--------------------------------------+--------------+-------------+------------------+-----------+--------+
      | acafc7c0-40aa-4026-9673-b879898e1fc2 | CirrOS 0.3.1 | qcow2       | bare             | 13147648  | active |
      | 62f9278e-a26e-4fa0-9537-1eb503aa2f01 | Ubuntu       | qcow2       | bare             | 253755392 | active |
      +--------------------------------------+--------------+-------------+------------------+-----------+--------+

 OpenStack Compute (Cloud Controller services)

The OpenStack Compute Service provides the cloud environment with the ability to manage the scheduling, creation and deletion of virtual machines (instances).

  1. Install the Nova packages:

    # apt-get install -y nova-api nova-cert nova-common nova-conductor \
        nova-scheduler python-nova python-novaclient nova-consoleauth novnc \
        nova-novncproxy

  2. Configure Nova:

    • Edit /etc/nova/api-paste.ini:

      admin_tenant_name = service 
      admin_user = nova 
      admin_password = password

    • Add the following to the /etc/nova/nova.conf file. This file is the main configuration file of Nova. There is a large amount of configuration options that can go in this file. This guide illustrates the minimum needed for a simple environment. Note that the nova.conf file supplied by your distribution will have some options already set. Leave them as-is.

      [DEFAULT]
      
      sql_connection=mysql://nova:password@localhost/nova
      my_ip=10.10.10.10
      rabbit_password=password
      auth_strategy=keystone
      
      # Networking
      network_api_class=nova.network.quantumv2.api.API
      quantum_url=http://10.10.10.10:9696
      quantum_auth_strategy=keystone
      quantum_admin_tenant_name=service
      quantum_admin_username=quantum
      quantum_admin_password=password
      quantum_admin_auth_url=http://10.10.10.10:35357/v2.0
      libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
      linuxnet_interface_driver=nova.network.linux_net.LinuxOVSInterfaceDriver  
      
      # Security Groups                                    
      firewall_driver=nova.virt.firewall.NoopFirewallDriver
      security_group_api=quantum                           
                                                           
      # Metadata                                           
      quantum_metadata_proxy_shared_secret=password          
      service_quantum_metadata_proxy=true                  
      metadata_listen = 10.10.10.10        
      metadata_listen_port = 8775                          
      
      # Cinder
      volume_api_class=nova.volume.cinder.API
      
      # Glance
      glance_api_servers=10.10.10.10:9292
      image_service=nova.image.glance.GlanceImageService
      
      # novnc
      novnc_enable=true             
      novncproxy_port=6080          
      novncproxy_host=10.0.0.10
      vncserver_listen=0.0.0.0      
      

    • Create Nova tables into the database:

      # nova-manage db sync
    • Restart Nova services:

      # service nova-api restart
      # service nova-cert restart
      # service nova-consoleauth restart
      # service nova-scheduler restart
      # service nova-novncproxy restart

      [Note]Note

      Check the /var/log/nova/nova-* files for any errors that would prevent the Compute Service from successfully starting.

 OpenStack Block Storage

While Cinder contains many different storage drivers, the most common and basic configuration uses LVM and iSCSI. This guide illustrates how to use one disk (/dev/sdb) in an LVM Volume Group called cinder-volumes. When a user requests a block storage volume, a Logical Volume is created from this Volume Group and then mounted on the user's instance by way of iSCSI.

  1. Install the Cinder packages:

    # apt-get install -y cinder-api cinder-scheduler cinder-volume iscsitarget \
        open-iscsi iscsitarget-dkms python-cinderclient linux-headers-`uname -r`

  2. Configure & start the iSCSI services:

    # sed -i 's/false/true/g' /etc/default/iscsitarget
    # service iscsitarget start
    # service open-iscsi start

  3. Configure Cinder:

    • Edit /etc/cinder/cinder.conf:

      [DEFAULT]
      sql_connection = mysql://cinder:password@localhost/cinder
      rabbit_password = password

    • Edit /etc/cinder/api-paste.ini:

      admin_tenant_name = service
      admin_user = cinder 
      admin_password = password

    • Create the LVM Physical Volume and Logical Volume:

      # pvcreate /dev/sdb
      # vgcreate cinder-volumes /dev/sdb

    • Create Cinder tables into the database:

      # cinder-manage db sync

    • Restart the services:

      # service cinder-api restart
      # service cinder-scheduler restart
      # service cinder-volume restart

 OpenStack Network Service (Cloud Controller)

The OpenStack Network Service provides a comprehensive and extendible networking service to the cloud. Some features include, but are not limited to, the ability for instances to reach an external network outside of the cloud as well as the ability for each user of the cloud to create multiple internal subnets of their own.

  1. Install the Quantum Server:

    # apt-get install -y quantum-server

  2. Configure the Quantum service:

    • Edit /etc/quantum/quantum.conf:

      [DEFAULT]
      verbose = True
      rabbit_password = password
      [keystone_authtoken]
      admin_tenant_name = service
      admin_user = quantum 
      admin_password = password

    • Edit /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini:

      [DATABASE]
      sql_connection = mysql://quantum:password@localhost/quantum
      [OVS]
      tenant_network_type = gre 
      tunnel_id_ranges = 1:1000
      enable_tunneling = True
      local_ip = 10.10.10.10
      [SECURITYGROUP]
      firewall_driver = quantum.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver

      [Note]Note

      It's more handy to choose tunnel mode since you don't have to configure your physical switches for VLANs.

  3. Enable the OVS plugin:

    # ln -s /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini /etc/quantum/plugin.ini

  4. Start the services:

    # service quantum-server restart

 OpenStack Dashboard

The OpenStack Dashboard service provides users of the cloud environment with a web-accessible GUI as an alternative to using the command-line tools.

To enable it, install the Horizon package and its dependencies:

# apt-get install -y openstack-dashboard memcached python-memcache

[Note]Note

Optional, but recommended: remove the openstack-dashboard-ubuntu-theme package. This theme prevents several menus as well as the network map from rendering correctly:

# apt-get remove --purge openstack-dashboard-ubuntu-theme

OpenStack Dashboard is now available at http://10.10.10.10/horizon. We can login with the admin / password credentials or demo / password.

[Note]Note

Check the /var/log/apache2/error.log file for errors that wold prevent either the Apache service or the Dashboard service from successfully starting.

Log a bug against this page