OpenStack logo

Collaboratively managing the OpenStack project with Puppet

Monty Taylor <mordred@inaugust.com>

Whoami

About this Presentation

Design Assumptions

Shameless Plug

OpenStack

Is open source software for building private and public clouds.

Projects

Servers

Back in the Day...

Typical First Steps

Complexity Rises

Investigation

Small Infrastructure

RANT

WHY IS CREATING USERS SO MUCH WORK???

class user::virtual {
  define localuser ($realname,$sshkeys='',$shell="/bin/bash") {
    group { $title:
      ensure => 'present'
    }

    user { $title:
      ensure  => "present",
      comment => $realname,
      home    => "/home/$title",
      shell   => $shell,
      gid     => $title,
      groups  => ['sudo','admin'],
      membership => 'minimum',
      managehome => true,  # creates the home directory (does not actually manage it)
      require => Group[$title],
    }
    
    file { "${title}_sshdir":
      name => "/home/$title/.ssh",
      owner => $title,
      group => $title,
      mode => 700,
      ensure => 'directory',
      require => User[$title],
    }
  

RANT

WHY IS CREATING USERS SO MUCH WORK???

    file { "${title}_keys":
      name => "/home/$title/.ssh/authorized_keys",
      owner => $title,
      group => $title,
      mode => 400,
      content => $sshkeys,
      ensure => 'present',
      require => File["${title}_sshdir"],
    }
  }
}
  

RANT

WHY IS CREATING USERS SO MUCH WORK???

  @user::virtual::localuser { 'mordred':
    realname => 'Monty Taylor',
    sshkeys  => "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyxfIpVCvZyM8BIy7r7WOSIG6Scxq4afean1Pc/bej5ZWHXCu1QnhGbI7rW3sWciEhi375ILejfODl2TkBpfdJe/DL205lLkTxAa+FUqcZ5Ymwe+jBgCH5XayzyhRPFFLn07IfA/BDAjGPqFLvq6dCEHVNJIui6oEW7OUf6a3376YF55r9bw/8Ct00F9N7zrISeSSeZXbNR+dEqcsBEKBqvZGcLtM4jzDzNXw1ITPPMGaoEIIszLpkkJcy8u/13GIrbAwNrB2wjl6Mzj+N9nTsB4rFtxRXp31ZbytCH5G9CL/mFard7yi8NLVEJPZJvAifNVhooxGN06uAiTFE8EsuQ== mtaylor@qualinost\n",
  }
  

RANT

WHY IS CREATING USERS SO MUCH WORK???

    realize (
      User::Virtual::Localuser['mordred'],
      User::Virtual::Localuser['corvus'],
      User::Virtual::Localuser['soren'],
      User::Virtual::Localuser['linuxjedi'],
      User::Virtual::Localuser['devananda'],
      User::Virtual::Localuser['clarkb'],
    )
  

Shameless Plug

First (weird) choice

Usage Pattern 0

Once you have that ...

Usage Pattern 1

Secrets Solution 0

Copy secret files to machine by hand, then reference

  file { '/usr/local/jenkins_jobs/jenkins_jobs.ini':
    owner => 'root',
    group => 'root',
    mode => 440,
    ensure => 'present',
    source => 'file:///root/secret-files/jenkins_jobs.ini',
    replace => 'true',
    require => File['/usr/local/jenkins_jobs']
  }

Copying Files

RANT

WHY IS INSTALLING PACKAGES FAIL???

class jenkins_jobs {
    package { 'python-yaml':
        ensure => present;
    }
    ...
}

class zuul {
    package { 'python-yaml':
        ensure => present;
    }
    ...
}

node "jenkins.openstack.org" {
    include zuul
    include jenkins_jobs
}
 

RANT

WHY IS INSTALLING PACKAGES FAIL???

 # A lot of things need yaml, be conservative requiring this package to avoid
 # conflicts with other modules.
 if ! defined(Package['python-yaml']) {
   package { 'python-yaml':
     ensure => "present",
 }

 

Perhaps package commands should be fundamental units, and should create an idempotent package installation set. You know, kinda like how apt and yum already work?

Shameless Plug

And then there were more

Now we have ...

Usage Pattern 2

heira

node 'wiki.openstack.org' {
  class { 'openstack_project::wiki':
    mysql_root_password => hiera('wiki_db_password'),
    sysadmins           => hiera('sysadmins'),
  }  
}

forge modules

RANT

WHY IS INSTALLING PACKAGES FAIL???

# Array of modules to be installed key:value is module:version.
 declare -A MODULES
 MODULES["openstackci-dashboard"]="0.0.4"
 MODULES["openstackci-vcsrepo"]="0.0.6"
 MODULES["puppetlabs-apache"]="0.0.4"
 MODULES["puppetlabs-apt"]="0.0.4"
 MODULES["puppetlabs-mysql"]="0.5.0"
 MODULES["saz-memcached"]="2.0.2"

for MOD in ${!MODULES[*]} ; do
  # If the module at the current version does not exist upgrade or install it.
  if ! echo $MODULE_LIST | grep "$MOD.*${MODULES[$MOD]}" >dev/null 2>&1
  then
    # Attempt module upgrade. If that fails try installing the module.
    if ! puppet module upgrade $MOD --version ${MODULES[$MOD]} >dev/null 2>&1
    then
      # This will get run in cron, so silence non-error output
      puppet module install $MOD --version ${MODULES[$MOD]} >dev/null
    fi
  fi
done
 

RANT

WHY IS INSTALLING PACKAGES FAIL???

Shameless Plug

dashboard

node 'puppet-dashboard.openstack.org' {
  class { 'openstack_project::dashboard':
    password => hiera('dashboard_password'),
    mysql_password => hiera('dashboard_mysql_password'),
    sysadmins => hiera('sysadmins'),
  }
}

build slaves

What next?/Usage Pattern 3

Development / Contributing

Thanks!

These slides available at: https://github.com/openstack-ci/publications