Ansible config-download differences

With the Queens release, it became possible to use Ansible to apply the overcloud configuration and this method became the default behavior with the Rockt release.

The feature is fully documented at TripleO config-download User’s Guide: Deploying with Ansible, while this page details the differences to the deployer experience with config-download.

Ansible vs. os-collect-config

Previously, TripleO used an agent running on each overcloud node called os-collect-config. This agent periodically polled the undercloud Heat API for software configuration changes that needed to be applied to the node.

os-collect-config ran os-refresh-config and os-apply-config as needed whenever new software configuration changes were detected. This model is a “pull” style model given each node polled the Heat API and pulled changes, then applied them locally.

With config-download, TripleO has switched to a “push” style model. Ansible is run from a central control node which is the undercloud. ansible-playbook is run from the undercloud and software configuration changes are pushed out to each overcloud node via ssh.

With the new model, os-collect-config, os-refresh-config, and os-apply-config are no longer used in a TripleO deployment. The os-collect-config service is now disabled by default and won’t start on boot.

Note

Heat standalone software deployments still rely on os-collect-config. They are a type of deployment that can be applied to overcloud nodes directly via Heat outside of the overcloud stack, and without having to do a full stack update of the overcloud stack.

These types of deployments are NOT typically used when doing TripleO.

However, if these deployments are being used in an environment to manage overcloud nodes, then the os-collect-config service must be started and enabled on the overcloud nodes where these types of deployments are applied.

For reference, the Heat CLI commands that are used to create these types of deployments are:

openstack software config create ...
openstack software deployment create ...

If these commands are not being used in the environment, then os-collect-config can be left disabled.

Deployment workflow

The default workflow executed by openstack overcloud deploy takes care of all the necessary changes when using config-download. In both the previous and new workflows, openstack overcloud deploy (tripleoclient) takes care of automating all the steps through Mistral workflow(s). Therefore, existing CLI scripts that called openstack overcloud deploy will continue to work with no changes.

It’s important to recognize the differences in the workflow to aid in understanding the deployment and operator experience. Previously, Heat was responsible for:

  1. (Heat) Creating OpenStack resources (Neutron networks, Nova/Ironic instances, etc)

  2. (Heat) Creating software configuration

  3. (Heat) Applying the created software configuration to the Nova/Ironic instances

With config-download, Heat is no longer responsible for the last item of applying the created software configuration as ansible-playbook is used instead.

Therefore, only creating the Heat stack for an overcloud is no longer all that is required to fully deploy the overcloud. Ansible also must be run from the undercloud to apply the software configuration, and do all the required tasks to fully deploy an overcloud such as configuring services, bootstrap tasks, and starting containers.

The new steps are summarized as:

  1. (Heat) Creating OpenStack resources (Neutron networks, Nova/Ironic instances, etc)

  2. (Heat) Creating software configuration

  3. (tripleoclient) Enable tripleo-admin ssh user

  4. (ansible) Applying the created software configuration to the Nova/Ironic instances

See TripleO config-download User’s Guide: Deploying with Ansible for details on the tripleo-admin ssh user step.

Deployment CLI output

During a deployment, the expected output from openstack overcloud deploy has changed. Output up to and including the stack create/update is similar to previous releases. Stack events will be shown until the stack operation is complete.

After the stack goes to CREATE_COMPLETE (or UPDATE_COMPLETE), output from the steps to enable the tripleo-admin user via ssh are shown.

Deployment Output

After the tripleo-admin user is created, ansible-playbook will be used to configure the overcloud nodes.

The output from ansible-playbook will begin to appear in the console and will be updated periodically as more tasks are applied.

When ansible is finished a play recap will be shown, and the usual overcloudrc details will then be displayed. The following is an example of the end of the output from a successful deployment:

PLAY RECAP ****************************************************************
compute-0                  : ok=134  changed=48   unreachable=0    failed=0
openstack-0                : ok=164  changed=28   unreachable=0    failed=1
openstack-1                : ok=160  changed=28   unreachable=0    failed=0
openstack-2                : ok=160  changed=28   unreachable=0    failed=0
pacemaker-0                : ok=138  changed=30   unreachable=0    failed=0
pacemaker-1                : ok=138  changed=30   unreachable=0    failed=0
pacemaker-2                : ok=138  changed=30   unreachable=0    failed=0
undercloud                 : ok=2    changed=0    unreachable=0    failed=0

Overcloud configuration completed.
Overcloud Endpoint: http://192.168.24.8:5000/
Overcloud rc file: /home/stack/overcloudrc
Overcloud Deployed

When a failure happens, the deployment will stop and the error will be shown.

Review the PLAY RECAP which will show each host that is part of the overcloud and the grouped count of each task status.

Deployment Status

Since Heat is no longer the source of authority on the status of the overcloud deployment, a new tripleoclient command is available to show the overcloud deployment status:

openstack overcloud status

The output will report the status of the deployment, taking into consideration the result of all the steps to do the full deployment. The following is an example of the output:

[stack@undercloud ]$ openstack overcloud status

+-----------+---------------------+---------------------+-------------------+
| Plan Name |       Created       |       Updated       | Deployment Status |
+-----------+---------------------+---------------------+-------------------+
| overcloud | 2018-05-03 21:24:50 | 2018-05-03 21:27:59 |   DEPLOY_SUCCESS  |
+-----------+---------------------+---------------------+-------------------+

A different plan name can be specified with --plan:

[stack@undercloud ]$ openstack overcloud status --plan my-deployment

+---------------+---------------------+---------------------+-------------------+
| Plan Name     |       Created       |       Updated       | Deployment Status |
+-----------+-------------------------+---------------------+-------------------+
| my-deployment | 2018-05-03 21:24:50 | 2018-05-03 21:27:59 |   DEPLOY_SUCCESS  |
+---------------+---------------------+---------------------+-------------------+

Deployment failures can also be shown with a new command:

[stack@undercloud ]$ openstack overcloud failures --plan my-deployment

Note

Heat CLI commands such as openstack stack failures list can still be used to show stack failures, however since Heat no longer applies software configuration, it will no longer show any errors related to configuration.

Setting the status

The status of the deployment will be automatically set by the API used by the Mistral workflows. However, in some cases, it may be required to manually set the status to reflect what has been done manually outside of the API. The following commands can be used to manually set the status.

Set the status to DEPLOY_SUCCESS:

openstack workflow execution create tripleo.deployment.v1.set_deployment_status_success

Set the status to DEPLOYING:

openstack workflow execution create tripleo.deployment.v1.set_deployment_status_deploying

Set the status to DEPLOY_FAILED:

openstack workflow execution create tripleo.deployment.v1.set_deployment_status_failed

The default plan name of overcloud will be used in the above commands. It can be overridden with any of the above commands if needed:

openstack workflow execution create tripleo.deployment.v1.set_deployment_status_success '{"plan":"my-cloud-name"}'

Deployment Log

The ansible part of the deployment creates a log file that is saved on the undercloud. The log file is available at $HOME/ansible.log.

config-download Use Cases

config-download exposes the ability to manually run the ansible-playbook command against the playbooks that are generated for the deployment. This leads to many advantages over the older Heat deployment model.

  • Test deployments. Using the ansible-playbook --check --diff deploy_steps_playbook.yaml arguments will not modify an existing deployment. Instead, it will only show any changes that would be made.

  • Development environment testing. Ansible variables can be modified to do quick testing. Once verified, Heat environment templates need to be updated to reflect the change permanently. Then the config-download content should be re-generated by running openstack overcloud deploy --stack-only.

  • Run specific tasks. It is possible to run certain parts of a deployment by using --tags.

  • Prepare the deployment or update ahead of time and then run the playbooks later. The operations around a deployment can be done at different times to minimize risk.

  • Integration with CI/CD. Additional checks and verification can be added to a CI/CD pipeline relating to updating Heat templates and the Ansible config-download content.

  • AWX or Ansible Tower integration. Ansible content can be imported and ran through a scalable and distributed system.