TripleO CI Zuul Jobs Parenting¶
When a developer submits a patch to TripleO repositories, their code is tested against a series of different TripleO CI jobs. Each job creates a different scenario for testing purposes.
The TripleO CI jobs are Zuul jobs, defined within TripleO projects under one of several locations: zuul.d directory, .zuul.yaml or zuul.yaml.
A Zuul job can be inherited in various child jobs as parent.
Zuul Job Parenting¶
In order to re-use a particular Zuul job, we create a set of standard base jobs, which contain ansible variables, required projects, pre-run, run, post-run steps and Zuul related variables.
These base job definitions are used as parent in various tripleo-ci jobs. The child job inherits attributes from the parent unless these are overridden by the child.
A child job can override the variable which is also defined in parent job.
TripleO CI Base jobs¶
TripleO CI base jobs are defined in zuul.d/base-upstream.yaml file in tripleo-ci repo.
Below is the list of base jobs and each is explained in a little more detail in subsequent sections:
It contains a list of common required projects and ansible roles which are needed to start the deployment. It is used in upstream, RDO and Downstream. If a new project is needed in all types of deployment (upstream, RDO and Downstream) it can be added here.
It contains a set of ansible variables and playbooks used in most deployments.
It contains a set of ansible variables and playbooks used in most containers multinode and scenarios job.
It is used in those jobs where the user needs to deploy OpenStack using one undercloud and one controller.
It contains a set of ansible variables and playbooks used in most single node jobs.
It is used in those jobs where user needs to build containers and overcloud images which later can be used in another deployment.
It can also be used for undercloud deployment.
It contains a set of ansible variables and playbooks used in vanilla standalone and standalone based scenario jobs.
The standalone job consists of single node overcloud deployment.
It contains a set of ansible variables and playbooks used in the standalone upgrade job.
The singlenode job consists of single node overcloud deployment where we upgrade a deployment from an older release to a newer one.
It contains a set of ansible variables and playbooks used in the virtual baremetal deployment.
The ovb job consists of one undercloud and four overcloud nodes (one compute and multiple controllers) deployed as virtual baremetal nodes. It is a replica of real world customer deployments.
It is used in RDO and downstream jobs.
It contains a set of ansible variables and playbooks used during build containers and pushing it to specific registry.
It contains a set of ansible variables and playbooks used during build overcloud images and pushing it to image server.
It contains a set of ansible variables and playbooks used for building containers and pushing them to a local registry. Depends-on patches are built into respective rpm packages via DLRN and served by a local yum repos.
The job is paused to serve container registry and yum repos which can be used later in dependent jobs.
Currently these jobs are running in Upstream and Downstream.
Required Project Jobs¶
It contains the list of required projects needed for specific type of deployment.
Upstream job tripleo-ci-build-containers-required-projects-upstream requires projects like ansible-role-container-registry, kolla, python-tripleoclient, tripleo-ansible to build containers.
In case of RDO tripleo-ci-build-containers-required-projects-rdo serves the same purpose.
Many Upstream OpenStack projects are forked downstream and have different branches.
To accommodate the downstream namespace and branches we use the downstream specific required project job (required-projects-downstream) as a base job with proper branches and override-checkout.
tripleo-ci-base-required-projects-multinode-internal job defined in the examples are perfect example for the same.
Below is one of the examples of container multinode required projects job.
- job: name: tripleo-ci-base-required-projects-multinode-upstream description: | Base abstract job to add required-projects for Upstream Multinode Jobs abstract: true parent: tripleo-ci-base-multinode-standard required-projects: - opendev.org/openstack/tripleo-ansible - opendev.org/openstack/tripleo-common - opendev.org/openstack/tripleo-operator-ansible - name: opendev.org/openstack/ansible-config_template override-checkout: master
- job: name: tripleo-ci-base-required-projects-multinode-rdo abstract: true description: | Base abstract job for multinode in RDO CI zuulv3 jobs parent: tripleo-ci-base-multinode-standard pre-run: - playbooks/tripleo-rdo-base/pre.yaml - playbooks/tripleo-rdo-base/container-login.yaml roles: - zuul: opendev.org/openstack/ansible-role-container-registry - zuul: opendev.org/openstack/tripleo-ansible required-projects: - opendev.org/openstack/ansible-role-container-registry - opendev.org/openstack/tripleo-ansible secrets: - rdo_registry vars: registry_login_enabled: true
- job: name: tripleo-ci-base-required-projects-multinode-internal description: | Base abstract job to add required-projects for multinode downstream job abstract: true override-checkout: <downstream branch name> parent: tripleo-ci-base-multinode-standard required-projects: - name: tripleo-ansible branch: <downstream-branch> - ansible-config_template - tripleo-operator-ansible - rdo-jobs - tripleo-environments roles: - zuul: rdo-jobs pre-run: - playbooks/configure-mirrors.yaml - playbooks/tripleo-rdo-base/cert-install.yaml - playbooks/tripleo-rdo-base/pre-keys.yaml vars: mirror_locn: <downstream mirror address> featureset_override: artg_repos_dir: /home/zuul/src/<downstream-url>/openstack
The TripleO deployment is supported on multiple distro versions. Here is the current supported matrix in RDO, Downstream and Upstream.
CentOS/CentOS Stream Version
Each of these distros have different settings which are used in deployment. It’s easier to maintain separate variables based on distributions.
Below is an example of distro jobs for containers multinode at different levels.
- job: name: tripleo-ci-base-multinode abstract: true description: | Base abstract job for multinode TripleO CI C7 zuulv3 jobs parent: tripleo-ci-base-required-projects-multinode-upstream nodeset: two-centos-7-nodes - job: name: tripleo-ci-base-multinode-centos-8 abstract: true description: | Base abstract job for multinode TripleO CI centos-8 zuulv3 jobs parent: tripleo-ci-base-required-projects-multinode-upstream nodeset: two-centos-8-nodes - job: name: tripleo-ci-base-multinode-centos-9 abstract: true description: | Base abstract job for multinode TripleO CI centos-9 zuulv3 jobs parent: tripleo-ci-base-required-projects-multinode-upstream nodeset: two-centos-9-nodes
- job: name: tripleo-ci-base-multinode-periodic parent: tripleo-ci-base-multinode-rdo pre-run: playbooks/tripleo-ci-periodic-base/pre.yaml post-run: playbooks/tripleo-ci-periodic-base/post.yaml required-projects: - config - rdo-infra/ci-config roles: - zuul: rdo-infra/ci-config secrets: - dlrnapi - job: name: tripleo-ci-base-multinode-periodic-centos-8 parent: tripleo-ci-base-multinode-rdo-centos-8 pre-run: playbooks/tripleo-ci-periodic-base/pre.yaml post-run: playbooks/tripleo-ci-periodic-base/post.yaml required-projects: - config - rdo-infra/ci-config roles: - zuul: rdo-infra/ci-config vars: promote_source: tripleo-ci-testing secrets: - dlrnapi - job: name: tripleo-ci-base-multinode-periodic-centos-9 parent: tripleo-ci-base-multinode-rdo-centos-9 pre-run: playbooks/tripleo-ci-periodic-base/pre.yaml post-run: playbooks/tripleo-ci-periodic-base/post.yaml required-projects: - config - rdo-infra/ci-config roles: - zuul: rdo-infra/ci-config vars: promote_source: tripleo-ci-testing secrets: - dlrnapi
Zuul Job Inheritance Order¶
Here is an example of Upstream inheritance of tripleo-ci-centos-9-containers-multinode job.:
tripleo-ci-base-common-required-projects | v tripleo-ci-base-standard | v tripleo-ci-base-multinode-standard | v tripleo-ci-base-required-projects-multinode-upstream | v tripleo-ci-base-multinode-centos-9 | v tripleo-ci-centos-9-containers-multinode
Here is the another example of RDO job periodic-tripleo-ci-centos-8-containers-multinode-master
tripleo-ci-base-multinode-standard | v tripleo-ci-base-required-projects-multinode-rdo | v tripleo-ci-base-multinode-rdo-centos-8 | v tripleo-ci-base-multinode-periodic-centos-8 | v periodic-tripleo-ci-centos-8-containers-multinode-master
TripleO CI Zuul Job Repos¶
Below is the list of repos where tripleo-ci related Zuul jobs are defined.
FAQs regarding TripleO CI jobs¶
If we have a new project, which needs to be tested at all places and installed from source but
cloned from upstream source, then it must be added under required-projects at tripleo-ci-base-common-required-projects job.
the project namespace is different in Upstream and downstream, then it must be added under required-projects at Downstream (tripleo-ci-base-required-projects-multinode-internal) or Upstream (tripleo-ci-base-required-projects-multinode-upstream) specific required-projects parent job.
if the project is only developed at downstream or RDO or Upstream, then it must be added under required project at downstream or RDO or Upstream required-projects parent job.
In order to add support for new distros, please use required-projects job as a parent and then create distro version specific child job with required nodeset.
If a project with different branch is re-added in child job required-projects, then the child job project will be used in the deployment.
If a playbook (which calls another role, exists in different repo) is called at pre-run step in Zuul job, then role specific required projects and roles needs to be added at that job level. For example: In tripleo-ci-containers-rdo-upstream-pre job, ansible-role-container-registry and triple-ansible is needed for pre.yaml playbook. So both projects are added in roles and required-projects.
If a job having pre/post run playbook needs zuul secrets and playbook depends on distros, then the job needs to be defined in config repo.
We should not use branches attributes in Zuul Distro jobs or options jobs.