Testing

Zuul provides an extensive framework for performing functional testing on the system from end-to-end with major external components replaced by fakes for ease of use and speed.

Test classes that subclass ZuulTestCase have access to a number of attributes useful for manipulating or inspecting the environment being simulated in the test:

tests.base.simple_layout(path, driver='gerrit')

Specify a layout file for use by a test method.

Parameters:
  • path (str) – The path to the layout file.
  • driver (str) – The source driver to use, defaults to gerrit.

Some tests require only a very simple configuration. For those, establishing a complete config directory hierachy is too much work. In those cases, you can add a simple zuul.yaml file to the test fixtures directory (in fixtures/layouts/foo.yaml) and use this decorator to indicate the test method should use that rather than the tenant config file specified by the test class.

The decorator will cause that layout file to be added to a config-project called “common-config” and each “project” instance referenced in the layout file will have a git repo automatically initialized.

class tests.base.ZuulTestCase(*args, **kwargs)

A test case with a functioning Zuul.

The following class variables are used during test setup and can be overidden by subclasses but are effectively read-only once a test method starts running:

Variables:
  • config_file (str) – This points to the main zuul config file within the fixtures directory. Subclasses may override this to obtain a different behavior.
  • tenant_config_file (str) – This is the tenant config file (which specifies from what git repos the configuration should be loaded). It defaults to the value specified in config_file but can be overidden by subclasses to obtain a different tenant/project layout while using the standard main configuration. See also the simple_layout() decorator.
  • create_project_keys (bool) – Indicates whether Zuul should auto-generate keys for each project, or whether the test infrastructure should insert dummy keys to save time during startup. Defaults to False.

The following are instance variables that are useful within test methods:

Variables:
  • fake_<connection> (FakeGerritConnection) – A FakeGerritConnection will be instantiated for each connection present in the config file and stored here. For instance, fake_gerrit will hold the FakeGerritConnection object for a connection named gerrit.
  • gearman_server (FakeGearmanServer) – An instance of FakeGearmanServer which is the Gearman server that all of the Zuul components in this test use to communicate with each other.
  • executor_server (RecordingExecutorServer) – An instance of RecordingExecutorServer which is the Ansible execute server used to run jobs for this test.
  • builds (list) – A list of FakeBuild objects representing currently running builds. They are appended to the list in the order they are executed, and removed from this list upon completion.
  • history (list) – A list of BuildHistory objects representing completed builds. They are appended to the list in the order they complete.
addEvent(connection, event)

Inject a Fake (Gerrit) event.

This method accepts a JSON-encoded event and simulates Zuul having received it from Gerrit. It could (and should) eventually apply to any connection type, but is currently only used with Gerrit connections. The name of the connection is used to look up the corresponding server, and the event is simulated as having been received by all Zuul connections attached to that server. So if two Gerrit connections in Zuul are connected to the same Gerrit server, and you invoke this method specifying the name of one of them, the event will be received by both.

Note

“self.fake_gerrit.addEvent” calls should be migrated to this method.

Parameters:
  • connection (str) – The name of the connection corresponding to the gerrit server.
  • event (str) – The JSON-encoded event.
assertBuilds(builds)

Assert that the running builds are as described.

The list of running builds is examined and must match exactly the list of builds described by the input.

Parameters:builds (list) – A list of dictionaries. Each item in the list must match the corresponding build in the build history, and each element of the dictionary must match the corresponding attribute of the build.
assertHistory(history, ordered=True)

Assert that the completed builds are as described.

The list of completed builds is examined and must match exactly the list of builds described by the input.

Parameters:
  • history (list) – A list of dictionaries. Each item in the list must match the corresponding build in the build history, and each element of the dictionary must match the corresponding attribute of the build.
  • ordered (bool) – If true, the history must match the order supplied, if false, the builds are permitted to have arrived in any order.
commitConfigUpdate(project_name, source_name)

Commit an update to zuul.yaml

This overwrites the zuul.yaml in the specificed project with the contents specified.

Parameters:
  • project_name (str) – The name of the project containing zuul.yaml (e.g., common-config)
  • source_name (str) – The path to the file (underneath the test fixture directory) whose contents should be used to replace zuul.yaml.
getSortedBuilds()

Return the list of currently running builds sorted by name

getUpstreamRepos(projects)

Return upstream git repo objects for the listed projects

Parameters:projects (list) – A list of strings, each the canonical name of a project.
Returns:A dictionary of {name: repo} for every listed project.
Return type:dict
newTenantConfig(source_name)

Use this to update the tenant config file in tests

This will update self.tenant_config_file to point to a temporary file for the duration of this particular test. The content of that file will be taken from FIXTURE_DIR/source_name

After the test the original value of self.tenant_config_file will be restored.

Parameters:source_name (str) – The path of the file under FIXTURE_DIR that will be used to populate the new tenant config file.
printHistory()

Log the build history.

This can be useful during tests to summarize what jobs have completed.

class tests.base.FakeGerritConnection(driver, connection_name, connection_config, changes_db=None, upstream_root=None)

A Fake Gerrit connection for use in tests.

This subclasses GerritConnection to add the ability for tests to add changes to the fake Gerrit it represents.

addFakeChange(project, branch, subject, status='NEW', files=None, parent=None)

Add a change to the fake Gerrit.

registerHttpHandler(path, handler)

Add connection handler for HTTP URI.

Connection can use builtin HTTP server for listening on incoming event requests. The resulting path will be /connection/connection_name/path.

unregisterHttpHandler(path)

Remove the connection handler for HTTP URI.

class tests.base.FakeGearmanServer(use_ssl=False)

A Gearman server for use in tests.

Variables:hold_jobs_in_queue (bool) – If true, submitted jobs will be added to the queue but will not be distributed to workers until released. This attribute may be changed at any time and will take effect for subsequently enqueued jobs, but previously held jobs will still need to be explicitly released.
release(regex=None)

Release a held job.

Parameters:regex (str) – A regular expression which, if supplied, will cause only jobs with matching names to be released. If not supplied, all jobs will be released.
class tests.base.RecordingExecutorServer(*args, **kw)

An Ansible executor to be used in tests.

Variables:hold_jobs_in_build (bool) – If true, when jobs are executed they will report that they have started but then pause until released before reporting completion. This attribute may be changed at any time and will take effect for subsequently executed builds, but previously held builds will still need to be explicitly released.
failJob(name, change)

Instruct the executor to report matching builds as failures.

Parameters:
  • name (str) – The name of the job to fail.
  • change (Change) – The FakeChange instance which should cause the job to fail. This job will also fail for changes depending on this change.
release(regex=None)

Release a held build.

Parameters:regex (str) – A regular expression which, if supplied, will cause only builds with matching names to be released. If not supplied, all builds will be released.
class tests.base.FakeBuild(executor_server, job)
getWorkspaceRepos(projects)

Return workspace git repo objects for the listed projects

Parameters:projects (list) – A list of strings, each the canonical name of a project.
Returns:A dictionary of {name: repo} for every listed project.
Return type:dict
hasChanges(*changes)

Return whether this build has certain changes in its git repos.

Parameters:changes (FakeChange) – One or more changes (varargs) that are expected to be present (in order) in the git repository of the active project.
Returns:Whether the build has the indicated changes.
Return type:bool
isWaiting()

Return whether this build is being held.

Returns:Whether the build is being held.
Return type:bool
release()

Release this build.

class tests.base.BuildHistory(**kw)