Most of the handler code in the placement API is tested using gabbi. Some
utility code is tested with unit tests found in placement/tests/unit. The
back-end objects are tested with a combination of unit and functional tests
When writing tests for handler code (that is, the code found in
placement/handlers) a good rule of thumb is that if you feel like there
needs to be a unit test for some of the code in the handler, that is a good
sign that the piece of code should be extracted to a separate method. That
method should be independent of the handler method itself (the one decorated by
wsgify method) and testable as a unit, without mocks if possible. If
the extracted method is useful for multiple resources consider putting it in
As a general guide, handler code should be relatively short and where there are conditionals and branching, they should be reachable via the gabbi functional tests. This is merely a design goal, not a strict constraint.
Gabbi was developed in the telemetry project to provide a declarative way to test HTTP APIs that preserves visibility of both the request and response of the HTTP interaction. Tests are written in YAML files where each file is an ordered suite of tests. Fixtures (such as a database) are set up and torn down at the beginning and end of each file, not each test. JSON response bodies can be evaluated with JSONPath. The placement WSGI application is run via wsgi-intercept, meaning that real HTTP requests are being made over a file handle that appears to Python to be a socket.
In the placement API the YAML files (aka “gabbits”) can be found in
placement/tests/functional/gabbits. Fixture definitions are in
placement/tests/functional/fixtures/gabbits.py. Tests are frequently
grouped by handler name (e.g.,
inventory.yaml). This is not a requirement and as we increase the number of
tests it makes sense to have more YAML files with fewer tests, divided up by
the arc of API interaction that they test.
The gabbi tests are integrated into the functional tox target, loaded via
placement/tests/functional/test_api.py. If you
want to run just the gabbi tests one way to do so is:
tox -efunctional test_api
If you want to run just one yaml file (in this example
tox -efunctional api.inventory
It is also possible to run just one test from within one file. When you do this
every test prior to the one you asked for will also be run. This is because
the YAML represents a sequence of dependent requests. Select the test by using
the name in the yaml file, replacing space with
tox -efunctional api.inventory_post_new_ipv4_address_inventory
tox.ini in the placement repository is configured by a
group_regex so that each gabbi YAML is considered a group. Thus,
all tests in the file will be run in the same process when running
stestr concurrently (the default).
Writing More Gabbi Tests¶
The docs for gabbi try to be complete and explain the syntax in some depth. Where something is missing or confusing, please log a bug.
While it is possible to test all aspects of a response (all the response headers, the status code, every attribute in a JSON structure) in one single test, doing so will likely make the test harder to read and will certainly make debugging more challenging. If there are multiple things that need to be asserted, making multiple requests is reasonable. Since database set up is only happening once per file (instead of once per test) and since there is no TCP overhead, the tests run quickly.
While fixtures can be used to establish entities that are required for
tests, creating those entities via the HTTP API results in tests which are more
descriptive. For example the
inventory.yaml file creates the resource
provider to which it will then add inventory. This makes it easy to explore a
sequence of interactions and a variety of responses with the tests:
create a resource provider
confirm it has empty inventory
add inventory to the resource provider (in a few different ways)
confirm the resource provider now has inventory
modify the inventory
delete the inventory
confirm the resource provider now has empty inventory
Nothing special is required to add a new set of tests: create a YAML file with a unique name in the same directory as the others. The other files can provide examples. Gabbi can provide a useful way of doing test driven development of a new handler: create a YAML file that describes the desired URLs and behavior and write the code to make it pass.
It’s also possible to use gabbi against a running placement service, for example in devstack. See gabbi-run to get started. If you don’t want to go to the trouble of using devstack, but do want a live server see Quick Placement Development.
If you wish to profile requests to the placement service, to get an idea of which methods are consuming the most CPU or are being used repeatedly, it is possible to enable a ProfilerMiddleware to output per-request python profiling dumps. The environment (Quick Placement Development is a good place to start) in which the service is running will need to have Werkzeug added.
If the service is already running, stop it.
Set an environment variable,
OS_WSGI_PROFILER, to a directory where profile results will be written.
Make sure the directory exists.
Start the service, ensuring the environment variable is passed to it.
Make an HTTP request that exercises the code you wish to profile.
The profiling results will be in the directory named by
There are many ways to analyze the files. See Profiling WSGI Apps for an
Profiling with OSProfiler¶
To use OSProfiler with placement:
Add a [profiler] section to the placement.conf:
[profiler] connection_string = mysql+pymysql://root:firstname.lastname@example.org/osprofiler?charset=utf8 hmac_keys = my-secret-key enabled = True
Include the hmac_keys in your API request:
$ openstack resource provider list --os-profile my-secret-key
The openstack client will return the trace id:
Trace ID: 67428cdd-bfaa-496f-b430-507165729246
Extract the trace in html format:
$ osprofiler trace show --html 67428cdd-bfaa-496f-b430-507165729246 \ --connection-string mysql+pymysql://root:email@example.com/osprofiler?charset=utf8