API reference for storage drivers

API reference for storage drivers

class DataDriverBase(conf, cache, control_driver)[source]

Interface definition for storage drivers.

Data plane storage drivers are responsible for implementing the core functionality of the system.

Connection information and driver-specific options are loaded from the config file or the pool catalog.

Parameters:
  • conf (oslo_config.ConfigOpts) – Configuration containing options for this driver.
  • cache (dogpile.cache.region.CacheRegion) – Cache instance to use for reducing latency for certain lookups.
capabilities

Returns storage’s capabilities.

claim_controller

Returns the driver’s claim controller.

close()[source]

Close connections to the backend.

gc()[source]

Perform manual garbage collection of claims and messages.

This method can be overridden in order to provide a trigger that can be called by so-called “garbage collection” scripts that are required by some drivers.

By default, this method does nothing.

health()[source]

Return the health status of service.

is_alive()[source]

Check whether the storage is ready.

message_controller

Returns the driver’s message controller.

subscription_controller

Returns the driver’s subscription controller.

class ControlDriverBase(conf, cache)[source]

Interface definition for control plane storage drivers.

Storage drivers that work at the control plane layer allow one to modify aspects of the functionality of the system. This is ideal for administrative purposes.

Allows access to the pool registry through a catalogue and a pool controller.

Parameters:
  • conf (oslo_config.ConfigOpts) – Configuration containing options for this driver.
  • cache (dogpile.cache.region.CacheRegion) – Cache instance to use for reducing latency for certain lookups.
catalogue_controller

Returns the driver’s catalogue controller.

close()[source]

Close connections to the backend.

flavors_controller

Returns storage’s flavor management controller.

pools_controller

Returns storage’s pool management controller.

queue_controller

Returns the driver’s queue controller.

class Queue(driver)[source]

This class is responsible for managing queues.

Queue operations include CRUD, monitoring, etc.

Storage driver implementations of this class should be capable of handling high workloads and huge numbers of queues.

create(name, metadata=None, project=None)[source]

Base method for queue creation.

Parameters:
  • name – The queue name
  • project – Project id
Returns:

True if a queue was created and False if it was updated.

delete(name, project=None)[source]

Base method for deleting a queue.

Parameters:
  • name – The queue name
  • project – Project id
exists(name, project=None)[source]

Base method for testing queue existence.

Parameters:
  • name – The queue name
  • project – Project id
Returns:

True if a queue exists and False if it does not.

get(name, project=None)[source]

Base method for queue metadata retrieval.

Parameters:
  • name – The queue name
  • project – Project id
Returns:

Dictionary containing queue metadata

Raises:

DoesNotExist – if queue metadata does not exist

get_metadata(name, project=None)[source]

Base method for queue metadata retrieval.

Parameters:
  • name – The queue name
  • project – Project id
Returns:

Dictionary containing queue metadata

Raises:

DoesNotExist – if queue metadata does not exist

list(project=None, marker=None, limit=10, detailed=False)[source]

Base method for listing queues.

Parameters:
  • project – Project id
  • marker – The last queue name
  • limit – (Default 10) Max number of queues to return
  • detailed – Whether metadata is included
Returns:

An iterator giving a sequence of queues and the marker of the next page.

set_metadata(name, metadata, project=None)[source]

Base method for updating a queue metadata.

Parameters:
  • name – The queue name
  • metadata – Queue metadata as a dict
  • project – Project id
Raises:

DoesNotExist – if queue metadata can not be updated

stats(name, project=None)[source]

Base method for queue stats.

Parameters:
  • name – The queue name
  • project – Project id
Returns:

Dictionary with the queue stats

class Message(driver)[source]

This class is responsible for managing message CRUD.

bulk_delete(queue, message_ids, project=None)[source]

Base method for deleting multiple messages.

Parameters:
  • queue – Name of the queue to post message to.
  • message_ids – A sequence of message IDs to be deleted.
  • project – Project id
bulk_get(queue, message_ids, project=None)[source]

Base method for getting multiple messages.

Parameters:
  • queue – Name of the queue to get the message from.
  • project – Project id
  • message_ids – A sequence of message IDs.
Returns:

An iterable, yielding dicts containing message details

delete(queue, message_id, project=None, claim=None)[source]

Base method for deleting a single message.

Parameters:
  • queue – Name of the queue to post message to.
  • message_id – Message to be deleted
  • project – Project id
  • claim – Claim this message belongs to. When specified, claim must be valid and message_id must belong to it.
first(queue, project=None, sort=1)[source]

Get first message in the queue (including claimed).

Parameters:
  • queue – Name of the queue to list
  • sort – (Default 1) Sort order for the listing. Pass 1 for ascending (oldest message first), or -1 for descending (newest message first).
Returns:

First message in the queue, or None if the queue is empty

get(queue, message_id, project=None)[source]

Base method for getting a message.

Parameters:
  • queue – Name of the queue to get the message from.
  • project – Project id
  • message_id – Message ID
Returns:

Dictionary containing message data

Raises:

DoesNotExist – if message data can not be got

list(queue, project=None, marker=None, limit=10, echo=False, client_uuid=None, include_claimed=False, include_delayed=False)[source]

Base method for listing messages.

Parameters:
  • queue – Name of the queue to get the message from.
  • project – Project id
  • marker – Tail identifier
  • limit (Maybe int) – (Default 10) Max number of messages to return.
  • echo – (Default False) Boolean expressing whether or not this client should receive its own messages.
  • client_uuid – A UUID object. Required when echo=False.
  • include_claimed (bool) – omit claimed messages from listing?
  • include_delayed (bool) – omit delayed messages from listing
Returns:

An iterator giving a sequence of messages and the marker of the next page.

pop(queue, limit, project=None)[source]

Base method for popping messages.

Parameters:
  • queue – Name of the queue to pop message from.
  • limit – Number of messages to pop.
  • project – Project id
post(queue, messages, client_uuid, project=None)[source]

Base method for posting one or more messages.

Implementations of this method should guarantee and preserve the order, in the returned list, of incoming messages.

Parameters:
  • queue – Name of the queue to post message to.
  • messages – Messages to post to queue, an iterable yielding 1 or more elements. An empty iterable results in undefined behavior.
  • client_uuid – A UUID object.
  • project – Project id
Returns:

List of message ids

class Claim(driver)[source]
create(queue, metadata, project=None, limit=10)[source]

Base method for creating a claim.

Parameters:
  • queue – Name of the queue this claim belongs to.
  • metadata – Claim’s parameters to be stored.
  • project – Project id
  • limit – (Default 10) Max number of messages to claim.
Returns:

(Claim ID, claimed messages)

delete(queue, claim_id, project=None)[source]

Base method for deleting a claim.

Parameters:
  • queue – Name of the queue this claim belongs to.
  • claim_id – Claim to be deleted
  • project – Project id
get(queue, claim_id, project=None)[source]

Base method for getting a claim.

Parameters:
  • queue – Name of the queue this claim belongs to.
  • claim_id – The claim id
  • project – Project id
Returns:

(Claim’s metadata, claimed messages)

Raises:

DoesNotExist – if claimed messages can not be got

update(queue, claim_id, metadata, project=None)[source]

Base method for updating a claim.

Parameters:
  • queue – Name of the queue this claim belongs to.
  • claim_id – Claim to be updated
  • metadata – Claim’s parameters to be updated.
  • project – Project id

MongoDB Driver

MongoDB Storage Driver for Zaqar.

About the store

MongoDB is a nosql, eventually consistent, reliable database with support for horizontal-scaling and capable of handling different levels of throughputs.

Supported Features

  • FIFO
  • Unlimited horizontal-scaling [1]
  • Reliability [2]
[1]This is only possible with a sharding environment
[2]Write concern must be equal or higher than 2

Supported Deployments

MongoDB can be deployed in 3 different ways. The first and most simple one is to deploy a standalone mongod node. The second one is to use a Replica Sets which gives a master-slave deployment but cannot be scaled unlimitedly. The third and last one is a sharded cluster.

The second and third methods are the ones recommended for production environments where durability and scalability are a must-have. The driver itself forces operators to use such environments by checking whether it is talking to a replica-set or sharded cluster. Such enforcement can be disabled by running Zaqar in an unreliable mode.

Replica Sets

When running on a replica-set, Zaqar won’t try to be smart and it’ll rely as much as possible on the database and pymongo.

Sharded Cluster

TBD

Creative Commons Attribution 3.0 License

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.