Scheduler

Scheduler

The manila.scheduler.manager Module

Scheduler Service

class SchedulerManager(scheduler_driver=None, service_name=None, *args, **kwargs)

Bases: manila.manager.Manager

Chooses a host to create shares.

RPC_API_VERSION = '1.9'
create_share_group(context, share_group_id, request_spec=None, filter_properties=None)
create_share_instance(context, request_spec=None, filter_properties=None)
create_share_replica(context, request_spec=None, filter_properties=None)
get_host_list(context)

Get a list of hosts from the HostManager.

get_pools(context, filters=None, cached=False)

Get active pools from the scheduler’s cache.

get_service_capabilities(context)

Get the normalized set of capabilities for this zone.

init_host()

Handle initialization if this is a standalone service.

Child classes should override this method.

manage_share(context, share_id, driver_options, request_spec, filter_properties=None)

Ensure that the host exists and can accept the share.

migrate_share_to_host(context, share_id, host, force_host_assisted_migration, preserve_metadata, writable, nondisruptive, preserve_snapshots, new_share_network_id, new_share_type_id, request_spec, filter_properties=None)

Ensure that the host exists and can accept the share.

request_service_capabilities(context)
update_service_capabilities(context, service_name=None, host=None, capabilities=None, **kwargs)

Process a capability update from a service node.

The manila.scheduler.base_handler Module

A common base for handling extension classes.

Used by BaseFilterHandler and BaseWeightHandler

class BaseHandler(modifier_class_type, modifier_namespace)

Bases: object

Base class to handle loading filter and weight classes.

get_all_classes()

The manila.scheduler.host_manager Module

Manage hosts in the current zone.

class HostManager

Bases: object

Base HostManager class.

get_all_host_states_share(context)

Returns a dict of all the hosts the HostManager knows about.

Each of the consumable resources in HostState are populated with capabilities scheduler received from RPC.

For example:
{‘192.168.1.100’: HostState(), …}
get_filtered_hosts(hosts, filter_properties, filter_class_names=None)

Filter hosts and return only ones passing all filters.

get_pools(context, filters=None, cached=False)

Returns a dict of all pools on all hosts HostManager knows about.

get_weighed_hosts(hosts, weight_properties, weigher_class_names=None)

Weigh the hosts.

host_state_cls

alias of HostState

update_service_capabilities(service_name, host, capabilities)

Update the per-service capabilities based on this notification.

class HostState(host, capabilities=None, service=None)

Bases: object

Mutable and immutable information tracked for a host.

consume_from_share(share)

Incrementally update host state from an share.

update_backend(capability)
update_capabilities(capabilities=None, service=None)
update_from_share_capability(capability, service=None, context=None)

Update information about a host from its share_node info.

‘capability’ is the status info reported by share backend, a typical capability looks like this:

capability = {
    'share_backend_name': 'Local NFS',    #                'vendor_name': 'OpenStack',           #  backend level
    'driver_version': '1.0',              #  mandatory/fixed
    'storage_protocol': 'NFS',            #/ stats&capabilities

    'active_shares': 10,                  #                'IOPS_provisioned': 30000,            #  optional custom
    'fancy_capability_1': 'eat',          #  stats & capabilities
    'fancy_capability_2': 'drink',        #/

    'pools':[
      {
         'pool_name': '1st pool',         #                     'total_capacity_gb': 500,        #  mandatory stats
         'free_capacity_gb': 230,         #   for pools
         'allocated_capacity_gb': 270,    # |
         'qos': 'False',                  # |
         'reserved_percentage': 0,        #/

         'dying_disks': 100,              #                     'super_hero_1': 'spider-man',    #  optional custom
         'super_hero_2': 'flash',         #  stats &
         'super_hero_3': 'neoncat',       #  capabilities
         'super_hero_4': 'green lantern', #/
       },
      {
         'pool_name': '2nd pool',
         'total_capacity_gb': 1024,
         'free_capacity_gb': 1024,
         'allocated_capacity_gb': 0,
         'qos': 'False',
         'reserved_percentage': 0,

         'dying_disks': 200,
         'super_hero_1': 'superman',
         'super_hero_2': 'Hulk',
      }]
}
update_pools(capability, service, context=None)

Update storage pools information from backend reported info.

class PoolState(host, capabilities, pool_name)

Bases: manila.scheduler.host_manager.HostState

update_from_share_capability(capability, service=None, context=None)

Update information about a pool from its share_node info.

update_pools(capability)

Update storage pools information from backend reported info.

class ReadOnlyDict(source=None)

Bases: collections.UserDict

A read-only dict.

clear() → None. Remove all items from D.
pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

update([E, ]**F) → None. Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

The manila.scheduler.rpcapi Module

Client side of the scheduler manager RPC API.

class SchedulerAPI

Bases: object

Client side of the scheduler rpc API.

API version history:

1.0 - Initial version. 1.1 - Add get_pools method 1.2 - Introduce Share Instances. Replace create_share() with create_share_instance() 1.3 - Add create_consistency_group method (renamed in 1.7) 1.4 - Add migrate_share_to_host method 1.5 - Add create_share_replica 1.6 - Add manage_share 1.7 - Updated migrate_share_to_host method with new parameters 1.8 - Rename create_consistency_group -> create_share_group method 1.9 - Add cached parameter to get_pools method
RPC_API_VERSION = '1.9'
create_share_group(context, share_group_id, request_spec=None, filter_properties=None)

Casts an rpc to the scheduler to create a share group.

Example of ‘request_spec’ argument value:

{

    'share_group_type_id': 'fake_share_group_type_id',
    'share_group_id': 'some_fake_uuid',
    'availability_zone_id': 'some_fake_az_uuid',
    'share_types': [models.ShareType],
    'resource_type': models.ShareGroup,

}
create_share_instance(context, request_spec=None, filter_properties=None)
create_share_replica(context, request_spec=None, filter_properties=None)
get_pools(context, filters=None, cached=False)
manage_share(context, share_id, driver_options, request_spec=None, filter_properties=None)
migrate_share_to_host(context, share_id, host, force_host_assisted_migration, preserve_metadata, writable, nondisruptive, preserve_snapshots, new_share_network_id, new_share_type_id, request_spec=None, filter_properties=None)
update_service_capabilities(context, service_name, host, capabilities)

The manila.scheduler.scheduler_options Module

SchedulerOptions monitors a local .json file for changes and loads it if needed. This file is converted to a data structure and passed into the filtering and weighing functions which can use it for dynamic configuration.

class SchedulerOptions

Bases: object

Monitor and load local .json file for filtering and weighing.

SchedulerOptions monitors a local .json file for changes and loads it if needed. This file is converted to a data structure and passed into the filtering and weighing functions which can use it for dynamic configuration.

get_configuration(filename=None)

Check the json file for changes and load it if needed.

The manila.scheduler.drivers.filter Module

The FilterScheduler is for scheduling of share and share group creation. You can customize this scheduler by specifying your own share/share group filters and weighing functions.

class FilterScheduler(*args, **kwargs)

Bases: manila.scheduler.drivers.base.Scheduler

Scheduler that can be used for filtering and weighing.

get_pools(context, filters, cached)

Must override schedule method for scheduler to work.

host_passes_filters(context, host, request_spec, filter_properties)

Must override schedule method for migration to work.

populate_filter_properties_share(request_spec, filter_properties)

Stuff things into filter_properties.

Can be overridden in a subclass to add more data.

schedule_create_replica(context, request_spec, filter_properties)

Must override schedule method for create replica to work.

schedule_create_share(context, request_spec, filter_properties)

Must override schedule method for scheduler to work.

schedule_create_share_group(context, share_group_id, request_spec, filter_properties)

Must override schedule method for scheduler to work.

The manila.scheduler.drivers.base Module

Scheduler base class that all Schedulers should inherit from

class Scheduler

Bases: object

The base class that all Scheduler classes should inherit from.

get_host_list()

Get a list of hosts from the HostManager.

get_pools(context, filters)

Must override schedule method for scheduler to work.

get_service_capabilities()

Get the normalized set of capabilities for the services.

host_passes_filters(context, host, request_spec, filter_properties)

Must override schedule method for migration to work.

hosts_up(context, topic)

Return the list of hosts that have a running service for topic.

schedule(context, topic, method, *_args, **_kwargs)

Must override schedule method for scheduler to work.

schedule_create_replica(context, request_spec, filter_properties)

Must override schedule method for create replica to work.

schedule_create_share(context, request_spec, filter_properties)

Must override schedule method for scheduler to work.

schedule_create_share_group(context, share_group_id, request_spec, filter_properties)

Must override schedule method for scheduler to work.

update_service_capabilities(service_name, host, capabilities)

Process a capability update from a service node.

share_group_update_db(context, share_group_id, host)

Set the host and set the updated_at field of a share group.

Returns:A share group with the updated fields set properly.
share_replica_update_db(context, share_replica_id, host)

Set the host and the scheduled_at field of a share replica.

Returns:A Share Replica with the updated fields set.
share_update_db(context, share_id, host)

Set the host and set the scheduled_at field of a share.

Returns:A Share with the updated fields set properly.

The manila.scheduler.drivers.chance Module

Chance (Random) Scheduler implementation

class ChanceScheduler

Bases: manila.scheduler.drivers.base.Scheduler

Implements Scheduler as a random node selector.

schedule_create_share(context, request_spec, filter_properties)

Picks a host that is up at random.

The manila.scheduler.drivers.simple Module

Simple Scheduler

class SimpleScheduler

Bases: manila.scheduler.drivers.chance.ChanceScheduler

Implements Naive Scheduler that tries to find least loaded host.

schedule_create_share(context, request_spec, filter_properties)

Picks a host that is up and has the fewest shares.

Scheduler Filters

The manila.scheduler.filters.availability_zone Filter

class AvailabilityZoneFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

Filters Hosts by availability zone.

host_passes(host_state, filter_properties)

Return True if the HostState passes the filter, otherwise False.

Override this in a subclass.

run_filter_once_per_request = True

The manila.scheduler.filters.base Filter

Filter support

class BaseFilter

Bases: object

Base class for all filter classes.

filter_all(filter_obj_list, filter_properties)

Yield objects that pass the filter.

Can be overridden in a subclass, if you need to base filtering decisions on all objects. Otherwise, one can just override _filter_one() to filter a single object.

run_filter_for_index(index)

Check if filter needs to be run for the “index-th” instance.

Return True if the filter needs to be run for the “index-th” instance in a request. Only need to override this if a filter needs anything other than “first only” or “all” behaviour.

run_filter_once_per_request = False
class BaseFilterHandler(modifier_class_type, modifier_namespace)

Bases: manila.scheduler.base_handler.BaseHandler

Base class to handle loading filter classes.

This class should be subclassed where one needs to use filters.

get_filtered_objects(filter_classes, objs, filter_properties, index=0)

Get objects after filter

Parameters:
  • filter_classes – filters that will be used to filter the objects
  • objs – objects that will be filtered
  • filter_properties – client filter properties
  • index – This value needs to be increased in the caller function of get_filtered_objects when handling each resource.

The manila.scheduler.filters.base_host Filter

Scheduler host filters

class BaseHostFilter

Bases: manila.scheduler.filters.base.BaseFilter

Base class for host filters.

host_passes(host_state, filter_properties)

Return True if the HostState passes the filter, otherwise False.

Override this in a subclass.

class HostFilterHandler(namespace)

Bases: manila.scheduler.filters.base.BaseFilterHandler

The manila.scheduler.filters.capabilities Filter

class CapabilitiesFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

HostFilter to work with resource (instance & volume) type records.

host_passes(host_state, filter_properties)

Return a list of hosts that can create resource_type.

The manila.scheduler.filters.capacity Filter

class CapacityFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

CapacityFilter filters based on share host’s capacity utilization.

host_passes(host_state, filter_properties)

Return True if host has sufficient capacity.

The manila.scheduler.filters.extra_specs_ops Filter

match(value, req)

The manila.scheduler.filters.ignore_attempted_hosts Filter

class IgnoreAttemptedHostsFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

Filter out previously attempted hosts

A host passes this filter if it has not already been attempted for scheduling. The scheduler needs to add previously attempted hosts to the ‘retry’ key of filter_properties in order for this to work correctly. For example:

{
 'retry': {
           'hosts': ['host1', 'host2'],
           'num_attempts': 3,
          }
}
host_passes(host_state, filter_properties)

Skip nodes that have already been attempted.

The manila.scheduler.filters.json Filter

class JsonFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

Host Filter to allow simple JSON-based grammar for selecting hosts.

commands = {'<': <function JsonFilter._less_than>, '<=': <function JsonFilter._less_than_equal>, '=': <function JsonFilter._equals>, '>': <function JsonFilter._greater_than>, '>=': <function JsonFilter._greater_than_equal>, 'and': <function JsonFilter._and>, 'in': <function JsonFilter._in>, 'not': <function JsonFilter._not>, 'or': <function JsonFilter._or>}
host_passes(host_state, filter_properties)

Filters hosts.

Return a list of hosts that can fulfill the requirements specified in the query.

The manila.scheduler.filters.retry Filter

class RetryFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

Filter out already tried nodes for scheduling purposes.

host_passes(host_state, filter_properties)

Skip nodes that have already been attempted.

The manila.scheduler.filters.share_replication Filter

class ShareReplicationFilter

Bases: manila.scheduler.filters.base_host.BaseHostFilter

ShareReplicationFilter filters hosts based on replication support.

host_passes(host_state, filter_properties)

Return True if ‘active’ replica’s host can replicate with host.

Design of this filter:

  • Share replication is symmetric. All backends that can replicate between each other must share the same ‘replication_domain’.
  • For scheduling a share that can be replicated in the future, this filter checks for ‘replication_domain’ capability.
  • For scheduling a replica, it checks for the ‘replication_domain’ compatibility.

Scheduler Weighers

The manila.scheduler.weighers.base Weigher

Pluggable Weighing support

class BaseWeigher

Bases: object

Base class for pluggable weighers.

The attributes maxval and minval can be specified to set up the maximum and minimum values for the weighed objects. These values will then be taken into account in the normalization step, instead of taking the values from the calculated weighers.

maxval = None
minval = None
weigh_objects(weighed_obj_list, weight_properties)

Weigh multiple objects.

Override in a subclass if you need access to all objects in order to calculate weighers. Do not modify the weight of an object here, just return a list of weighers.

weight_multiplier()

How weighted this weigher should be.

Override this method in a subclass, so that the returned value is read from a configuration option to permit operators specify a multiplier for the weigher.

class BaseWeightHandler(modifier_class_type, modifier_namespace)

Bases: manila.scheduler.base_handler.BaseHandler

get_weighed_objects(weigher_classes, obj_list, weighing_properties)

Return a sorted (descending), normalized list of WeighedObjects.

object_class

alias of WeighedObject

class WeighedObject(obj, weight)

Bases: object

Object with weight information.

normalize(weight_list, minval=None, maxval=None)

Normalize the values in a list between 0 and 1.0.

The normalization is made regarding the lower and upper values present in weight_list. If the minval and/or maxval parameters are set, these values will be used instead of the minimum and maximum from the list.

If all the values are equal, they are normalized to 0.

The manila.scheduler.weighers.base_host Weigher

Scheduler host weighers

class BaseHostWeigher

Bases: manila.scheduler.weighers.base.BaseWeigher

Base class for host weighers.

class HostWeightHandler(namespace)

Bases: manila.scheduler.weighers.base.BaseWeightHandler

object_class

alias of WeighedHost

class WeighedHost(obj, weight)

Bases: manila.scheduler.weighers.base.WeighedObject

to_dict()

The manila.scheduler.weighers.capacity Weigher

Capacity Weigher. Weigh hosts by their virtual or actual free capacity.

For thin provisioning, weigh hosts by their virtual free capacity calculated by the total capacity multiplied by the max over subscription ratio and subtracting the provisioned capacity; Otherwise, weigh hosts by their actual free capacity, taking into account the reserved space.

The default is to spread shares across all hosts evenly. If you prefer stacking, you can set the ‘capacity_weight_multiplier’ option to a negative number and the weighing has the opposite effect of the default.

class CapacityWeigher

Bases: manila.scheduler.weighers.base_host.BaseHostWeigher

weigh_objects(weighed_obj_list, weight_properties)

Weigh multiple objects.

Override in a subclass if you need access to all objects in order to calculate weighers. Do not modify the weight of an object here, just return a list of weighers.

weight_multiplier()

Override the weight multiplier.

The manila.scheduler.weighers.pool Weigher

class PoolWeigher

Bases: manila.scheduler.weighers.base_host.BaseHostWeigher

weight_multiplier()

Override the weight multiplier.

Creative Commons Attribution 3.0 License

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.