Scenario runner as a plugin

Let's create a scenario runner plugin that runs a given benchmark scenario a random number of times (chosen at random from a given range).


Inherit a class for your plugin from the base ScenarioRunner class and implement its API (the _run_scenario() method):

import random

from rally.task import runner
from rally import consts

class RandomTimesScenarioRunner(runner.ScenarioRunner):
    """Sample scenario runner plugin.

    Run scenario random number of times, which is chosen between min_times and

        "type": "object",
        "$schema": consts.JSON_SCHEMA,
        "properties": {
            "type": {
                "type": "string"
            "min_times": {
                "type": "integer",
                "minimum": 1
            "max_times": {
                "type": "integer",
                "minimum": 1
        "additionalProperties": True

    def _run_scenario(self, cls, method_name, context, args):
        # runners settings are stored in self.config
        min_times = self.config.get('min_times', 1)
        max_times = self.config.get('max_times', 1)

        for i in range(random.randrange(min_times, max_times)):
            run_args = (i, cls, method_name,
                        runner._get_scenario_context(context), args)
            result = runner._run_scenario_once(run_args)
            # use self.send_result for result of each iteration


You can refer to your scenario runner in the benchmark task configuration files in the same way as any other runners. Don't forget to put your runner-specific parameters in the configuration as well ("min_times" and "max_times" in our example):

    "Dummy.dummy": [
            "runner": {
                "type": "random_times",
                "min_times": 10,
                "max_times": 20,
            "context": {
                "users": {
                    "tenants": 1,
                    "users_per_tenant": 1

Different plugin samples are available here.