5.21. OpenStack load testing

status:

ready

version:

1.0

Abstract:

This test plan describes a set of scenarios to measure maximum number of requests per second for a particular OpenStack API service.

Conventions:
  • RPS Requests-per-second - number of requests send to an API endpoint per second

5.21.1. Test Plan

Rally is a benchmarking tool that was designed specifically for OpenStack API testing. To make this possible, Rally automates and unifies multi-node OpenStack deployment, cloud verification, benchmarking & profiling. This is a simple way to check cloud workability and performance of control plane operations running on it. This test plan describes several Rally scenarios that can cover almost all most important in perms of performance basic cloud operations e.g. VMs creation, work with the security groups, authentication and other operations.

5.21.1.1. Test Environment

5.21.1.1.1. Preparation

This test plan can be executed with help of Rally tool.

5.21.1.1.2. Environment description

The environment description includes hardware specification of servers, network parameters, operation system and OpenStack deployment characteristics.

5.21.1.1.2.1. Hardware

This section contains list of all types of hardware nodes.

Parameter Value Comments
model   e.g. Supermicro X9SRD-F
CPU   e.g. 6 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
role   e.g. compute or network
5.21.1.1.2.2. Network

This section contains list of interfaces and network parameters. For complicated cases this section may include topology diagram and switch parameters.

Parameter Value Comments
network role   e.g. provider or public
card model   e.g. Intel
driver   e.g. ixgbe
speed   e.g. 10G or 1G
MTU   e.g. 9000
offloading modes   e.g. default
5.21.1.1.2.3. Software

This section describes installed software.

Parameter Value Comments
OS   e.g. Ubuntu 14.04.3
OpenStack   e.g. Mitaka
Hypervisor   e.g. KVM
Neutron plugin   e.g. ML2 + OVS
L2 segmentation   e.g. VLAN / VxLAN / GRE
virtual routers   e.g. HA / DVR

5.21.1.2. Test Case 1: Neutron Load Testing

5.21.1.2.1. Description

In this scenario Neutron API is loaded with constant flow of requests. The number of requests per second is tuned to keep success rate at 100%. Duration of operations is collected and stats are calculated. The result is presented in table format.

5.21.1.2.2. List of performance metrics

Priority Value Measurement Units Description
1   RPS Number of API requests per second

5.21.2. Tools

This test plan can be executed with help of Rally. Following is the list of Rally scenarios.

5.21.2.1. Create Neutron networks

In this scenario Rally creates networks at constant rate. The single iteration includes the following operations:

  1. create network
{
  "NeutronNetworks.create_network": [
    {
      "runner": {
        "rps": 5,
        "type": "rps",
        "times": 1000
      },
      "args": {
        "network_create_args": {}
      },
      "sla": {
        "failure_rate": {
          "max": 0
        }
      },
      "context": {
        "users": {
          "users_per_tenant": 4,
          "project_domain": "default",
          "user_choice_method": "random",
          "user_domain": "default",
          "tenants": 30,
          "resource_management_workers": 30
        },
        "quotas": {
          "neutron": {
            "network": -1
          }
        }
      }
    }
  ]
}

5.21.2.2. Create Neutron network with 1 port

In this scenario Rally creates Neutron network with one port at constant rate. The single iteration includes the following operations:

  1. create network
  2. create one port
{
  "NeutronNetworks.create_port": [
    {
      "runner": {
        "rps": 5,
        "type": "rps",
        "times": 500
      },
      "args": {
        "network_create_args": {},
        "ports_per_network": 1,
        "port_create_args": {}
      },
      "sla": {
        "failure_rate": {
          "max": 0
        }
      },
      "context": {
        "users": {
          "users_per_tenant": 2,
          "project_domain": "default",
          "user_choice_method": "random",
          "user_domain": "default",
          "tenants": 20,
          "resource_management_workers": 30
        },
        "quotas": {
          "neutron": {
            "network": -1,
            "port": -1
          }
        }
      }
    }
  ]
}

5.21.2.3. Create Neutron ports

In this scenario Rally creates Neutron network and ports at constant rate. The single iteration includes the following operations:

  1. create network
  2. create 20 ports in each of network
  3. list all ports
{
  "NeutronNetworks.create_and_list_ports": [
    {
      "runner": {
        "rps": 0.1,
        "type": "rps",
        "times": 50
      },
      "args": {
        "network_create_args": {},
        "ports_per_network": 20,
        "port_create_args": {}
      },
      "sla": {
        "failure_rate": {
          "max": 0
        }
      },
      "context": {
        "users": {
          "users_per_tenant": 2,
          "project_domain": "default",
          "user_choice_method": "random",
          "user_domain": "default",
          "tenants": 20,
          "resource_management_workers": 30
        },
        "quotas": {
          "neutron": {
            "network": -1,
            "port": -1
          }
        }
      }
    }
  ]
}

5.21.2.4. Create Cinder volumes

In this scenario Rally creates volumes at constant rate. The single iteration includes the following operations:

  1. create volume
{
  "CinderVolumes.create_volume": [
    {
      "runner": {
        "rps": 8,
        "type": "rps",
        "times": 1000
      },
      "args": {
        "size": 10
      },
      "sla": {
        "failure_rate": {
          "max": 0
        }
      },
      "context": {
        "users": {
          "users_per_tenant": 1,
          "project_domain": "default",
          "user_choice_method": "random",
          "user_domain": "default",
          "tenants": 1,
          "resource_management_workers": 5
        },
        "quotas": {
          "cinder": {
            "gigabytes": -1,
            "volumes": -1,
            "snapshots": -1
          }
        }
      }
    }
  ]
}

5.21.2.5. Boot Nova servers

In this scenario Rally boots Nova servers at constant rate:

  1. boot server
{
  "NovaServers.boot_server": [
    {
      "runner": {
        "rps": 1.7,
        "type": "rps",
        "times": 500
      },
      "args": {
        "flavor": {
          "name": "m1.tiny"
        },
        "image": {
          "name": "TestVM"
        }
      },
      "sla": {
        "failure_rate": {
          "max": 0
        }
      },
      "context": {
        "network": {
          "subnets_per_network": 1,
          "start_cidr": "100.1.0.0/21",
          "networks_per_tenant": 1
        },
        "users": {
          "users_per_tenant": 2,
          "project_domain": "default",
          "user_choice_method": "random",
          "user_domain": "default",
          "tenants": 2,
          "resource_management_workers": 30
        },
        "quotas": {
          "neutron": {
            "subnet": -1,
            "network": -1,
            "port": -1
          },
          "nova": {
            "ram": -1,
            "floating_ips": -1,
            "security_group_rules": -1,
            "instances": -1,
            "cores": -1,
            "security_groups": -1
          }
        }
      }
    }
  ]
}

5.21.3. Reports

Test plan execution reports: