5.7. Fuel Containerized Control Plane upgradability performance testing

Abstract:This test plan aims to provide set of tests to identify OpenStack performance against given containerized OpenStack cloud (installed on the top of pre-deployed Kubernetes cluster) using simple minimalistic set of Rally tests during upgrade from Mitaka to Newton.

5.7.1. Test Plan

This test plan covers basic network performance with long-running test suites to verify cloud network stability and performance during update Open Stack from Mitaka to Newton. Test Environment Preparation

This test plan is performed against existing OpenStack cloud installed on top of pre-deployed Kubernetes cluster with fuel-ccp tool with pre-installed Rally framework. Environment description

The environment description includes hardware specification of servers, network parameters, operation system and OpenStack deployment characteristics. Hardware

This section contains list of all types of hardware nodes (table below is an example).

Parameter Value Comments
model   e.g. Supermicro X9SRD-F
CPU   e.g. 6 x Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz
role   e.g. compute or network Network

This section contains list of interfaces and network parameters. For complicated cases this section may include topology diagram and switch parameters (table below is an example).

Parameter Value Comments
network role   e.g. provider or public
card model   e.g. Intel
driver   e.g. ixgbe
speed   e.g. 10G or 1G
MTU   e.g. 9000
offloading modes   e.g. default Software

This section describes installed software (table below is an example).

Parameter Value Comments
OS   e.g. Ubuntu 14.04.3
OpenStack   e.g. Mitaka
Hypervisor   e.g. KVM
Neutron plugin   e.g. ML2 + OVS
L2 segmentation   e.g. VLAN / VxLAN / GRE
virtual routers   e.g. HA / DVR Test tool

Rally is a benchmarking tool that was designed specifically for OpenStack API testing. To make this possible, Rally automates and unifies multi-node OpenStack deployment, cloud verification, benchmarking & profiling. This is a simple way to check cloud workability and performance of control plane operations running on it. Test Case 1: Boot and delete server during Open Stack update Description

The most user-facing control plane operation is new virtual machine creation. This scenario covers the most basic OpenStack server creation to present the baseline numbers for Nova (OpenStack Compute) control plane. Parameters

Name Description
IMAGE Image from which boot server
FLAVOR Flavor type from which boot server
ASSIGN_NIC Bool, whether or not to auto assign NICs in Rally scenario
CONCURRENCY Amount of parallel executors
ITERATIONS Total amount of iterations processed by all executors List of performance metrics

Priority Value Measurement Units Description
1   sec Time of atomic operations Measuring performance values

  1. Create server with FLAVOR flavor from IMAGE image through Nova API
  2. Delete server through Nova API.

These 2 steps executed successively in CONCURRENCY parallel executors. ASSIGN_NIC parameter reflects Rally scenario configuration whether to assign NIC to the booted server in automatic fashion.

One cycle of these 2 steps is called an iteration. ITERATIONS is a total amount of iterations which was processed by executors.

During this testing

At the end of this test case you should calculate average, 90% percentile, 50% percentile, minimum and maximum for each step. You need to fill the following tables with calculated values: Nova

Operation Mean (sec) 90%ile (sec) 50%ile (sec) Max (sec) Min (sec)
delete_server   Example of Rally scenario configuration