Neutron Metering system¶
The Neutron metering service enables operators to account the traffic in/out
of the OpenStack environment. The concept is quite simple, operators can
create metering labels, and decide if the labels are applied to all projects
(tenants) or if they are applied to a specific one. Then, the operator needs
to create traffic rules in the metering labels. The traffic rules are used
to match traffic in/out of the OpenStack environment, and the accounting of
packets and bytes is sent to the notification queue for further processing
by Ceilometer (or some other system that is consuming that queue). The
message sent in the queue is of type event
. Therefore, it requires an
event processing configuration to be added/enabled in Ceilometer.
The metering agent has the following configurations:
driver
: the driver used to implement the metering rules. The default isneutron.services.metering.drivers.noop
, which means, we do not execute anything in the networking host. The only driver implemented so far isneutron.services.metering.drivers.iptables.iptables_driver.IptablesMeteringDriver
. Therefore, onlyiptables
is supported so far;measure_interval
: the interval in seconds used to gather the bytes and packets information from the network plane. The default value is30
seconds;report_interval
: the interval in seconds used to generated the report (message) of the data that is gathered. The default value is300
seconds.granular_traffic_data
: Defines if the metering agent driver should present traffic data in a granular fashion, instead of grouping all of the traffic data for all projects and routers where the labels were assigned to. The default value isFalse
for backward compatibility.
Non-granular traffic messages¶
The non-granular (granular_traffic_data = False
) traffic messages (here
also called as legacy) have the following format; bear in mind that if labels
are shared, then the counters are for all routers of all projects where the
labels were applied.
{
"pkts": "<the number of packets that matched the rules of the labels>",
"bytes": "<the number of bytes that matched the rules of the labels>",
"time": "<seconds between the first data collection and the last one>",
"first_update": "timeutils.utcnow_ts() of the first collection",
"last_update": "timeutils.utcnow_ts() of the last collection",
"host": "<neutron metering agent host name>",
"label_id": "<the label id>",
"tenant_id": "<the tenant id>"
}
The first_update
and last_update
timestamps represent the moment
when the first and last data collection happened within the report interval.
On the other hand, the time
represents the difference between those two
timestamp.
The tenant_id
is only consistent when labels are not shared. Otherwise,
they will contain the project id of the last router of the last project
processed when the agent is started up. In other words, it is better not
use it when dealing with shared labels.
All of the messages generated in this configuration mode are sent to the
message bus as l3.meter
events.
Granular traffic messages¶
The granular (granular_traffic_data = True
) traffic messages allow
operators to obtain granular information for shared metering labels.
Therefore, a single label, when configured as shared=True
and applied in
all projects/routers of the environment, it will generate data in a granular
fashion.
It (the metering agent) will account the traffic counter data in the following granularities.
label
– all of the traffic counter for a given label. One must bear in mind that a label can be assigned to multiple routers. Therefore, this granularity represents all aggregation for all data for all routers of all projects where the label has been applied.router
– all of the traffic counter for all labels that are assigned to the router.project
– all of the traffic counters for all labels of all routers that a project has.router-label
– all of the traffic counters for a router and the given label.project-label
– all of the traffic counters for all routers of a project that have a given label.
Each granularity presented here is sent to the message bus with different events types that vary according to the granularity. The mapping between granularity and event type is presented as follows.
label
– event typel3.meter.label
.router
– event typel3.meter.router
.project
– event typel3.meter.project
..router-label
– event typel3.meter.label_router
.project-label
– event typel3.meter.label_project
.
Furthermore, we have metadata that is appended to the messages depending on the granularity. As follows we present the mapping between the granularities and the metadata that will be available.
label
,router-label
, andproject-label
granularities – have the metadatalabel_id
,label_name
,label_shared
,project_id
(if shared, this value will come withall
for thelabel
granularity), androuter_id
(only forrouter-label
granularity).The
router
granularity – has therouter_id
andproject_id
metadata.The
project
granularity only has theproject_id
metadata.
The message will also contain some attributes that can be found in the
legacy mode such as bytes
, pkts
, time
, first_update
,
last_update
, and host
. As follows we present an example of JSON message
with all of the possible attributes.
{
"resource_id": "router-f0f745d9a59c47fdbbdd187d718f9e41-label-00c714f1-49c8-462c-8f5d-f05f21e035c7",
"project_id": "f0f745d9a59c47fdbbdd187d718f9e41",
"first_update": 1591058790,
"bytes": 0,
"label_id": "00c714f1-49c8-462c-8f5d-f05f21e035c7",
"label_name": "test1",
"last_update": 1591059037,
"host": "<hostname>",
"time": 247,
"pkts": 0,
"label_shared": true
}
The resource_id
is a unique identified for the “resource” being
monitored. Here we consider a resource to be any of the granularities that
we handle.
Sample of metering_agent.ini¶
As follows we present all of the possible configuration one can use in the metering agent init file.
DEFAULT¶
- ovs_use_veth¶
- Type:
boolean
- Default:
False
Uses veth for an OVS interface or not. Support kernels with limited namespace support (e.g. RHEL 6.5) and rate limiting on router’s gateway port so long as ovs_use_veth is set to True.
- interface_driver¶
- Type:
string
- Default:
<None>
The driver used to manage virtual interfaces.
- rpc_response_max_timeout¶
- Type:
integer
- Default:
600
Maximum seconds to wait for a response from an RPC call.
- driver¶
- Type:
string
- Default:
neutron.services.metering.drivers.noop.noop_driver.NoopMeteringDriver
Metering driver
- measure_interval¶
- Type:
integer
- Default:
30
Interval between two metering measures
- report_interval¶
- Type:
integer
- Default:
300
Interval between two metering reports
- granular_traffic_data¶
- Type:
boolean
- Default:
False
Defines if the metering agent driver should present traffic data in a granular fashion, instead of grouping all of the traffic data for all projects and routers where the labels were assigned to. The default value is False for backward compatibility.
- debug¶
- Type:
boolean
- Default:
False
- Mutable:
This option can be changed without restarting.
If set to true, the logging level will be set to DEBUG instead of the default INFO level.
- log_config_append¶
- Type:
string
- Default:
<None>
- Mutable:
This option can be changed without restarting.
The name of a logging configuration file. This file is appended to any existing logging configuration files. For details about logging configuration files, see the Python logging module documentation. Note that when logging configuration files are used then all logging configuration is set in the configuration file and other logging configuration options are ignored (for example, log-date-format).
¶ Group
Name
DEFAULT
log-config
DEFAULT
log_config
- log_date_format¶
- Type:
string
- Default:
%Y-%m-%d %H:%M:%S
Defines the format string for %(asctime)s in log records. Default: the value above . This option is ignored if log_config_append is set.
- log_file¶
- Type:
string
- Default:
<None>
(Optional) Name of log file to send logging output to. If no default is set, logging will go to stderr as defined by use_stderr. This option is ignored if log_config_append is set.
¶ Group
Name
DEFAULT
logfile
- log_dir¶
- Type:
string
- Default:
<None>
(Optional) The base directory used for relative log_file paths. This option is ignored if log_config_append is set.
¶ Group
Name
DEFAULT
logdir
- watch_log_file¶
- Type:
boolean
- Default:
False
Uses logging handler designed to watch file system. When log file is moved or removed this handler will open a new log file with specified path instantaneously. It makes sense only if log_file option is specified and Linux platform is used. This option is ignored if log_config_append is set.
- use_syslog¶
- Type:
boolean
- Default:
False
Use syslog for logging. Existing syslog format is DEPRECATED and will be changed later to honor RFC5424. This option is ignored if log_config_append is set.
- use_journal¶
- Type:
boolean
- Default:
False
Enable journald for logging. If running in a systemd environment you may wish to enable journal support. Doing so will use the journal native protocol which includes structured metadata in addition to log messages.This option is ignored if log_config_append is set.
- syslog_log_facility¶
- Type:
string
- Default:
LOG_USER
Syslog facility to receive log lines. This option is ignored if log_config_append is set.
- use_json¶
- Type:
boolean
- Default:
False
Use JSON formatting for logging. This option is ignored if log_config_append is set.
- use_stderr¶
- Type:
boolean
- Default:
False
Log output to standard error. This option is ignored if log_config_append is set.
- use_eventlog¶
- Type:
boolean
- Default:
False
Log output to Windows Event Log.
Warning
This option is deprecated for removal. Its value may be silently ignored in the future.
- Reason:
Windows support is no longer maintained.
- log_rotate_interval¶
- Type:
integer
- Default:
1
The amount of time before the log files are rotated. This option is ignored unless log_rotation_type is set to “interval”.
- log_rotate_interval_type¶
- Type:
string
- Default:
days
- Valid Values:
Seconds, Minutes, Hours, Days, Weekday, Midnight
Rotation interval type. The time of the last file change (or the time when the service was started) is used when scheduling the next rotation.
- max_logfile_count¶
- Type:
integer
- Default:
30
Maximum number of rotated log files.
- max_logfile_size_mb¶
- Type:
integer
- Default:
200
Log file maximum size in MB. This option is ignored if “log_rotation_type” is not set to “size”.
- log_rotation_type¶
- Type:
string
- Default:
none
- Valid Values:
interval, size, none
Log rotation type.
Possible values
- interval
Rotate logs at predefined time intervals.
- size
Rotate logs once they reach a predefined size.
- none
Do not rotate log files.
- logging_context_format_string¶
- Type:
string
- Default:
%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(user_identity)s] %(instance)s%(message)s
Format string to use for log messages with context. Used by oslo_log.formatters.ContextFormatter
- logging_default_format_string¶
- Type:
string
- Default:
%(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s
Format string to use for log messages when context is undefined. Used by oslo_log.formatters.ContextFormatter
- logging_debug_format_suffix¶
- Type:
string
- Default:
%(funcName)s %(pathname)s:%(lineno)d
Additional data to append to log message when logging level for the message is DEBUG. Used by oslo_log.formatters.ContextFormatter
- logging_exception_prefix¶
- Type:
string
- Default:
%(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s
Prefix each line of exception output with this format. Used by oslo_log.formatters.ContextFormatter
- logging_user_identity_format¶
- Type:
string
- Default:
%(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s
Defines the format string for %(user_identity)s that is used in logging_context_format_string. Used by oslo_log.formatters.ContextFormatter
- default_log_levels¶
- Type:
list
- Default:
['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO']
List of package logging levels in logger=LEVEL pairs. This option is ignored if log_config_append is set.
- publish_errors¶
- Type:
boolean
- Default:
False
Enables or disables publication of error events.
- instance_format¶
- Type:
string
- Default:
"[instance: %(uuid)s] "
The format for an instance that is passed with the log message.
- instance_uuid_format¶
- Type:
string
- Default:
"[instance: %(uuid)s] "
The format for an instance UUID that is passed with the log message.
- rate_limit_interval¶
- Type:
integer
- Default:
0
Interval, number of seconds, of log rate limiting.
- rate_limit_burst¶
- Type:
integer
- Default:
0
Maximum number of logged messages per rate_limit_interval.
- rate_limit_except_level¶
- Type:
string
- Default:
CRITICAL
Log level name used by rate limiting: CRITICAL, ERROR, INFO, WARNING, DEBUG or empty string. Logs with level greater or equal to rate_limit_except_level are not filtered. An empty string means that all levels are filtered.
- fatal_deprecations¶
- Type:
boolean
- Default:
False
Enables or disables fatal status of deprecations.
agent¶
- report_interval¶
- Type:
floating point
- Default:
30
Seconds between nodes reporting state to server; should be less than agent_down_time, best if it is half or less than agent_down_time.
- log_agent_heartbeats¶
- Type:
boolean
- Default:
False
Log agent heartbeats
ovs¶
- ovsdb_connection¶
- Type:
string
- Default:
tcp:127.0.0.1:6640
The connection string for the OVSDB backend. Will be used for all OVSDB commands and by ovsdb-client when monitoring
- ssl_key_file¶
- Type:
string
- Default:
<None>
The SSL private key file to use when interacting with OVSDB. Required when using an “ssl:” prefixed ovsdb_connection
- ssl_cert_file¶
- Type:
string
- Default:
<None>
The SSL certificate file to use when interacting with OVSDB. Required when using an “ssl:” prefixed ovsdb_connection
- ssl_ca_cert_file¶
- Type:
string
- Default:
<None>
The Certificate Authority (CA) certificate to use when interacting with OVSDB. Required when using an “ssl:” prefixed ovsdb_connection
- ovsdb_debug¶
- Type:
boolean
- Default:
False
Enable OVSDB debug logs
- ovsdb_timeout¶
- Type:
integer
- Default:
10
Timeout in seconds for OVSDB commands. If the timeout expires, OVSDB commands will fail with ALARMCLOCK error.
- bridge_mac_table_size¶
- Type:
integer
- Default:
50000
The maximum number of MAC addresses to learn on a bridge managed by the Neutron OVS agent. Values outside a reasonable range (10 to 1,000,000) might be overridden by Open vSwitch according to the documentation.
- igmp_snooping_enable¶
- Type:
boolean
- Default:
False
Enable IGMP snooping for integration bridge. If this option is set to True, support for Internet Group Management Protocol (IGMP) is enabled in integration bridge.
- igmp_flood¶
- Type:
boolean
- Default:
False
Multicast packets (except reports) are unconditionally forwarded to the ports bridging a logical network to a physical network.
- igmp_flood_reports¶
- Type:
boolean
- Default:
True
Multicast reports are unconditionally forwarded to the ports bridging a logical network to a physical network.
- igmp_flood_unregistered¶
- Type:
boolean
- Default:
False
This option enables or disables flooding of unregistered multicast packets to all ports. If False, The switch will send unregistered multicast packets only to ports connected to multicast routers.