Object expirer configuration

Object expirer configuration

Find an example object expirer configuration at etc/object-expirer.conf-sample in the source code repository.

The available configuration options are:

Description of configuration options for [DEFAULT] in object-expirer.conf
Configuration option = Default value Description
log_address = /dev/log Location where syslog sends the logs to
log_custom_handlers = Comma-separated list of functions to call to setup custom log handlers.
log_facility = LOG_LOCAL0 Syslog log facility
log_level = INFO Logging level
log_max_line_length = 0 Caps the length of log lines to the value given; no limit if set to 0, the default.
log_name = swift Label used when logging
log_statsd_default_sample_rate = 1.0 Defines the probability of sending a sample for any given event or timing measurement.
log_statsd_host = localhost If not set, the StatsD feature is disabled.
log_statsd_metric_prefix = Value will be prepended to every metric sent to the StatsD server.
log_statsd_port = 8125 Port value for the StatsD server.
log_statsd_sample_rate_factor = 1.0 Not recommended to set this to a value less than 1.0, if frequency of logging is too high, tune the log_statsd_default_sample_rate instead.
log_udp_host = If not set, the UDP receiver for syslog is disabled.
log_udp_port = 514 Port value for UDP receiver, if enabled.
swift_dir = /etc/swift Swift configuration directory
user = swift User to run as
Description of configuration options for [app-proxy-server] in object-expirer.conf
Configuration option = Default value Description
use = egg:swift#proxy Entry point of paste.deploy in the server
Description of configuration options for [filter-cache] in object-expirer.conf
Configuration option = Default value Description
use = egg:swift#memcache Entry point of paste.deploy in the server
Description of configuration options for [filter-catch_errors] in object-expirer.conf
Configuration option = Default value Description
use = egg:swift#catch_errors Entry point of paste.deploy in the server
Description of configuration options for [filter-proxy-logging] in object-expirer.conf
Configuration option = Default value Description
access_log_address = /dev/log Location where syslog sends the logs to. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_facility = LOG_LOCAL0 Syslog facility to receive log lines. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_headers = false Header to receive log lines. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_headers_only = If access_log_headers is True and access_log_headers_only is set only these headers are logged. Multiple headers can be defined as comma separated list like this: access_log_headers_only = Host, X-Object-Meta-Mtime
access_log_level = INFO Syslog logging level to receive log lines. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_name = swift Label used when logging. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_statsd_default_sample_rate = 1.0 Defines the probability of sending a sample for any given event or timing measurement. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_statsd_host = localhost You can use log_statsd_* from [DEFAULT], or override them here. StatsD server. IPv4/IPv6 addresses and hostnames are supported. If a hostname resolves to an IPv4 and IPv6 address, the IPv4 address will be used.
access_log_statsd_metric_prefix = Value will be prepended to every metric sent to the StatsD server. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_statsd_port = 8125 Port value for the StatsD server. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_statsd_sample_rate_factor = 1.0 Not recommended to set this to a value less than 1.0, if frequency of logging is too high, tune the log_statsd_default_sample_rate instead. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_udp_host = If not set, the UDP receiver for syslog is disabled. If not set, logging directives from [DEFAULT] without “access_” will be used.
access_log_udp_port = 514 Port value for UDP receiver, if enabled. If not set, logging directives from [DEFAULT] without “access_” will be used.
log_statsd_valid_http_methods = GET,HEAD,POST,PUT,DELETE,COPY,OPTIONS What HTTP methods are allowed for StatsD logging (comma-sep). request methods not in this list will have “BAD_METHOD” for the <verb> portion of the metric.
reveal_sensitive_prefix = 16

By default, the X-Auth-Token is logged. To obscure the value, set reveal_sensitive_prefix to the number of characters to log. For example, if set to 12, only the first 12 characters of the token appear in the log. An unauthorized access of the log file won’t allow unauthorized usage of the token. However, the first 12 or so characters is unique enough that you can trace/debug token usage. Set to 0 to suppress the token completely (replaced by ‘...’ in the log).

Note

reveal_sensitive_prefix will not affect the value logged with access_log_headers=True.

use = egg:swift#proxy_logging Entry point of paste.deploy in the server
Description of configuration options for [object-expirer] in object-expirer.conf
Configuration option = Default value Description
auto_create_account_prefix = . Prefix to use when automatically creating accounts
concurrency = 1 Number of replication workers to spawn
expiring_objects_account_name = expiring_objects Account name for expiring objects.
interval = 300 Minimum time for a pass to take
process = 0 (it will actually accept(2) N + 1). Setting this to one (1) will only handle one request at a time, without accepting another request concurrently.
processes = 0 for each port (disk) in the ring. If you have 24 disks per server, and this setting is 4, then each storage node will have 1 + (24 * 4) = 97 total object-server processes running. This gives complete I/O isolation, drastically reducing the impact of slow disks on storage node performance. The object-replicator and object-reconstructor need to see this setting too, so it must be in the [DEFAULT] section.
reclaim_age = 604800 Time elapsed in seconds before an object can be reclaimed
recon_cache_path = /var/cache/swift Directory where stats for a few items will be stored
report_interval = 300 Interval in seconds between reports.
Description of configuration options for [pipeline-main] in object-expirer.conf
Configuration option = Default value Description
pipeline = catch_errors proxy-logging cache proxy-server Pipeline to use for processing operations.

Sample object expirer configuration file

[DEFAULT]
# swift_dir = /etc/swift
# user = swift
# You can specify default log routing here if you want:
# log_name = swift
# log_facility = LOG_LOCAL0
# log_level = INFO
# log_address = /dev/log
# The following caps the length of log lines to the value given; no limit if
# set to 0, the default.
# log_max_line_length = 0
#
# comma separated list of functions to call to setup custom log handlers.
# functions get passed: conf, name, log_to_console, log_route, fmt, logger,
# adapted_logger
# log_custom_handlers =
#
# If set, log_udp_host will override log_address
# log_udp_host =
# log_udp_port = 514
#
# You can enable StatsD logging here:
# log_statsd_host =
# log_statsd_port = 8125
# log_statsd_default_sample_rate = 1.0
# log_statsd_sample_rate_factor = 1.0
# log_statsd_metric_prefix =
#
# You can set scheduling priority of processes. Niceness values range from -20
# (most favorable to the process) to 19 (least favorable to the process).
# nice_priority =
#
# You can set I/O scheduling class and priority of processes. I/O niceness
# class values are realtime, best-effort and idle. I/O niceness
# priority is a number which goes from 0 to 7. The higher the value, the lower
# the I/O priority of the process. Work only with ionice_class.
# ionice_class =
# ionice_priority =

[object-expirer]
# interval = 300
# auto_create_account_prefix = .
# expiring_objects_account_name = expiring_objects
# report_interval = 300
# concurrency is the level of concurrency o use to do the work, this value
# must be set to at least 1
# concurrency = 1
# processes is how many parts to divide the work into, one part per process
#   that will be doing the work
# processes set 0 means that a single process will be doing all the work
# processes can also be specified on the command line and will override the
#   config value
# processes = 0
# process is which of the parts a particular process will work on
# process can also be specified on the command line and will override the config
#   value
# process is "zero based", if you want to use 3 processes, you should run
#  processes with process set to 0, 1, and 2
# process = 0
# The expirer will re-attempt expiring if the source object is not available
# up to reclaim_age seconds before it gives up and deletes the entry in the
# queue.
# reclaim_age = 604800
# recon_cache_path = /var/cache/swift
#
# You can set scheduling priority of processes. Niceness values range from -20
# (most favorable to the process) to 19 (least favorable to the process).
# nice_priority =
#
# You can set I/O scheduling class and priority of processes. I/O niceness
# class values are realtime, best-effort and idle. I/O niceness
# priority is a number which goes from 0 to 7. The higher the value, the lower
# the I/O priority of the process. Work only with ionice_class.
# ionice_class =
# ionice_priority =

[pipeline:main]
pipeline = catch_errors proxy-logging cache proxy-server

[app:proxy-server]
use = egg:swift#proxy
# See proxy-server.conf-sample for options

[filter:cache]
use = egg:swift#memcache
# See proxy-server.conf-sample for options

[filter:catch_errors]
use = egg:swift#catch_errors
# See proxy-server.conf-sample for options

[filter:proxy-logging]
use = egg:swift#proxy_logging
# If not set, logging directives from [DEFAULT] without "access_" will be used
# access_log_name = swift
# access_log_facility = LOG_LOCAL0
# access_log_level = INFO
# access_log_address = /dev/log
#
# If set, access_log_udp_host will override access_log_address
# access_log_udp_host =
# access_log_udp_port = 514
#
# You can use log_statsd_* from [DEFAULT] or override them here:
# access_log_statsd_host =
# access_log_statsd_port = 8125
# access_log_statsd_default_sample_rate = 1.0
# access_log_statsd_sample_rate_factor = 1.0
# access_log_statsd_metric_prefix =
# access_log_headers = false
#
# If access_log_headers is True and access_log_headers_only is set only
# these headers are logged. Multiple headers can be defined as comma separated
# list like this: access_log_headers_only = Host, X-Object-Meta-Mtime
# access_log_headers_only =
#
# By default, the X-Auth-Token is logged. To obscure the value,
# set reveal_sensitive_prefix to the number of characters to log.
# For example, if set to 12, only the first 12 characters of the
# token appear in the log. An unauthorized access of the log file
# won't allow unauthorized usage of the token. However, the first
# 12 or so characters is unique enough that you can trace/debug
# token usage. Set to 0 to suppress the token completely (replaced
# by '...' in the log).
# Note: reveal_sensitive_prefix will not affect the value
# logged with access_log_headers=True.
# reveal_sensitive_prefix = 16
#
# What HTTP methods are allowed for StatsD logging (comma-sep); request methods
# not in this list will have "BAD_METHOD" for the <verb> portion of the metric.
# log_statsd_valid_http_methods = GET,HEAD,POST,PUT,DELETE,COPY,OPTIONS
Creative Commons Attribution 3.0 License

Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.