2023.1 Series (21.2.0 - 21.4.x) Release Notes¶
21.4.0-19¶
Bug Fixes¶
Fixes Ironic integration with Cinder because of changes which resulted as part of the recent Security related fix in bug 2004555. The work in Ironic to track this fix was logged in bug 2019892. Ironic now sends a service token to Cinder, which allows for access restrictions added as part of the original CVE-2023-2088 fix to be appropriately bypassed. Ironic was not vulnerable, but the restrictions added as a result did impact Ironic’s usage. This is because Ironic volume attachments are not on a shared “compute node”, but instead mapped to the physical machines and Ironic handles the attachment life-cycle after initial attachment.
Fixes
Invalid cross-device link
in some cases when usingfile://
image URLs.
Fixes bug of iRMC driver in parse_driver_info where, if FIPS is enabled, SNMP version is always required to be version 3 even though iRMC driver’s xxx_interface doesn’t use SNMP actually.
Fixes an issue where an agent token could be inadvertently orphaned if a node is already in the target power state when we attempt to turn the node off.
Fixes scope classification check with the “self_owned_node” policy check where it was limited to check execution with only project scoped, so system scoped users who ticked the policy endpoint would basically get an incorrect error.
Fixes the bug where provisioning a Redfish managed node fails if the BMC doesn’t support EthernetInterfaces attribute, even if MAC address information is provided manually. This is done by handling of MissingAttributeError sushy exception in get_mac_addresses() method. This fix is needed to successfully provision machines such as Cisco UCSB and UCSX.
No longer re-calculates checksums for images that are already raw. Previously, it would cause significant delays in deploying raw images.
Fixes an issue where an agent token was being orphaned if a baremetal node timed out during cleaning operations, leading to issues where the node would not be able to establish a new token with Ironic upon future in some cases. We now always wipe the token in this case.
21.4.0¶
Prelude¶
The Ironic team hereby announces the release of OpenStack 2023.1 (Ironic 23.4.0). This repesents the completion of a six month development cycle, which primarily focused on internal and scaling improvements. Those improvements included revamping the database layer to improve performance and ensure compatability with new versions of SQLAlchemy, enhancing the ironic-conductor service to export application metrics to prometheus via the ironic-prometheus-exporter, and the addition of a new API concept of node sharding to help with scaling of services that make frequent API calls to Ironic. The new Ironic release also comes with a slew of bugfixes for Ironic services and hardware drivers. We sincerely hope you enjoy it!
New Features¶
Adds support for the
service
role, which is intended for service to service communication, such as for those whereironic-inspector
,nova-compute
, ornetworking-baremetal
needs to communicate with Ironic’s API.
Adds the ability for Ironic to send conductor process metrics for monitoring. This requires the use of a new
[metrics]backend
option value ofcollector
. This data was previously only available through the use of statsd. This requiresironic-lib
version5.4.0
or newer. This capability can be disabled using the[sensor_data]enable_for_conductor
option if set to False.
Adds a
[sensor_data]enable_for_nodes
configuration option to allow operators to disable sending node metric data via the message bus notifier.
Adds a new gauge metric
ConductorManager.PowerSyncNodesCount
which tracks the nodes considered for power state synchrnozation.
Adds a new gauge metric
ConductorManager.PowerSyncRecoveryNodeCount
which represents the number of nodes which are being evaluated for power state recovery checking.
Adds a new gauge metric
ConductorManager.SyncLocalStateNodeCount
which represents the number of nodes being tracked locally by the conductor.
There are now configurable random wait times for fake drivers in a new ironic.conf [fake] section. Each supported driver having one configuration option controlling the delay. These delays are applied to operations which typically block in other drivers. This allows more realistic scenarios to be arranged for performance and functional testing of ironic itself.
Adds support for setting a shard key on a node, and filtering node or port lists by shard. This shard key is not used for any purpose internally in Ironic, but instead is intended to allow API clients to filter for a subset of nodes or ports. Being able to fetch only a subset of nodes or ports is useful for parallelizing any operational task that needs to be performed across all nodes or ports.
Adds support for querying for nodes which are sharded or unsharded. This is useful for allowing operators to find nodes which have not been assigned a shard key.
Adds support for querying for a list of shards via
/v1/shards
. This endpoint will return a list of currently assigned shard keys as well as the count of nodes which has those keys assigned. Using this API endpoint, operators can see a high level listing of how their nodes are sharded.
Known Issues¶
Sensor data notifications to the message bus, such as using the
[metrics]backend
configuration option ofcollector
on a dedicated API service process or instance, is not presently supported. This functionality requires a periodic task to trigger the transmission of metrics messages to the message bus notifier.
Upgrade Notes¶
Ironic now has support for the
service
role, which is available in thesystem
scope as well as theproject
scope. This functionality is for service to service communication, if desired. Effective access rights are similar to themanager
or theowner
scoped admin privileges.
Two statsd metrics names have been modified to provide structural clarity and consistency for consumers of statistics metrics. Consumers of metrics statistics may need to update their dashboards as the
post_clean_step_hook
metric is now namedAgentBase.post_clean_step_hook
, and thepost_deploy_step_hook
is now namedAgentBase.post_deploy_step_hook
.
Deprecation Notes¶
The setting values starting with
send_sensor
in the[conductor]
configuration group have been deprecated and moved to a[sensor_data]
configuration group. The names have been updated to shorter, operator friendly names..
Bug Fixes¶
When aborting cleaning, the
last_error
field is no longer initially empty. It is now populated on the state transition toclean failed
.
When cleaning or deployment fails, the
last_error
field is no longer temporary set toNone
while the power off action is running.
Fixes an issue that when a node has console enabled but pid file missing, the console could not be disabled as well as be restarted, which makes the console feature unusable.
Fixes issues that auto-allocated console port could conflict on the same host under certain circumstances related to conductor takeover.
For more information, see story 2010489.
Fixes a database API internal check to update the
inspection_finished_at
field upon the completion of inspection.
Fixes an issue in the online upgrade logic where database models for Node Traits and BIOS Settings resulted in an error when performing the online data migration. This was because these tables were originally created as extensions of the Nodes database table, and the schema of the database was slightly different enough to result in an error if there was data to migrate in these tables upon upgrade, which would have occured if an early BIOS Setting adopter had data in the database prior to upgrading to the Yoga release of Ironic.
The online upgrade parameter now subsitutes an alternate primary key name name when applicable.
When a conductor service is stopped it will now continue to respond to RPC requests until
[DEFAULT]hash_ring_reset_interval
has elapsed, allowing a hash ring reset to complete on the cluster after conductor is unregistered. This will improve the reliability of the cluster when scaling down or rolling out updates.This delay only occurs when there is more than one online conductor, to allow fast restarts on single-node ironic installs (bifrost, metal3).
Other Notes¶
The default logging level for the
oslo_concurrencty.lockutils
module logging has been changed toWARNING
. By default, the debug logging was resulting in lots of noise. Operators wishing to view debug logging for this module can tuilize the[DEFAULT]default_log_levels
configuration option.
21.3.0¶
New Features¶
Adds API version
1.81
which enables fetching node inventory which might have been stored during introspection
Adds verify step and node vendor passthru method to deal with a firmware incompatibility issue with iRMC versions S6 2.00 and later in which HTTP connection to REST API is not supported and HTTPS connections to REST API is required.
Verify step checks connection to iRMC REST API and if connection succeeds, it fetches version of iRMC firmware and store it in
driver_internal_info/irmc_fw_version
. Ironic operators use node vendor passthru method to fetch & update iRMC firmware version cached indriver_internal_info/irmc_fw_version
.
Upgrade Notes¶
Since iRMC versions S6 2.00 and later, iRMC firmware doesn’t support HTTP connection to REST API. Operators need to set
[irmc] port
in ironic.conf ordriver_info/irmc_port
to 443.
When Ironic operator uses irmc driver against Fujitsu server which runs iRMC version S6 2.00 or later, operator may need to set Redfish parameters in
driver_info
so this fix can operate properly or operator should enable IPMI over LAN through BMC settings, if possible.
Bug Fixes¶
Fixes an issue where if selinux is enabled and enforcing, and the published image is a hardlink, the source selinux context is preserved, causing access denied when retrieving the image using hardlink URL.
Fixes the default value for the
[DEFAULT]grub_config_path
variable to be the default path for UEFI bootloader configurations, where as the default was previously the BIOS grub2 configuration path.
Fixes a firmware incompatibility issue with iRMC versions S6 2.00 and later now doesn’t support IPMI over LAN by default. To deal with this problem, irmc driver first tries IPMI operation then, if IPMI operation fails, it tries Redfish API of Fujitsu server. The operator must set Redfish parameters in the
driver_info
if iRMC disable or doesn’t support IPMI over LAN.
Fixes
'NoneType' object is not iterable
in conductor logs forredfish
andidrac-redfish
RAID clean and deploy steps. The message should no longer appear. For affected nodes re-create the node or deleteraid_configs
entry fromdriver_internal_info
field.
21.2.0¶
Upgrade Notes¶
Adds an upgrade status check for the Allocation table engine and character set encoding on MySQL. This is a result of a missing encoding definition on the table schema when originally created. This issue will be remedied, in part, in a later version of Ironic, but the upgrade status check will provide advance operator visibility.
This upgrade updates the default character set to utilized in the database tables when using MySQL/MariaDB. Previously, the default for Ironic was
UTF8
, however we now explicitly setUTF8MB3
which is short for “3 byte UTF8” encoding. The exception to this is theallocations
table, which would just rely upon the database default. This was done as Ironic’s database schema is incompatible with MySQL/MariaDB’sUTF8MB4
, or “4 byte UTF8” character encoding and storage constraints.
Upgrading will change the default chracter encoding of all tables. For most tables, this should be an effective noop, but may result in transitory table locks. For the
allocations
table, it will need to be re-written, during which the database engine will have locked the table from being used. Operators are advised to perform test upgrades and set expectation and upgrade plans accordingly.
Adds
sha256
,sha384
andsha512
as supported SNMPv3 authentication protocols to iRMC driver.
Operators who are upgrading should be aware that a bug was discovered with the automatic selection of
boot_interface
for users of theirmc
hardware types. This was an inconsistency, resulting inirmc-pxe
being selected instead ofipxe
if these boot interfaces were enabled. Depending on the local configuration, this may, or may not have happened and will remain static on preexisting baremetal nodes. Some users may have been relying upon this incorrect behavior by having mis-alligned defaults by trying to use theirmc-pxe
interface foripxe
. Users wishing to continue this usage as it was previously will need to explicitly set aboot_interface
value to eitherpxe
orirmc-pxe
, depending on the local configuration. Most operators have leveraged the default examples, and thus did not explicitly encounter this condition. Operators explicitly wishing to usepxe
boot interfaces with theipxe
templates and defaults set to override the defaults forironic.conf
will need to either continue to leverage default override configurations in theirironic.conf
file.
Ironic has started the process of upgrading the code base to support SQLAlchemy 2.0 in anticipation of it’s release. This results in the minimum version of SQLAlchemy becoming 1.4.0 as it contains migration features for the move to SQLAlchemy 2.0.
Bug Fixes¶
Fixes an missing MySQL/MariaDB character set configuration and default table type encoding for the
allocations
database table. Previously, If Ironic’s database was attempted to be populated on a machine which was using 4 byte character encoding, such as MySQL/MariaDB on Debian based systems, then the database schema creation would fail.
Fixes an issue where unexpected exceptions coming from the process to start cleaning would not trigger the cleaning_error_handler which performs the needful internal resets to permit cleaning to be retried again in the future. Now any error which is encountered during the launch of cleaning will trigger the error handler.
Fixes the URL based anaconda deployment for parsing the given
image_source
url.
Fixes URL based anaconda deploy to work in pxe boot. It also enables grub based pxe anaconda deploy which is required for
ilo
hardware type.
Modify iRMC driver to use ironic.conf [deploy] default_boot_mode to determine default boot_mode.
Fixes the default boot interface order for the
irmc
hardware type where previously it would preferirmc-pxe
overipxe
. This created inconsistencies for operators using multiple hardware types, where both interfaces were enabled in the deployment.