Zed Series (20.2.0 - 21.1.x) Release Notes¶
21.1.0-20¶
Upgrade Notes¶
When Ironic operator uses irmc driver against Fujitsu server which runs iRMC version S6 2.00 or later, operator may need to set Redfish parameters in
driver_info
so this fix can operate properly or operator should enable IPMI over LAN through BMC settings, if possible.
Bug Fixes¶
Fixes Ironic integration with Cinder because of changes which resulted as part of the recent Security related fix in bug 2004555. The work in Ironic to track this fix was logged in bug 2019892. Ironic now sends a service token to Cinder, which allows for access restrictions added as part of the original CVE-2023-2088 fix to be appropriately bypassed. Ironic was not vulnerable, but the restrictions added as a result did impact Ironic’s usage. This is because Ironic volume attachments are not on a shared “compute node”, but instead mapped to the physical machines and Ironic handles the attachment life-cycle after initial attachment.
When aborting cleaning, the
last_error
field is no longer initially empty. It is now populated on the state transition toclean failed
.
When cleaning or deployment fails, the
last_error
field is no longer temporary set toNone
while the power off action is running.
Fixes
Invalid cross-device link
in some cases when usingfile://
image URLs.
Fixes an issue where if selinux is enabled and enforcing, and the published image is a hardlink, the source selinux context is preserved, causing access denied when retrieving the image using hardlink URL.
Fixes a firmware incompatibility issue with iRMC versions S6 2.00 and later now doesn’t support IPMI over LAN by default. To deal with this problem, irmc driver first tries IPMI operation then, if IPMI operation fails, it tries Redfish API of Fujitsu server. The operator must set Redfish parameters in the
driver_info
if iRMC disable or doesn’t support IPMI over LAN.
Fixes
'NoneType' object is not iterable
in conductor logs forredfish
andidrac-redfish
RAID clean and deploy steps. The message should no longer appear. For affected nodes re-create the node or deleteraid_configs
entry fromdriver_internal_info
field.
Fixes an issue in the online upgrade logic where database models for Node Traits and BIOS Settings resulted in an error when performing the online data migration. This was because these tables were originally created as extensions of the Nodes database table, and the schema of the database was slightly different enough to result in an error if there was data to migrate in these tables upon upgrade, which would have occured if an early BIOS Setting adopter had data in the database prior to upgrading to the Yoga release of Ironic.
The online upgrade parameter now subsitutes an alternate primary key name name when applicable.
Fixes an issue where an agent token could be inadvertently orphaned if a node is already in the target power state when we attempt to turn the node off.
Modify iRMC driver to use ironic.conf [deploy] default_boot_mode to determine default boot_mode.
No longer re-calculates checksums for images that are already raw. Previously, it would cause significant delays in deploying raw images.
Fixes an issue where an agent token was being orphaned if a baremetal node timed out during cleaning operations, leading to issues where the node would not be able to establish a new token with Ironic upon future in some cases. We now always wipe the token in this case.
21.1.0¶
Prelude¶
The Ironic team hereby announces the release of the Zed version of Ironic. This version, 21.1.0, represents the collaboration of Ironic’s contributors during the Zed release cycle, which first saw the release of Ironic 20.2.0, and Ironic 21.1.0. These versions saw improvements in functionality to better support infrastructure operators from the configuration of individual nodes, to support a greater separation of duties, and ultimately Self-Service Bare Metal as a Service, or “SSBMaaS”. Along with these features, these releases have seen numerous bug fixes. We sincerely hope you enjoy it!
New Features¶
Adds
raritan_pdu2
,servertech_sentry3
,servertech_sentry4
, andvertivgest_pdu
snmp drivers to support additional PDU models.
Adds an automatic switch to
url
for the kickstart template when the source is a URL path as opposed to astage2
ramdisk.
Adds a concurrency limiter for number of nodes in states related to Cleaning and Provisioning operations across the ironic deployment. These settings default to a maximum number of concurrent deployments to
250
and a maximum number of concurrent deletes and cleaning operations to50
. These settings can be tuned using[conductor]max_concurrent_deploy
and[conductor]max_concurrent_clean
, respectively. The defaults should generally be good for most operators in most cases. Large scale operators should evaluate the defaults and tune appropriately as this feature cannot be disabled, as it is a security mechanism.
Adds new clean steps
create_csr
andadd_https_certificate
toilo
andilo5
hardware types which allows users to create Certificate Signing Request(CSR) and adds signed HTTPS certificate to the iLO.
The
[dhcp]dhcp_provider
configuration option can now be set todnsmasq
as an alternative tonone
for standalone deployments. This enables the same node-specific DHCP capabilities as theneutron
provider. See the[dnsmasq]
section for configuration options.
Provides vendor passthru methods for
ilo
andilo5
hardware types to create, delete and get subscriptions for BMC events. These methods are supported forHPE ProLiant Gen10
andHPE ProLiant Gen10 Plus
servers.
Adds the capability for a project scoped
admin
user to be able to create nodes in Ironic, which are then manageable by the project scopedadmin
user. Effectively, this is self service Bare Metal as a Service, however more advanced fields such as drivers, chassies, are not available to these users. This is controlled through an auto-population of the Nodeowner
field, and can be controlled through the[api]project_admin_can_manage_own_nodes
setting, which defaults toTrue
, and the new policybaremetal:node:create:self_owned_node
.
Adds the capability for a project scoped
admin
user to be able to delete nodes from Ironic which their project owns. This can be contolled through the[api]project_admin_can_manage_own_nodes
setting, which defaults toTrue
, as well as thebaremetal:node:delete:self_owned_node
policy.
Upgrade Notes¶
Large scale operators should be aware that a new feature, referred to as “Concurrent Action Limit” was introduced as a security mechanism to provide a means to limit attackers, or faulty scripts, from potentially causing irreperable harm to an environment. This feature cannot be disabled, and operators are encouraged to tune the new settings
[conductor]max_concurrent_deploy
and[conductor]max_concurrent_clean
to match the needs of their environment.
Operators who are upgrading should be aware that a bug was discovered with the automatic selection of
boot_interface
for users of theilo
andilo5
hardware types. This was an inconsistency, resulting inpxe
being selected instead ofipxe
if both boot interfaces were enabled. Depending on the local configuration, this may, or may not have happened and will remain static on preexisting baremetal nodes. Some users may have been relying upon this incorrect behavior by having misalligned defaults by trying to use thepxe
interface foripxe
. Users wishing to continue this usage as it was previously will need to explicitly set aboot_interface
value to eitherpxe
orilo-ipxe
by default, depending on the local configuration. Most operators have leveraged the default examples, and thus did not explicitly encounter this condition. Operators explicitly wishing to usepxe
boot interfaces with theipxe
templates and defaults set to override the defaults forironic.conf
will need to either continue to leverage default override configurations in theirironic.conf
file.
Security Issues¶
This release contains an improvement which, by default, allows users to create and delete baremetal nodes inside their own project. This can be disabled using the
[api]project_admin_can_manage_own_nodes
setting.
Bug Fixes¶
Fixes detecting of allowable values for a BIOS settings enumeration in the
redfish
BIOS interface when onlyValueDisplayName
is provided.
Adds a configuration option,
[anaconda]insecure_heartbeat
to allow for TLS certificate validation to be disabled in theanaconda
deployment interface, which is needed for continious integration to be able to be performed without substantial substrate image customization. This option is not advised for any production usage.
Fixes an issue where image information retrieval would fail when a path was supplied when using the
anaconda
deploy interface, as HTTPHEAD
requests on a URL path have noContent-Length
. We now consider if a path is used prior to attempting to collect additional configuration data from what is normally expected to be Glance.
Fixes an issue where the fallback to a default kickstart template value would result in error indicating “Scheme-less image href is not a UUID”. This was becaues the handling code falling back to the default did not explicitly indicate it was a file URL before saving the value.
Fixes an issue where cleaning operations could fail in such a way that was not easily recoverable when pre-cleaning network interface configuration was validated, yet contained invalid configuration. Now Ironic properly captures the error and exits from cleaning in a state which allows for cleaning to be retried.
Fixes
idrac-redfish
RAIDdelete_configuration
step to convert PERC 9 and PERC 10 controllers to RAID mode if it is not already set.
Fixes the default boot interface order for the
ilo
hardware type where previously it would preferpxe
overipxe
. This created inconsistencies for operators using multiple hardware types, where both interfaces were enabled in the deployment.
Fixes API error messages with jsonschema>=4.8. A possible root cause is now detected for generic schema errors.
Fixes an issue where the Redfish session cache would continue using an old session when a password for a Redfish BMC was changed. Now the old session will not be found in this case, and a new session will be created with the latest credential information available.
Other Notes¶
The maximum disk erasure concurrency setting,
[deploy]disk_erasure_concurrency
has been incremed to 4. Previously, this was kept at 1 in order to maintain continuity of experience, but operators have not reported any issues with an increased concurrency, and as such we feel comfortable upstream enabling concurrent disk erasure/cleaning. This setting applies to theerase_devices
clean step.
21.0.0¶
New Features¶
Adds a new feature to permit Ironic to automatically provide an instance requestor’s project,
lessee
rights to the Bare Metal machine under the Role Based Access Control model implemented in Ironic. It does this by saving the project ID of the requestor to the Nodelessee
field automatically, and removing the rights when undeploying the machine. This feature, is normally disabled, but can be enabled using the[conductor]automatic_lessee
configuration option. This option will not work in a mixed-version upgrade with older API services.
Adds a default
project
scopedmanager
role to the RBAC model. In the ironic model, access is generally explicitly delegated, and such themanager
role is presently equivelent to project scopedadmin
role, however future delineation may occur as the new features and capabilities are added.
When an allocation is being processed, the randomized candidate list is now modified so that a node with a matching name to the allocation is moved to the beginning of the list. This greatly increases the chance of node name and allocation name matching in environments where the naming schemes align.
Adds driver_info/irmc_verify_ca option to specify certification file. Default value of driver_info/irmc_verify_ca is True.
Adds SNMPv3 message authentication and encryption features to iRMC driver. To enable these features, the following parameters should be used in the node’s
driver_info
:irmc_snmp_user
irmc_snmp_auth_password
irmc_snmp_priv_password
irmc_snmp_auth_proto
(Optional, defaults tosha
)irmc_snmp_priv_proto
(Optional, defaults toaes
)
irmc_snmp_auth_proto
andirmc_snmp_priv_proto
can also be set through the following options in the[irmc]
section of/etc/ironic/ironic.conf
:snmp_auth_proto
snmp_priv_proto
Nodes using virtual media can now specify their own external URL. This setting can be leveraged via the
driver_info\external_http_url
node setting. When used, this setting overrides the[deploy]http_url
and[deploy]external_http_url
settings in the configuration file.
The
ramdisk
deploy interface is now enabled by default. When the defaultdirect
deploy is also enabled, theramdisk
deploy must be explicitly requested on the node level.
Known Issues¶
When using
jsonschema
4.0.0 or newer, make sure to include a proper$schema
field in your custom network data or RAID schemas.
Upgrade Notes¶
The deprecated support for instance network booting (not to be confused with the
ramdisk
deploy, iSCSI boot or Anaconda deploy) has been removed. Theboot_option
capability is no longer supported.
Support for trusted boot has been removed. This feature requires instance network booting, which is also removed this cycle.
Deprecation Notes¶
Support for the syslinux and the often separately package isolinux and pxelinux. Legacy BIOS boot loaders, is deprecated and will be removed the future. This is for two reasons. The first is because it is a BIOS mode bootloader, and the shift to UEFI booting has reduced the need for these packages. The second reason is a lack of maintenance, which is ultimately the same reason Linux distributions are discussing dropping support and packaging. Syslinux has not been updated since 2019. Incidently, the file format it utilizes is supported by Power hardware, and that may result in the configuration template remaining in Ironic for the forseeable future.
Support for use of pxelinux is deprecated due to the deprecation of support for Syslinux.
Support for virtual media booting with x86 Hardware in Legacy BIOS mode is deprecated due to the dependency upon isolinux for use of Legacy BIOS boot mode.
isolinux
is from thesyslinux
package and source tree.
Deprecation of
pxelinux
, as a result of the deprecation ofsyslinux
, does ultimately mean the default for thepxe
boot_interface to carry defaults for the use of grub based network booting, specifically for operators who are unable to use iPXE.
Deprecates the
irmc_snmp_security
field indriver_info
for iRMC driver, it will be removed in the future. Please useirmc_snmp_user
field instead.
Security Issues¶
Modifies the
irmc
hardware type to include a capability to control enforcement of HTTPS certificate verification. By default this is enforced. python-scciclient >= 0.12.0 is required.
Bug Fixes¶
Fixes an issue where
root_gb
became a required field when using theanaconda
deployment interface, with a recent bug fix as the code path largely expected all deployment operations to utilize images, which is not the case. The case handling for non-image based deployments is now explicitly in internal parameter validation code.
Fixes handling of
image_source
parameters where internal validations would not gracefully handle received redirects and treat it as a failure. We now no longer explicitly fail when a redirect is received.
Fixes an issue where an
image_source
could not be set to a mirror URL to facilitate deployments using a mirror with theanaconda
deployment interface. Ironic still presently has an explicit requirement on astage2
parameter to be explicitly defined.
Fixes rebooting into the agent after changing BIOS settings in fast-track mode with the
redfish-virtual-media
boot interface. Previously, the ISO would not be configured.
Fixes
OSError: [Errno 36] File name too long
when building a virtual media ISO from a long kernel, ramdisk or ESP URL.
Fixes an issue in the
anaconda
deployment interface where PXE argument processing and preparation was erroneously directly connecting to Glance, potentially leading to an exception in the standalone use case.
Fixes
redfish
andidrac-redfish
RAIDcreate_configuration
,apply_configuration
,delete_configuration
clean and deploy steps to update node’sraid_config
field at the end of the steps.
Fixes
redfish-virtual-media
boot
interface to allow it with iDRAC firmware from 6.00.00.00 (released June 2022) as it has virtual media boot issue fixed that prevented iDRAC firmware to work withredfish-virtual-media
before. Consider upgrading iDRAC firmware if not done already, otherwise will still get an error when trying to useredfish-virtual-media
with iDRAC.
Fixes compatibility with
jsonschema
package version 4.0.0 or newer by providing a proper schema version (Draft-07 currently).
Fixes a race condition in PXE initialization where the logic to retry what we suspect as potentially failed PXE boot operations was not consulting if an
agent token
had been established, which is the very first step in agent initialization.
When the
ramdisk
deploy interface is used and automated cleaning is disabled, thepxe
,ipxe
andredfish-virtual-media
boot interfaces no longer require a deploy kernel/ramdisk to be provided.
Anaconda supports the ability to explicitly pass a URL instead of a
stage2
ramdisk parameter. This has resulted in confusion in use of theanaconda
deployment interface, as astage2
ramdisk is typically not used, but made sense with Glance images in a fully integrated OpenStack deployment. Now a URL to a path can be supplied to theanaconda
deployment interface to simplify the interaction and use, and a redundantstage2
parameter is no longer required.
Resolved
clear_job_queue
andreset_idrac
verify step failures which occur when the functionality is not supported by the iDRAC. When this condition is detected, the code in the step handles the exception and logs a warning and completes successfully in case of verification steps but fails in case of cleaning steps.
Fixes an issue where an API user, when requesting a node list or single node object, could get an error indicating that the request was bad as the chassis was not found. This can occur when in-flight delete operations are in progress on another thread. Instead of surfacing a request breaking error, the API now suppresses the error and just treats it as if there is no Chassis.
Fixes
enable_netboot_fallback
to cause iPXE config to exit 0 when “sanboot –no-describe” fails. Allowing the firmware to move onto the next device in the boot order.
Other Notes¶
Adds documentation of standalone deployment use case with the
anaconda
deployment interface.
Updates the minimum version of
python-scciclient
library to0.12.1
.
Known issue when using iDRAC with Swift to stage firmware update files in Management interface
firmware_update
clean step ofredfish
oridrac
hardware type has been fixed in iDRAC firmware 6.00.00.00. Upgrade when possible or use HTTP service to stage firmware files for iDRAC.
20.2.0¶
New Features¶
When using per-node kernel parameters, i.e.
kernel_append_params
in the node’sinstance_info
ordriver_info
, it is now possible to insert the value of the driver’s configuration option (e.g.[pxe]kernel_append_params
via the special%default%
syntax, e.g.:baremetal node set node-0 \ --driver-info kernel_append_params="%default% foo=bar"
Bug Fixes¶
The combined
ironic
executable now starts the API only after the built-in conductor starts. This avoids error 500 on requests while the conductor is starting.
Fixes an issue where a conductor would attempt local takeover. In case of heartbeat failure due to resource starvation, the current conductor was detected as offline when querying the database. In this scenario the conductor would forcibly remove reservations of it’s own and initiate takeover. Current conductor is now excluded from the list of offline conductors, so that local takeover does not occur for this case. A warning is logged to highlight the potential resource starvation issue. See bug: 2010016.
Fix a bug when configuring RAID caused by not converting the port value to int type when the node managed by the irmc hardware type.