Ocata Series Release Notes

11.0.4

Bug Fixes

• [bug 1704205] All users and groups are required to have a name. Prior to this fix, Keystone was not properly enforcing this for LDAP users and groups. Keystone will now ignore users and groups that do not have a value for the LDAP attribute which Keystone has been configured to use for that entity’s name.

• [bug 1718747] Fixes a regression where deleting a domain with users in it caues a server error. This bugfix restores the previous behavior of deleting the users namespaced in the domain. This only applies when using the SQL identity backend.

Other Notes

• [bug 1718747] As part of solving a regression in the identity SQL backend that prevented domains containing users from being deleted, a notification callback was altered so that users would only be deleted if the identity backend is SQL. If you have a custom identity backend that is not read-only, deleting a domain in keystone will not delete the users in your backend unless your driver has an is_sql property that evaluates to true.

11.0.3

Security Issues

• [bug 1703369] There was a typo for the identity:get_identity_provider rule in the default policy.json file in previous releases. The default value for that rule was the same as the default value for the default rule (restricted to admin) so this typo was not readily apparent. Anyone customizing this rule should review their settings and confirm that they did not copy that typo. More context regarding the purpose of this backport can be found in the bug report.

Bug Fixes

• [bug 1689616] Significant improvements have been made when performing a token flush on massive data sets.

• [bug 1687593] Ensure that the URL used to make the request when creating OAUTH1 request tokens is also the URL that verifies the request token.

11.0.1

Bug Fixes

• [bug 1674415] Fixed issue with translation of keystone error messages which was not happening in case of any error messages from identity API with locale being set.

11.0.0

Prelude

• The default token provider is now Fernet.

• The PKI and PKIz token format has been removed. See Other Notes for more details.

• Support for writing to LDAP has been removed. See Other Notes for more details.

New Features

• [blueprint allow-expired] An allow_expired flag is added to the token validation call (GET/HEAD  /v3/auth/tokens) that allows fetching a token that has expired. This allows for validating tokens in long running operations.

• [blueprint password-expires-validation] Token responses will now have a password_expires_at field in the user object, this can be expressed briefly as:

  {"token": {"user": {"password_expires_at": null}}}
  

  If PCI support is enabled, via the [security_compliance] configuration options, then the password_expires_at field will be populated with a timestamp. Otherwise, it will default to null, indicating the password does not expire.

• [blueprint pci-dss-notifications] CADF notifications now extend to PCI-DSS events. A reason object is added to the notification. A reason object has both a reasonType (a short description of the reason) and reasonCode (the HTTP return code). The following events will be impacted:

  • If a user does not change their passwords at least once every X days. See [security_compliance] password_expires_days.

  • If a user is locked out after many failed authentication attempts. See [security_compliance] lockout_failure_attempts.

  • If a user submits a new password that was recently used. See [security_compliance] unique_last_password_count.

  • If a password does not meet the specified criteria. See [security_compliance] password_regex.

  • If a user attempts to change their password too often. See [security_compliance] minimum_password_age.

  For additional details see: event notifications

• [blueprint pci-dss-password-requirements-api] Added a new API (/v3/domains/{domain_id}/config/security_compliance) to retrieve regular expression requirements for passwords. Specifically, [security_compliance] password_regex and [security_compliance] password_regex_description will be returned. Note that these options are only meaningful if PCI support is enabled, via various [security_compliance] configuration options.

• [blueprint pci-dss-query-password-expired-users] Added a password_expires_at query to /v3/users and /v3/groups/{group_id}/users. The password_expires_at query is comprised of two parts, an operator (valid choices listed below) and a timestamp (of form YYYY-MM-DDTHH:mm:ssZ). The APIs will filter the list of users based on the operator and timestamp given.

  • lt - password expires before the timestamp

  • lte - password expires at or before timestamp

  • gt - password expires after the timestamp

  • gte - password expires at or after the timestamp

  • eq - password expires at the timestamp

  • neq - password expires not at the timestamp

• [blueprint per-user-auth-plugin-reqs] Per-user Multi-Factor-Auth rules (MFA Rules) have been implemented. These rules define which auth methods can be used (e.g. Password, TOTP) and provides the ability to require multiple auth forms to successfully get a token.

  The MFA rules are set via the user create and update API (POST/PATCH /v3/users) call; the options allow an admin to force a user to use specific forms of authentication or combinations of forms of authentication to get a token. The rules are specified as follows:

  user["options"]["multi_factor_auth_rules"] = [["password", "totp"], ["password", "custom-auth-method"]]
  

  The rules are specified as a list of lists. The elements of the sub-lists must be strings and are intended to mirror the required authentication method names (e.g. password, totp, etc) as defined in the keystone.conf file in the [auth] methods option.

  Each list of methods specifies a rule. If the auth methods provided by a user match (or exceed) the auth methods in the list, that rule is used. The first rule found (rules will not be processed in a specific order) that matches will be used. If a user has the ruleset defined as [["password", "totp"]] the user must provide both password and totp auth methods (and both methods must succeed) to receive a token. However, if a user has a ruleset defined as [["password"], ["password", "totp"]] the user may use the password method on it’s own but would be required to use both password and totp if totp is specified at all.

  Any auth methods that are not defined in keystone.conf in the [auth] methods option are ignored when the rules are processed. Empty rules are not allowed. If a rule is empty due to no-valid auth methods existing within it, the rule is discarded at authentication time. If there are no rules or no valid rules for the user, authentication occurs in the default manner: any single configured auth method is sufficient to receive a token.

  In the case a user should be exempt from MFA Rules, regardless if they are set, the User-Option multi_factor_auth_enabled may be set to False for that user via the user create and update API (POST/PATCH /v3/users) call. If this option is set to False the MFA rules will be ignored for the user. Any other value except False will result in the MFA Rules being processed; the option can only be a boolean (True or False) or “None” (which will result in the default behaviour (same as True) but the option will no longer be shown in the user["options"] dictionary.

  To mark a user exempt from the MFA Rules:

  user["options"]["multi_factor_auth_enabled"] = False
  

  The token auth method typically should not be specified in any MFA Rules. The token auth method will include all previous auth methods for the original auth request and will match the appropriate ruleset. This is intentional, as the token method is used for rescoping/changing active projects.

  SECURITY INFO: The MFA rules are only processed when authentication happens through the V3 authentication APIs. If V2 Auth is enabled it is possible to circumvent the MFA rules if the user can authenticate via V2 Auth API. It is recommended to disable V2 authentication for full enforcement of the MFA rules.

• [blueprint shadow-mapping] The federated identity mapping engine now supports the ability to automatically provision projects for federated users. A role assignment will automatically be created for the user on the specified project. If the project specified within the mapping does not exist, it will be automatically created in the domain associated with the identity provider. This behavior can be triggered using a specific syntax within the local rules section of a mapping. For more information see: mapping combinations

• [blueprint support-federated-attr] Added new filters to the list user API (GET /v3/users) to support querying federated identity attributes: idp_id, protocol_id, and unique_id.

• [bug 1638603] Add support for nested groups in Active Directory. A new boolean option [ldap] group_ad_nesting has been added, it defaults to False. Enable the option is using Active Directory with nested groups. This option will impact the list_users_in_group, list_groups_for_user, and check_user_in_group operations.

• [bug 1641645] RBAC protection was removed from the Self-service change user password API (/v3/user/$user_id/password), meaning, a user can now change their password without a token specified in the X-Auth-Token header. This change will allow a user, with an expired password, to update their password without the need of an administrator.

• [bug 1641654] The healthcheck middleware from oslo.middleware has been added to the keystone application pipelines by default. This middleware provides a common method to check the health of keystone. Refer to the example paste provided in keystone-paste.ini to see how to include the healthcheck middleware.

• [bug 1641816] The [token] cache_on_issue option is now enabled by default. This option has no effect unless global caching and token caching are enabled.

• [bug 1642348] Added new option [security_compliance] lockout_ignored_user_ids to allow deployers to specify users that are exempt from PCI lockout rules.

• [Bug 1645487] Added a new PCI-DSS feature that will require users to immediately change their password upon first use for new users and after an administrative password reset. The new feature can be enabled by setting [security_compliance] change_password_upon_first_use to True.

Upgrade Notes

• [blueprint allow-expired] To allow long running operations to complete services must be able to fetch expired tokens via the allow_expired flag. The length of time a token is retrievable for beyond its traditional expiry is managed by the [token] allow_expired_window option and so the data must be retrievable for this about of time. When using fernet tokens this means that the key rotation period must exceed this time so that older tokens are still decrytable. Ensure that you do not rotate fernet keys faster than [token] expiration + [token] allow_expired_window seconds.

• [bug 1547684] A minor change to the policy.v3cloudsample.json sample file was performed so the sample file loads correctly. The cloud_admin rule has changed from:

  "role:admin and (token.is_admin_project:True or domain_id:admin_domain_id)"
  

  To the properly written:

  "role:admin and (is_admin_project:True or domain_id:admin_domain_id)"
  

  Adjust configuration tools as necessary, see the fixes section for more details on this change.

• [bug 1561054] The default token provider has switched from UUID to Fernet. Please note that Fernet requires a key repository to be in place prior to running Ocata, this can be done running keystone-manage fernet_setup. Additionally, for multi-node deployments, it is imperative that a key distribution process be in use before upgrading. Once a key repository has been created it should be distributed to all keystone nodes in the deployment. This ensures that each keystone node will be able to validate tokens issued across the deployment. If you do not wish to switch token formats, you will need to explicitly set the token provider for each node in the deployment by setting [token] provider to uuid in keystone.conf. Documentation can be found at fernet-tokens.

• [bug 1641654] The healthcheck middleware from oslo.middleware has been added to the keystone application pipelines by default. The following section has been added to keystone-paste.ini:

  [filter:healthcheck]
  use = egg:oslo.middleware#healthcheck
  

  It is recommended to have the healthcheck middleware first in the pipeline:

  pipeline = healthcheck cors sizelimit http_proxy_to_wsgi osprofiler ...
  

• [bug 1641660] The default value for [DEFAULT] notification_format has been changed from basic to cadf. The CADF notifications have more information about the user that initiated the request.

• [bug 1641660] The default value for [DEFAULT] notification_opt_out has been changed to include: identity.authenticate.success, identity.authenticate.pending and identity.authenticate.failed. If a deployment relies on these notifications, then override the default setting.

• [bug 1642687] Upon a successful upgrade, all existing identity providers will now be associated with a automatically created domain. Each identity provider that existed prior to the Ocata release will now have a domain_id field. The new domain will have an id (random UUID), a name (that will match the identity provider ID , and be enabled by default.

• [Related to Bug 1649446] The identity:list_revoke_events rule has been changed in both sample policy files, policy.json and policy.v3cloudsample.json. From:

  "identity:list_revoke_events": ""
  

  To:

  "identity:list_revoke_events": "rule:service_or_admin"
  

Deprecation Notes

• [bug 1659995] The config option [security_compliance] password_expires_ignore_user_ids has been deprecated in favor of using the option value set, available via the user create and update API call

• [blueprint deprecated-as-of-ocata] The catalog backend endpoint_filter.sql has been deprecated in the Ocata release, it has been consolidated with the sql backend. It is recommended to replace the endpoint_filter.sql catalog backend with the sql backend. The endpoint_filter.sql backend will be removed in the Pike release.

• [blueprint deprecated-as-of-ocata] Various KVS backends and config options have been deprecated and will be removed in the Pike release. This includes:

  • keystone.common.kvs.backends.inmemdb.MemoryBackend

  • keystone.common.kvs.backends.memcached.MemcachedBackend

  • keystone.token.persistence.backends.kvs.Token

  • all config options under [kvs] in keystone.conf

  • the config option [memcached] servers in keystone.conf

Critical Issues

• [bug 1561054] If upgrading to Fernet tokens, you must have a key repository and key distribution mechanism in place, otherwise token validation may not work. Please see the upgrade section for more details.

Security Issues

• [bug 1650676] Authentication plugins now required AuthContext objects to be used. This has added security features to ensure information such as the user_id does not change between authentication methods being processed by the server. The keystone.controllers.Auth.authenticate method now requires the argument auth_context to be an actual AuthContext object.

Bug Fixes

• [bug 1524030] During token validation we have reduced the number of revocation events returned, only returning a subset of events relevant to the token. Thus, improving overall token validation performance.

• [bug 1651989] Due to bug 1547684, when using the policy.v3cloudsample.json sample file, a domain admin token was being treated as a cloud admin. Since the is_admin_project functionality only supports project-scoped tokens, we automatically set any domain scoped token to have the property is_admin_project to False.

  [bug 1547684] A typo in the policy.v3cloudsample.json sample file was causing oslo.policy to not load the file. See the upgrades section for more details.

• [bug 1571878] A valid mapping_id is now required when creating or updating a federation protocol. If the mapping_id does not exist, a 400 - Bad Request will be returned.

• [bug 1616424] Provide better exception messages when creating OAuth request tokens and OAuth access tokens via the /v3/OS-OAUTH1/request_token and /v3/OS-OAUTH1/access_token APIs, respectively.

• [bug 1622310] Trusts will now be invalidated if: the project to which the trust is scoped, or the user (trustor or trustee) for which the delegation is assigned, has been deleted.

• [bug 1636950] New option [ldap] connection_timeout allows a deployer to set a OPT_NETWORK_TIMEOUT value to use with the LDAP server. This allows the LDAP server to return a SERVER_DOWN exception, if the LDAP URL is incorrect or if there is a connection failure. By default, the value for [ldap] connection_timeout is -1, meaning it is disabled. Set a positive value (in seconds) to enable the option.

• [bug 1642457] Handle disk write and IO failures when rotating keys for Fernet tokens. Rather than creating empty keys, properly catch and log errors when unable to write to disk.

• [bug 1642687] When registering an identity provider via the OS-FEDERATION API, it is now recommended to include a domain_id to associate with the identity provider in the request. Federated users that authenticate with the identity provider will now be associated with the domain_id specified. If no domain_id is specified, then a domain will be automatically created.

• [bug 1642687] Users that authenticate with an identity provider will now have a domain_id attribute, that is associated with the identity provider.

• [bug 1642692] When a federation protocol is deleted, all users that authenticated with the federation protocol will also be deleted.

• [bug 1649138] When using LDAP as an identity backend, the initial bind will now occur upon creation of a connection object, i.e. early on when performing LDAP queries, no matter whether the bind is authenticated or anonymous, so that any connection errors can be handled correctly and early.

• [Bug 1649446] The default policy for listing revocation events has changed. Previously, any authenticated user could list revocation events; it is now, by default, an admin or service user only function. This can be changed by modifying the policy file being used by Keystone.

• [bug 1656076] The various plugins under keystone.controllers.Auth.authenticate now require AuthContext objects to be returned.

• [bug 1659995] New options have been made available via the user create and update API (POST/PATCH /v3/users) call, the options will allow an admin to mark users as exempt from certain PCI requirements via an API.

  Set the following user attributes to True or False in an API request. To mark a user as exempt from the PCI password lockout policy:

  user['options']['ignore_lockout_failure_attempts']
  

  To mark a user as exempt from the PCI password expiry policy:

  user['options']['ignore_password_expiry']
  

  To mark a user as exempt from the PCI reset policy:

  user['options']['ignore_change_password_upon_first_use']
  

Other Notes

• [bug 1017606] The signature on the get_catalog and get_v3_catalog methods of keystone.catalog.backends.base.CatalogDriverBase have been updated. Third-party extensions that extend the abstract class (CatalogDriverBase) should be updated according to the new parameter names. The method signatures have changed from:

  get_catalog(self, user_id, tenant_id)
  get_v3_catalog(self, user_id, tenant_id)
  

  to:

  get_catalog(self, user_id, project_id)
  get_v3_catalog(self, user_id, project_id)
  

• [bug 1524030] The signature on the list_events method of keystone.revoke.backends.base.RevokeDriverBase has been updated. Third-party extensions that extend the abstract class (RevokeDriverBase) should update their code according to the new parameter names. The method signature has changed from:

  list_events(self, last_fetch=None)
  

  to:

  list_events(self, last_fetch=None, token=None)
  

• [bug 1563101] The token provider driver interface has moved from keystone.token.provider.Provider to keystone.token.providers.base.Provider. If implementing a custom token provider, subclass from the new location.

• [bug 1582585] A new method get_domain_mapping_list was added to keystone.identity.mapping_backends.base.MappingDriverBase. Third-party extensions that extend the abstract class (MappingDriverBase) should implement this new method. The method has the following signature:

  get_domain_mapping_list(self, domain_id)
  

  and will return a list of mappings for a given domain ID.

• [bug 1611102] The methods list_endpoints_for_policy() and get_policy_for_endpoint() have been removed from the keystone.endpoint_policy.backends.base.EndpointPolicyDriverBase abstract class, they were unused.

• [bug 1622310] A new method delete_trusts_for_project has been added to keystone.trust.backends.base.TrustDriverBase. Third-party extensions that extend the abstract class (TrustDriverBase) should be updated according to the new parameter names. The signature for the new method is:

  delete_trusts_for_project(self, project_id)
  

• [bug 1642687] The signature on the create_federated_user method of keystone.identity.shadow_backends.base.ShadowUsersDriverBase has been updated.

  Third-party extensions that extend the abstract class (ShadowUsersDriverBase) should be updated according to the new parameter names.

  The method signature has changed from:

  create_federated_user(self, federated_dict)
  

  to:

  create_federated_user(self, domain_id, federated_dict)
  

• [bug 1659730] The signature on the authenticate method of keystone.auth.plugins.base.AuthMethodHandler has been updated. Third-party extensions that extend the abstract class (AuthMethodHandler) should update their code according to the new parameter names. The method signature has changed from:

  authenticate(self, context, auth_payload, auth_context)
  

  to:

  authenticate(self, request, auth_payload, auth_context)
  

• PKI and PKIz token formats have been removed in favour of Fernet tokens.

• Write support for the LDAP has been removed in favour of read-only support. The following operations are no longer supported for LDAP:

  • create user

  • create group

  • delete user

  • delete group

  • update user

  • update group

  • add user to group

  • remove user from group

• Routes and SQL backends for the contrib extensions have been removed, they have been incorporated into Keystone and are no longer optional. This affects:

  • keystone/contrib/admin_crud

  • keystone/contrib/endpoint_filter

  • keystone/contrib/federation

  • keystone/contrib/oauth1

  • keystone/contrib/revoke

  • keystone/contrib/simple_cert

  • keystone/contrib/user_crud

• Keystone cache backends have been removed in favour of their oslo.cache counter-part. This affects:

  • keystone/common/cache/backends/mongo

  • keystone/common/cache/backends/memcache_pool

  • keystone/common/cache/backends/noop

• Several token validation methods from the abstract class keystone.token.providers.base.Provider were removed (see below) in favour of a single method to validate tokens (validate_token), that has the signature validate_token(self, token_ref). If using a custom token provider, update the custom provider accordingly.

  • validate_v2_token

  • validate_v3_token

  • validate_non_persistent_token

• Several token issuance methods from the abstract class keystone.token.providers.base.Provider were removed (see below) in favour of a single method to issue tokens (issue_token). If using a custom token provider, updated the custom provider accordingly.

  • issue_v2_token

  • issue_v3_token

• The [DEFAULT] domain_id_immutable configuration option has been removed in favour of strictly immutable domain IDs.

• The [endpoint_policy] enabled configuration option has been removed in favour of always enabling the endpoint policy extension.

• The auth plugin keystone.auth.plugins.saml2.Saml2 has been removed in favour of the auth plugin keystone.auth.plugins.mapped.Mapped.

• The memcache and memcache_pool token persistence backends have been removed in favour of using Fernet tokens (which require no persistence).

• The httpd/keystone.py file has been removed in favour of the keystone-wsgi-admin and keystone-wsgi-public scripts.

• The keystone/service.py file has been removed, the logic has been moved to the keystone/version/service.py.

• The check for admin token from build_auth_context middleware has been removed. If your deployment requires the use of admin token, update keystone-paste.ini so that admin_token_auth is before build_auth_context in the paste pipelines, otherwise remove the admin_token_auth middleware from keystone-paste.ini entirely.

• The [assignment] driver now defaults to sql. Logic to determine the default assignment driver if one wasn’t supplied through configuration has been removed. Keystone only supports one assignment driver and it shouldn’t be changed unless you’re deploying a custom assignment driver.

• The [resource] driver now defaults to sql. Logic to determine the default resource driver if one wasn’t supplied through configuration has been removed. Keystone only supports one resource driver and it shouldn’t be changed unless you’re deploying a custom resource driver.

• The [os_inherit] enabled config option has been removed, the OS-INHERIT extension is now always enabled.

• The [DEFAULT] domain_id_immutable option has been removed. This removes the ability to change the domain_id attribute of users, groups, and projects. The behaviour was introduced to allow deployers to migrate entities from one domain to another by updating the domain_id attribute of an entity. This functionality was deprecated in the Mitaka release is now removed.