Using Continuous Audit¶
Continuous audits allow Watcher to continuously monitor and optimize your OpenStack infrastructure based on predefined schedules or intervals. This guide demonstrates how to set up and use continuous audits with the dummy strategy, which is useful for testing, development, and understanding the continuous audit workflow. However, this doc is valid for any other combination of strategy and goal.
Overview¶
A continuous audit differs from a oneshot audit in that it runs repeatedly at specified intervals. It supports both time-based intervals (in seconds) and cron-like expressions for more complex scheduling patterns.
The dummy strategy is a test strategy that doesn’t perform actual optimization but creates sample actions (nop and sleep) to demonstrate the complete audit workflow. It’s ideal for:
Testing continuous audit functionality
Development and debugging
Learning how Watcher works
Prerequisites¶
Before setting up continuous audits, ensure:
Watcher services are running and configured properly
You have administrator access to OpenStack
You can verify the services are running:
$ openstack optimize service list
+----+-------------------------+------------+--------+
| ID | Name | Host | Status |
+----+-------------------------+------------+--------+
| 1 | watcher-decision-engine | controller | ACTIVE |
| 2 | watcher-applier | controller | ACTIVE |
+----+-------------------------+------------+--------+
Continuous Audit State Machine¶
You can view the Audit state machine diagram in the Watcher documentation: Audit State Machine
Transitions:
An audit is created and enters the PENDING state.
When the scheduled time arrives, a PENDING audit becomes ONGOING.
A continuous audit remains in the ONGOING state across executions. It does not switch to SUCCEEDED after each run.
If an execution fails, the audit transitions to FAILED and is no longer executed.
Each execution produces a new action plan. When a new action plan is created by the same continuous audit, previous RECOMMENDED action plans are moved to CANCELLED. Only the latest action plan remains in RECOMMENDED.
An administrator can CANCEL an audit that is PENDING or ONGOING.
An administrator can SUSPEND an ONGOING audit.
A SUSPENDED audit can be resumed by an administrator, at which point it becomes ONGOING again.
An administrator can DELETE an audit only when its state is SUCCEEDED, FAILED, or CANCELLED.
Note
You can enable the auto-trigger option if you want to automatically apply action plans generated by continuous audits as soon as they are created. Depending on the environment, continuous audits are often good candidates for auto-trigger.
Create a Continuous Audit¶
Create a continuous audit that will run at regular intervals. You can specify the interval in seconds or use cron-like expressions.
Using Time Interval (seconds)¶
This example creates a continuous audit that runs every 5 minutes indefinitely (300 seconds):
$ openstack optimize audit create \
--goal dummy \
--strategy dummy \
--audit_type CONTINUOUS \
--interval 300 \
--name "continuous-dummy-5min"
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | 7607cf57-ea05-4e1a-b8d7-34e570f95132 |
| Name | continuous-dummy-5min |
| Created At | 2025-08-12T07:26:18.496536+00:00 |
| Updated At | None |
| Deleted At | None |
| State | PENDING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 300 |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | None |
| Hostname | None |
| Start Time | None |
| End Time | None |
| Force | False |
+---------------+--------------------------------------+
Using Cron Expression¶
For more complex scheduling, you can use cron-like expressions. This example runs the audit every hour at the 15-minute mark:
$ openstack optimize audit create \
--goal dummy \
--strategy dummy \
--audit_type CONTINUOUS \
--interval "15 * * * *" \
--name "continuous-dummy-hourly"
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | 9cbce4f1-eb75-405a-8f4e-108eb08fdd0a |
| Name | continuous-dummy-hourly |
| Created At | 2025-08-12T07:32:31.469309+00:00 |
| Updated At | None |
| Deleted At | None |
| State | PENDING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 15 * * * * |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | None |
| Hostname | None |
| Start Time | None |
| End Time | None |
| Force | False |
+---------------+--------------------------------------+
Time Constraints via start_time and end_time¶
We can limit when the continuous audit runs by setting start and end times in a time-interval schedule. The interval can passed in seconds or cron expression.
Note
Start and End Time are interpreted in the timezone configured on the host where the
Watcher Decision Engine service is running. We can provide start_time and
end_time in ISO 8601 format, for example '2025-08-13T14:30:00'.
The example below creates a continuous audit that runs from 12:00 to 13:00 with a 5 minute interval.
$ openstack optimize audit create \
--goal dummy \
--strategy dummy \
--audit_type CONTINUOUS \
--interval 300 \
--start-time "$(date -d 'today 12:00' +%Y-%m-%dT%H:%M:%S)" \
--end-time "$(date -d 'today 13:00' +%Y-%m-%dT%H:%M:%S)" \
--name "continuous-dummy-5min"
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | dadd279b-1e3d-4c38-aba6-4a730a78589b |
| Name | continuous-dummy-5min |
| Created At | 2025-08-12T08:36:42.924460+00:00 |
| Updated At | None |
| Deleted At | None |
| State | PENDING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 300 |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | None |
| Hostname | None |
| Start Time | 2025-08-12T12:00:00 |
| End Time | 2025-08-12T13:00:00 |
| Force | False |
+---------------+--------------------------------------+
Monitoring Continuous Audit Execution¶
Create a continuous audit¶
Create a continuous audit with 5 second interval:
$ openstack optimize audit create \
--goal dummy \
--strategy dummy \
--audit_type CONTINUOUS \
--interval 5 \
--name "continuous-dummy-5sec"
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd |
| Name | continuous-dummy-5sec |
| Created At | 2025-08-12T09:27:33.592575+00:00 |
| Updated At | None |
| Deleted At | None |
| State | PENDING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 5 |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | None |
| Hostname | None |
| Start Time | None |
| End Time | None |
| Force | False |
+---------------+--------------------------------------+
Once created, the continuous audit will be automatically scheduled and executed by the Watcher Decision Engine. You can monitor its progress:
Check Audit Status¶
$ openstack optimize audit show 7d1f1961-41a6-47ae-a94a-cf5e43174fbd
+---------------+--------------------------------------+
| Field | Value |
+---------------+--------------------------------------+
| UUID | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd |
| Name | continuous-dummy-5sec |
| Created At | 2025-08-12T09:27:34+00:00 |
| Updated At | 2025-08-12T09:28:28+00:00 |
| Deleted At | None |
| State | ONGOING |
| Audit Type | CONTINUOUS |
| Parameters | {'para1': 3.2, 'para2': 'hello'} |
| Interval | 5 |
| Goal | dummy |
| Strategy | dummy |
| Audit Scope | [] |
| Auto Trigger | False |
| Next Run Time | 2025-08-12T09:28:33 |
| Hostname | chkumar-devstack-1 |
| Start Time | None |
| End Time | None |
| Force | False |
+---------------+--------------------------------------+
Note
The Next Run Time is the next time the audit will run. It is calculated based on the interval and the start and end times.
List Generated Action Plans¶
Each execution of the continuous audit generates a new action plan:
$ openstack optimize actionplan list --audit 7d1f1961-41a6-47ae-a94a-cf5e43174fbd
+--------------------------------------+--------------------------------------+-------------+
| UUID | Audit | State |
+--------------------------------------+--------------------------------------+-------------+
| b301dd17-a139-4a45-ade2-b2c2ddf006ef | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd | CANCELLED |
| 22a5bc60-adef-447a-aa27-731b4f5f7ee3 | 7d1f1961-41a6-47ae-a94a-cf5e43174fbd | RECOMMENDED |
+--------------------------------------+--------------------------------------+-------------+
Note
In continuous audits, when a new action plan is generated, previous RECOMMENDED action plans are automatically set to CANCELLED state to avoid conflicts.
Manage Continuous Audits¶
Stop a Continuous Audit¶
To stop a continuous audit, update its state:
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace state=CANCELLED
Modify Audit Interval¶
You can change the interval of a running continuous audit:
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace interval=900
The Decision Engine will automatically reschedule the audit with the new interval.
Modify End Time¶
You can change the end time of a running continuous audit:
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace end_time=2025-08-12T14:00:00
Delete a Continuous Audit¶
In order to delete a continuous audit, the audit state must be SUCCEEDED, FAILED, or CANCELLED. An audit with PENDING or ONGOING state cannot be deleted.
To delete an ongoing or pending continuous audit, update its state to CANCELLED:
$ openstack optimize audit update 550e8400-e29b-41d4-a716-446655440000 replace state=CANCELLED
Then, delete the audit:
$ openstack optimize audit delete 550e8400-e29b-41d4-a716-446655440000
Configuration Reference¶
Continuous Audit Intervals¶
Numeric Intervals (seconds):
Minimum recommended: 60 seconds
Common values: 300 (5 min), 600 (10 min), 1800 (30 min), 3600 (1 hour)
Cron Expressions (5 format fields):
See the POSIX crontab specification.
0 * * * *: Every hour at minute 0*/15 * * * *: Every 15 minutes0 9-17 * * 1-5: Every hour during business hours (9 AM - 5 PM, Mon-Fri)30 2 * * *: Daily at 2:30 AM
Decision Engine Configuration¶
The continuous audit polling interval is configured in watcher.conf:
[watcher_decision_engine]
# Interval for checking continuous audits (seconds)
continuous_audit_interval = 30
