Thresholds for data imports and provisioning jobs
The Configure threshold tasks located on the individual system’s page are used in the system onboarding process to allow you to set threshold values for the data import process from Omada Identity Data Warehouse and for the provisioning of rights from Omada Provisioning Service.
Data import threshold
The data import threshold feature is intended to prevent unintentional or unlucky incidents in Omada Identity in case of errors with a system's imported data. Examples of such errors could be an incomplete CSV file or accidental changes in connection details.
The Data import threshold feature works by calculating the percentile rate of created, changed, or deleted objects since the last import. It suspends the import process if the rate is equal to or higher than the configured maximum value. The system owner is notified about the suspension and can decide to either resume or discard the import.
The threshold calculation applies to accounts, resources, and resource assignments.
In the daily operation, you may often have to respond to exceeded thresholds for the data import.
Evaluating exceeded thresholds for data import
When a threshold value is exceeded, the dialog box named A configured threshold was exceeded appears. System owners must evaluate if the exceeded threshold is intended or not. You can inspect the suspended data if you need to, by browsing the data in Omada Identity Data Warehouse.
The following options are available:
- Accept data and commit data on the next import - resume the import of the suspended data. If you choose to resume the import, the suspended data is moved to the staging tables and imported on the next import.
- Reject data and stage new data on the next import - reject the import of the suspended data. If you choose to reject the suspended data, the same data is extracted from the target system on the next import.
When you have made your decision, the system is no longer suspended. This means that the system is no longer excluded from imports. If the system contains identity data, none of the systems in the system category are excluded from imports any longer.
Provisioning thresholds
The OPS Thresholds feature is a feature designed to prevent that a high number of unwanted provisioning tasks from runs on Omada Identity.
This feature works as a type of an "emergency brake" where provisioning to a specific target system is suspended when the number of performed operations exceeds a defined number within a defined time interval.
It is important to note that the operations that you have started before the threshold is reached are still processed. You cannot set up the system to roll back the operations.
If you click the Configure thresholds task, the following dialog box opens:
The dialog box shows the defined threshold values and the defined interval. You can define a threshold for Create, Update, and Delete. Create includes the actions Create and CreateOrUpdate. Delete includes the actions Delete and DeleteIfExists.
In the Value, enter the number of tasks of the selected operation that is allowed before the threshold is exceeded. Set the value to 0 (zero) to disable the threshold for the selected operation.
In Interval, enter the interval in the following format hh:mm:ss. Here, hh is the number of hours, mm is the number of minutes, and ss is the number of seconds.
For example, if you set Value to 50 and Interval to 00:10:00, provisioning is suspended if 50 tasks of the selected operation happens within 10 minutes.
The Threshold value is not an exact number. Because of the multithreaded nature and the options for multiple instances, there is no way of guaranteeing that the system is stopped at the exact number when the threshold is exceeded.
Configuring threshold statistics
OPS calculates the threshold statistics in memory to ensure a minimal load on the OPS database.
The threshold statistics refresh at a configured interval to ensure that the number is as accurate as possible.
Configure the refresh interval in the omada.ops.service.exe.config configuration using the ThresholdStatisticsValidity application settings:
<add key="ThresholdStatisticsValidity" value="60"></add>
You must type in the value in minutes. Set the value to 0 (zero) to refresh statistics before you run each task.
If you run multiple OPS instances, they do not share the threshold statistics. This means that the validity should be set to a low number.
Evaluating exceeded thresholds for provisioning
When a threshold is exceeded, the system overall status shows an Error in the Status column. All provisioning is suspended, including operations that are not exceeded.
On the individual system page, the status of Enable provisioning is now Error. In the detail's column, a new link shows up: A configured threshold was exceeded.
When you click this link, you are presented with the following options:
- Resume processing till thresholds is exceeded again - when you select the first option, provisioning is resumed, and the calculation of thresholds statistics is started.
- Allow processing current pending jobs and resume processing - when you select the second option, provisioning is also resumed, but the calculation of threshold statistics is not started until all currently pending jobs have run.
If new jobs are received during processing the already pending jobs, these are included in the threshold statistics.
- Suspend threshold settings for a specified number of hours - use this option if you know that there are many tasks to complete within a given timeframe. The option allows you to suspend the checking of thresholds for a specified number of hours.
When you suspend thresholds, the system goes into a warning state. To resume threshold checking earlier, select the first option at a later stage. Open the dialog from the ellipsis menu (…) in the Provisioning jobs dialog box.
Thresholds for Relayed provisioning
If the selected connector is a relay connector and thus the Relayed provisioning is used, the provisioning thresholds are not supported. For such a connector, the Provisioning thresholds task on the system's page will not be available.