Manage Full and Delta Read

When you set up your systems and start a scheduled provisioning task, the standard behavior of the process reads all the entities from the source system. This mode prevents data loss and always keeps your target system synchronized with the source. However, it may take a long time for every job to be executed.

Context

Delta read is a concept for optimizing the amount of data retrieved from the source system. Delta read is much faster, but sometimes might have limitations. In order for a source system to support delta read mode, its API should allow the implementation of this feature.

For example, the Microsoft Active Directory source system uses the uSNChanged attribute. For more information, see Microsoft: Polling for Changes Using USNChanged.

The main difference between delta and full read is:

Delta read – only modified data is read from the source system and triggered to the target one. Modified data means: new entities and updates on existing entities. Entities deleted from the source system will not be deleted from the target. They can be deleted only during a full read job.
Full read – all entities (new, updated, deleted, and existing unchanged ones) are read and checked every time a provisioning job is triggered to the target system.

To keep source and target systems completely synchronized, you can use the Resync type of provisioning job.

Tip:

We recommend that you enforce full reads from time to time if the connector is in delta read mode. To achieve this, you need to set up the following source system property: ips.full.read.force.count. For example, ips.full.read.force.count = 10 will result in alternating full reads after every 10 delta reads are performed.

This property only impacts scheduled runs; manually triggered runs are ignored. In case it is not set, only delta read jobs will be executed.

Remember:

When the Identity Provisioning reads entities from a source system for the first time, it always triggers a full read job. If the job is successful, the service can then continue with delta read jobs (if such are activated). During a delta read job, the service reads only the entities that are new or have been modified after the last successful job.

Below are listed all source systems that currently support delta read mode.

Supported Systems

System Type	Details
Microsoft Active Directory	Default mode: Full read You can switch to delta read, if you set up the relevant property: `ips.delta.read` = enabled Bear in mind the following specifics and limitations: Make sure that the service user, which is used in the AD destination, has a Domain Admin role, otherwise the connector won't be able to extract any data from the recycle bin. Due to the linked attributes concept of AD, there is a limitation in the Microsoft Active Directory read connector, when performing in delta read mode. We recommend that you enforce full reads periodically in order to avoid data loss. See: Microsoft: Linked Attributes You need to set limitations about which particular attributes to be read. For this purpose, set the properties `ldap.user.attributes` and `ldap.group.attributes` and add uSNChanged to the attributes list. Otherwise, the provisioning job will run in full read mode. If an entity is moved outside the base path (another directory context), the connector won't recognize this change during delta read.
SAP SuccessFactors	Default mode: Delta read You can switch to full read, if you set up the relevant property: `ips.delta.read` = disabled
SAP SuccessFactors Learning	Default mode: Delta read You can switch to full read, if you set up the relevant property: `ips.delta.read` = disabled
Identity Authentication	Default mode: Full read You can switch to delta read, if you set up the relevant property: `ips.delta.read` = enabled Note: When using SAP Central Business Configuration and Identity Directory SCIM API (in short, SCIM API version 2), delta read mode is only supported for user resources. For delta read of resources (users and groups), bear in mind the following API requirements: The system API should return lastModified, which is a subattribute of the meta attribute. The lastModified subattribute denotes the most recent date and time when the resource details were updated at the service provider. See: SCIM: Common Attributes The system API has to also support filtering by the lastModified attribute, and the system should support the gt operator in filter expressions. See: SCIM: Filtering
Local Identity Directory
SAP Central Business Configuration
SAP Data Custodian
SAP Enterprise Portal
SAP SuccessFactors Employee Central Payroll
SAP Field Service Management
SAP CPQ
SAP Advanced Financial Closing
SAP Advanced Workflow
SAP Intelligent Agriculture
SCIM System (General SCIM system, if fulfills the API requirements)