Skip to main content

Data import

Connection details​

  1. In the Base URL field, enter the base URL of the service. It is used for all defined queries if these do not specify a full URL. The Base URL is persisted as part of the connection data which shouldn’t be transported between environments.

  2. In the Headers field, you can provide a JSON document where you specify custom headers. The JSON document must adhere to the following example:

    {
    "Request": {
      "Accept": " text/html",
      "content-type": "application/json",
      "SomeCustomHeaderWithValue": "custom value",
      "SomeCustomHeaderWithoutValue": null
      }
      }

  3. In the Authentication type drop-down list, choose the type of authentication to use for the OData Service system, for example, Basic, Kerberos, or OAuth2. Depending on the type of authentication that you choose to use with the system, you may see more or fewer settings appear in the dialog box.

Basic, Negotiate, Ntlm, or Digest authentication type

  1. In the User field, specify the username of the user to use for authenticating with the service.
  2. In the Password field, specify the password for the user.
  3. Optionally, in the Domain field, specify the domain name for the user.

Kerberos

  1. Optionally, in the User field, specify the username of the user to use for authenticating with the service.
  2. Optionally, in the Password field, specify the password for the user.
  3. Optionally, in the Domain field, specify the domain name for the user.

OAuth 2.0 SAML

  1. In the SAML IDP Endpoint field, enter the URL where SAML requests are posted.
  2. In the Token endpoint field, enter the URL used to exchange an authorization grant for an access token.
  3. In the Client ID field, enter the ID if the client is registered with the service. This is used to authenticate at the Token endpoint.
  4. In the Client secret field, enter the generated secret for the Client ID.
  5. In the User ID field, enter the ID if the user is registered with the service. This is used to authenticate at the Token endpoint.
  6. In the Company ID field, enter the ID if the company is registered with the service. This is used to authenticate at the Token endpoint.
  7. Optionally, in the Scope field provide the scope of the request.
  8. Optionally, specify the Resource for which to authorize for.

OAuth2 Client Credentials

  1. In the Token endpoint field, enter the URL used to exchange an authorization grant for an access token.
  2. In the Client ID field, enter the ID if the client is registered with the service. This is used to authenticate at the Token endpoint.
  3. In the Client secret field, enter the generated secret for the Client ID.
  4. Optionally, in the Scope field provide the scope of the request.
  5. Optionally, specify the Resource for which to authorize for.

OAuth2 Password

  1. In the User field, specify the username of the user to use for authenticating with the service.
  2. In the Password field, specify the password for the user.
  3. In the Token endpoint field, enter the URL used to exchange an authorization grant for an access token.
  4. In the Client ID field, enter the ID if the client registered with the service.
  5. In the Client secret field, enter the generated secret for the Client ID.
  6. Optionally, in the Scope field provide the scope of the request.
  7. Optionally, specify the Resource for which to authorize.

OAuth2 Custom

  1. In the Auth request body field, specify the request to authentication server to get access token. Sensitive values should be replaced with the placeholders starting with #SECURE.
  2. In the Placeholder values field, configure key value pairs, one per line, for sensitive data used in the request body to get the access token. Key names should start from #SECURE.
Example

#SECURE1=MySecurePass123!

  1. In the Auth request content type field, specify content type of the request sent to authentication server to get the access token.
  2. In the Authentication server response format field, choose between Access token only, JSON, Other, XML authentication server response format.
  3. Optionally, in the Access token Location field, if the Authentication server response format is set to value other than Access token only, specify the location of the access token in the response.

In addition to the authentication settings you can request the collector to Test connection to the service you configured. In such a case, fill in the Test query field. It is a query used to verify the connection. The query must be relative to the base address, for example, Users.

If you select the Include certificate option, a client certificate will be attached to the HTTP requests. Three additional parameters are required:

  • Certificate (in the PEM format)
    -----BEGIN CERTIFICATE----- <contents> -----END CERTIFICATE-----
  • Private Key (in the PEM format)
    -----BEGIN PRIVATE KEY----- .... -----END PRIVATE KEY-----
  • Passphrase for private key (optional, use it if the private key was created with the passphrase )

Queries and mappings​

For all tabs and field description, see the Queries and mappings in the REST section.

Since the release of the Omada Identity v14.0.7 and its updated version of REST-based collectors, arrays of source values cannot be mapped to a destination other than extension. An example of such an array of values might look like emails in the following JSON document:

odata-collector-migration

In the previous version of REST-based collectors, the last email address would be mapped to a destination. :::

Mapping of resource owners​

If you create a query to import resource owners, it is possible to specify the resource's owner in two ways. You can do it either by directly importing the UID of the identity or by specifying the account from which the resolved owner is imported as a resource owner.

When mapping directly to the UID of identity, ensure that identities are already imported to Omada Identity.

When mapping to an owned account, it is possible to either specify the business key of the account or the composed business key. The former should be used if the account is in the same system as the resource; the latter should be used if the account is imported into any of the trusted systems.

When the account stems from another system, you should use a Lookup mapping.

OData URL example​

An example of an OData URL: SomeObject?$select=Value1,Value2

info

You do not need to use $select-specific values as shown in the example, but it can be useful for limiting the amount of data returned by the OData service.

For retrieving linked objects, you can use the $expand query option. For example:

SomeObject?$select=Value1,level2Object/Level3Object/ValueOnLevel3Object&$expand=Level2Object/Level3Object

The queries support mapping of columns, constant values and expressions using dynamic expresso (limited C# syntax) for all Omada Identity Data Warehouse standard tables and columns. You can also add additional columns as extension attributes.

When objects are retrieved from OData, their primitive types are preserved which is important when you create expressions. For example, if you need to add an Edm.Int32 to an Edm.String, you must convert the Edm.String containing an integer in the expression.

Examples

Example 1:

(EdmInt32Value = 3 and EdmStringValue = “3”): EdmInt32Value + Int32.Parse(EdmStringValue) = 33

Example 2:

(EdmInt32Value1 = 3 and EdmInt32Value2 = 3): EdmInt32Value1 + EdmStringValue2 = 6

For building/testing of dynamic expresso, go to http://dynamic-expresso.azurewebsites.net/.

Mapping syntax​

To refer to a variable in the main response, you must use the actual name. For example, “level 1”.

To refer to a variable in a sub collector (e.g. when using “expand”), you need to provide the name of the variable including the path.

For example:

“level2Object_Level3Object_ValueOnLevel3Object”.

The “/” (forward slash) used in the OData URL is replaced with an underscore (“_”). This is to use the objects in Expressions, as C# variables can contain underscores but cannot contain forward slashes.

Advanced configuration​

  1. Enable Perform unfolding to unfold access to users given to resources which use groups to assign membership, and where a user is a member of one of these groups.

    For example, if a user is a member of Active Directory 1, there are certain rights assigned to this user.If Active Directory 1 is a member of Active Directory 2, the user can get the rights for both Active Directory 1 in addition to Active Directory 2. This inheritance of rights is not enabled by default, but if you enable the resource parent-child hierarchy, you can enable it.

    You can also provide Microsoft Office 365 licenses and access to applications in MyApps through groups.Omada recommends that you make this setting enabled to ensure that license and application access is unfolded.

  2. Optionally, in the Append URL parameters(s) field, enter any additional query parameters which should be added to any of the queries defined under Queries and Mappings.The value must be entered following the standard of the service, for example, parameter1=value&parameter2=value. Note that the collector ensures the correct formatting of the entire URL. This is useful if the paging URL returned by the service needs additional parameters.

  3. Optionally, in the Security Protocol drop-down list, choose a security protocol to use for an HTTPS connection, for example, TLS 1.2. Your organization may limit the use of one or more of the default available security protocols.

  4. Optionally, in the Content-type option, choose if the content that is returned from the system should be in atom+xml or JSON format.By default, JSON is selected.

  5. The Timeout in seconds field allows you to specify how long the collector should wait for a response of the REST service.The default value is set to 3600 seconds (1 hour).

  6. In the optional Row count per batch field, you can set the number of objects that will be collected and staged as a batch when paging is implemented in the collector.The default value is 100000.

Important

Bear in mind that applying a low value (e.g., 50 or 500) to this setting may result in extended import time.

Omada recommends that you choose JSON because this type of content takes up less space and generally should give you a better system performance compared to atom and xml.Your organization should test/verify if this content is the best solution for your organization’s purposes.

info

Even if you select application/json in the Content-type selection, some OData Services do not necessarily use this format. This is because these services return data in a specific format.So, the result may be that the collector reads that you have selected application/json, and the Content-type used the format for passing, but the service may still return the content in application/atom+xml.

Configure thresholds​

The Configure thresholds function allows you to set the amount of changes that cannot be exceeded, relevant to the last import.

In the Configure import thresholds view, type a number (integer) in percentage for New objects, Modified objects, and Deleted objects to enable thresholds for the import of objects from this system.

The value for each operation is by default set to 0, which means that no threshold calculations take place for the operations until you change the integer.

For more information, refer to the Thresholds chapter in the Omada Identity - Import and Onboarding Guide.

Important
  • If the system category is set to Identity data, the thresholds are calculated.
  • If the system category is set to Access data, the thresholds are calculated.
  • If the system category is set to Both, the thresholds only apply to Access data, that is, Accounts, Resources, and ResourceAssignments.