About External Identity Sources

External Identity Sources allow for the creation of People and Person Roles based on data received from an external source or "system of record". These sources can include SQL databases, flat files, REST APIs, LDAP servers, and so on. Custom plugins can be written for arbitrary sources.

For more information about how data flows from External Identity Sources into Registry, see Pipelines.

Configuring External Identity Sources

  1. Before starting, create a suitable Pipeline if one doesn't already exist.
  2. Add a new External Identity Source via Configuration > External Identity Sources > Add a New External Identity Source.
    1. Set the Status as described below.
    2. Select the desired Plugin.
    3. Select the Pipeline created previously.
    4. The System of Record Label is currently only used by the API Source Plugin.
    5. Click Save.
  3. Configure the Plugin as needed.

External Identity Source Status (Sync Mode)

Each External Identity Source has a Status or Sync Mode that controls how records may be synced from the External Identity Source to the Person Registry. See Syncing External Identities, below, for additional information.

  • Full: When a Sync Job is run, all known records in the Source will be synced to the Person Registry.
  • Update: When a Sync Job is run, only Source records that already exist in the Person Registry will be updated. Records in the Source that are not already synced to the Person Registry will not be added.
  • Manual: Records may only be synced manually.
  • Disabled: Records may not be synced, either manually or via a Sync Job.

Hashing Source Records

When a record has been synced to Registry from the External Identity Source, a cached copy is stored in the External Identity Source Record so that Registry may detect when the source record has been updated. By default, this is a full copy of the record as returned to Registry from the backend (in whatever format is returned from the source, eg JSON, XML, etc). This is also useful for tracing problems, as it is possible for an administrator to look at the cached copy of the data.

However, there may be privacy or data retention concerns that make storing a full copy less desirable for a given deployment. As an alternative, Registry can create a hash of the data to be stored instead. This can be enabled via the Hash Source Records configuration option.

When this configuration is changed, existing records are not affected. Furthermore, since the cached copy will no longer match the current source record, all records from the Source will be considered out of date the next time a Sync is performed. It is best to determine the appropriate value for this setting prior to significant production usage.

An additional consideration when enabling Hash Source Records for privacy or data retention reasons is that older copies of the source records are maintained by Changelog Behavior. It is insufficient to enable this setting and perform a full sync to remove all old records from the database, manual intervention is required. The following SQL is for general guidance and should not be used directly without first testing against a test dataset:

SQL> update ext_identity_source_records set source_record=md5(source_record) where ext_identity_source_id=? and (deleted=true or ext_identity_source_record_id is not null);

Suppressing No-op Logs

By default, when Sync Job processes an External Identity Source, Job History Records are created for each Source Key. For large datasets where most records don't change, this can be noisy. By enabling Suppress No-op Logs, Job History Records will not be created for entries that have not changed.

Syncing External Identities

Pipelines

Pipelines control how data is processed when it is synced from an External Identity Source. See the Pipeline documentation for more information.

Manually

Records may be manually synced by searching the backend for the appropriate record.

  1. ALL > Configuration > External Identity Sources.
  2. Select Search Source from the "Gear" menu for the desired Source.
  3. Search for the record using a suitable value, such as the Source Key, Email Address, or Name.
  4. Select the appropriate search result.
  5. Click Sync Record to CO.

(info) Not all Plugins support direct retrieval of records, and therefore manual syncing.

Via Sync Job

Records may be synced automatically via the Sync Job Plugin.


When performing a Full Sync with a External Identity Source Plugin that supports getChangeList(), any new records returned in the changelist will be ignored in the first pass (updates) of the Full Sync, but will be processed in the second pass when the full inventory is processed. This requires the plugin to also support inventory(). If it does not, then in effect only Update Mode is supported.

See Also

Changes From Earlier Versions

As of Registry v5.0.0

  • Organizational Identity Sources were renamed to External Identity Sources.
  • Pipelines are required, it is no longer possible to create an Organizational/External Identity without linking to a (CO) Person.
  • Because the underlying data model now supports External Identity Roles, External Identity Sources may convey multiple Roles within the same External Identity record. See the Pipeline documentation for more information.
  • Suppress No-op Logs is available.
  • No labels