About External Identity Sources

External Identity Sources allow for the creation of People and Person Roles based on data received from an external source or "system of record". These sources can include SQL databases, flat files, REST APIs, LDAP servers, and so on. Custom plugins can be written for arbitrary sources.

For more information about how data flows from External Identity Sources into Registry, see Pipelines.

Configuring External Identity Sources

  1. Before starting, create a suitable Pipeline if one doesn't already exist.
  2. Add a new External Identity Source via Configuration > External Identity Sources > Add a New External Identity Source.
    1. Set the Status as described below.
    2. Select the desired Plugin.
    3. Select the Pipeline created previously.
    4. The System of Record Label is currently only used by the API Source Plugin.
    5. Click Save.
  3. Configure the Plugin as needed.

External Identity Source Status (Sync Mode)

Each External Identity Source has a Status or Sync Mode that controls how records may be synced from the External Identity Source to the Person Registry. See Syncing External Identities, below, for additional information.

  • Full: When a Sync Job is run, all known records in the Source will be synced to the Person Registry.
  • Update: When a Sync Job is run, only Source records that already exist in the Person Registry will be updated. Records in the Source that are not already synced to the Person Registry will not be added.
  • Manual: Records may only be synced manually.
  • Disabled: Records may not be synced, either manually or via a Sync Job.

Hashing Source Records

When a record has been synced to Registry from the External Identity Source, a cached copy is stored in the External Identity Source Record so that Registry may detect when the source record has been updated. By default, this is a full copy of the record as returned to Registry from the backend (in whatever format is returned from the source, eg JSON, XML, etc). This is also useful for tracing problems, as it is possible for an administrator to look at the cached copy of the data.

However, there may be privacy or data retention concerns that make storing a full copy less desirable for a given deployment. As an alternative, Registry can create a hash of the data to be stored instead. This can be enabled via the Hash Source Records configuration option.

When this configuration is changed, existing records are not affected. Furthermore, since the cached copy will no longer match the current source record, all records from the Source will be considered out of date the next time a Sync is performed. It is best to determine the appropriate value for this setting prior to significant production usage.

An additional consideration when enabling Hash Source Records for privacy or data retention reasons is that older copies of the source records are maintained by Changelog Behavior. It is insufficient to enable this setting and perform a full sync to remove all old records from the database, manual intervention is required. The following SQL is for general guidance and should not be used directly without first testing against a test dataset:

SQL> update ext_identity_source_records set source_record=md5(source_record) where ext_identity_source_id=? and (deleted=true or ext_identity_source_record_id is not null);

Suppressing No-op Logs

By default, when Sync Job processes an External Identity Source, Job History Records are created for each Source Key. For large datasets where most records don't change, this can be noisy. By enabling Suppress No-op Logs, Job History Records will not be created for entries that have not changed.

Syncing External Identities

Pipelines

Pipelines control how data is processed when it is synced from an External Identity Source. See the Pipeline documentation for more information.

Manually

Records may be manually synced by searching the backend for the appropriate record.

  1. ALL > Configuration > External Identity Sources.
  2. Select Search Source from the "Gear" menu for the desired Source.
  3. Search for the record using a suitable value, such as the Source Key, Email Address, or Name.
  4. Select the appropriate search result.
  5. Click Sync Record to CO.

(info) Not all Plugins support direct retrieval of records, and therefore manual syncing.

Via Sync Job

Records may be synced automatically via the Sync Job Plugin.

When performing a Full Sync with a External Identity Source Plugin that supports getChangeList(), any new records returned in the changelist will be ignored in the first pass (updates) of the Full Sync, but will be processed in the second pass when the full inventory is processed. This requires the plugin to also support inventory(). If it does not, then in effect only Update Mode is supported.

Managing External Identities

Once synced, various tools are available to manage External Identities.

  • Adoption converts an External Identity (authoritative to an External Identity Source) to a native Registry Person (authoritative to Registry).
  • Deletion removes and External Identity from the Person to which it was attached.
  • Relinking moves the External Identity from one Person to another.
  • Resyncing updates the External Identity from the original External Identity Source Record.

Relinking

If an External Identity is inadvertently connected to the wrong Person, it is possible to relink it to the correct Person record (or even a new one).

  1. If the External Identity is to be moved to a new Person, create the new Person first (via People > Add a New Person).
  2. Navigate to the External Identity, via the original Person record.
  3. From the ⛭ menu, select Relink.
  4. Find the desired target Person record.

Relinking does not resync the External Identity from the External Identity Source.

Relinking moves the External Identity and its related pipelined attributes, including any Person Roles that were created by the Pipeline, to the target Person record.

An External Identity cannot be relinked if any of the attributes on the source Person that were created via the Pipeline are frozen (AR-ExternalIdentity-4).

Adoption

Adoption is the process of converting an External Identity to a Person. This may be useful when an External Identity Source is used to temporarily sync records to Registry, perhaps as part of a data migration strategy where records are transitioned to Registry as the authoritative System of Record.

To adopt a record, find the External Identity and select Adopt from the ⛭ menu. While adoption can be undone via annulment, it is not perfectly reversed, so care should be taken when performing an adoption.

Record adoption starts with an already synced External Identity. (It is not possible to adopt a record that has not yet been synced.) The Person attributes that were created by the Pipeline, including any Person Roles, are unlinked from the corresponding External Identity, meaning they switch from read-only to read/write and become Registry-authoritative attributes. (The corresponding Source Key Identifier is deleted, since it no longer has any meaning.)

The External Identity Source Record is updated to point to the Person that adopted the record, and the External Identity is deleted. Once this process completes it is no longer possible to sync the adopted Source Key from the External Identity Source.

Adopting All Records from a Source

All synced records from an External Identity Source can be adopted in one process using the Adopter Job. Before running the Adopter Job, set the External Identity Source Status to Disabled. The Adopter Job will not run for active Sources in order to prevent unintentional data transformations.

Annulling an Adoption

Annulment reverses the process of adoption, mostly. Because the adopted attributes become Registry native, it is not possible to canonically determine how to relink these attributes to the upstream source. (For example, if an administrator updates an adopted Email Address and the adoption is then annulled, which value for the Email Address should remain on the Person record?) As such, an administrator may need to manually clean Person records after an annulment.

To annul an adoption, search for the desired record in the External Identity Source. Select the Annul Adoption action from the set of available actions.

Record annulment deletes the External Identity Source Record created during adoption and then reruns the Pipeline associated with the External Identity Source. The Pipeline will link the newly (re)created External Identity to the adopted Person (the same Person the original External Identity was linked to).

(To annul and adoption but link the External Identity to a different Person, first annul the adoption and then relink the External Identity to the desired Person.)

Deletion

An External Identity can be deleted, by navigating to its ⛭ menu and selecting Delete. When deleted, the Pipelined attributes will also be deleted, and any associated Person Roles will be set to the configured status.

Note that deleting the External Identity does not prevent the External Identity Source from resyncing to the CO. If the record is resync'd, whether it attaches to the same Person or not depends on the Pipeline Match Strategy.

See Also

Changes From Earlier Versions

As of Registry v5.1.0

  • Record Adoption is available.

As of Registry v5.0.0

  • Organizational Identity Sources were renamed to External Identity Sources.
  • Pipelines are required, it is no longer possible to create an Organizational/External Identity without linking to a (CO) Person.
  • Because the underlying data model now supports External Identity Roles, External Identity Sources may convey multiple Roles within the same External Identity record. See the Pipeline documentation for more information.
  • Suppress No-op Logs is available.
  • No labels