About Pipelines

Pipelines connect Organizational Identities, typically created from Organizational Identity Sources, to CO Person Records. Pipelines can be used to automatically enroll, update, and expire CO Person records linked to external sources. Pipelines are available in COmanage Registry v1.1.0 and later.

• The External Data Source holds person related records. This is typically a SQL or LDAP database, a flat file, an API, or another similar repository.
• The Organizational identity Source is a configured Organizational Identity Source plugin, typically with a Sync Mode configured. It obtains information from the External Data Source and converts it to Organizational Identity Format.
• The Organizational Identity Source Group Mapping is a configuration, attached to the Organizational Identity Source configuration, that maps attributes from the External Data Source into candidate CO Group Memberships.
• The Organizational Identity Source Record is an artifact created when an Organizational Identity is instantiated from an Organizational Identity Source. It is a copy of the record of the External Data Source, linked to the Organizational Identity that was created from it.
• The Pipeline takes the Organizational Identity record (including any candidate CO Group Memberships), and syncs them to the operational CO Person, CO Person Role, and CO Group Membership records. As part of this process, the Pipeline may attempt to instantiate a match process to determine if a new Organizational Identity matches an existing CO Person record in some way.

The use of Organizational Identity Sources is not required in order to use a Pipeline, but other usage scenarios may not be fully implemented yet.

Configuring Pipelines

Match Strategies

Match Strategies are used to determine if an Organizational Identity should be connected to an existing CO Person. The following Match Strategies are supported:

• Email Address: The Pipeline looks for an existing CO Person record with an Email Address (of a specified type) that matches one attached to the Organizational Identity. The Email Address need not be verified, so be careful about matching on self-asserted email addresses.
• Identifier: The Pipeline looks for an existing CO Person record with an Identifier (of a specified type) that matches one attached to the Organizational Identity.
• External: Call out to an external matching service. Not currently supported (CO-1343).

Remember, Match Strategies apply against existing CO Person attributes, not Organizational Identity attributes.

If no existing CO Person is matched, then the Pipeline will create a new CO Person record.

If more than one candidate CO Person is found, an error is thrown.

Sync Strategies

Sync Strategies are used to determine when a CO Person record should be created or updated by a Pipeline, and whether an associated CO Person Role record should also be created/updated.

• Sync on Add/Update/Delete: These setting control when an Organizational Identity is processed using a Pipeline.
• Create CO Person Role Record: If checked, when the Pipeline executes it will create a CO Person Role record, not just a CO Person record. This is useful to (eg) automatically add someone to a COU based on their Organizational Identity Source.
• Sync to COU: If Create CO Person Role Record is set, this setting defines which COU the new Role Record will be placed into.
• CO Person Role Affiliation: If Create CO Person Role Record is set, this setting defines the affiliation given to the new Role Record. If not set, the affiliation of the source Org Identity will be used.
• Replace Record in COU: If the CO Person has an existing CO Person Role record in the specified COU, that Role Record will be expired. This is useful (eg) to expire a provisional record created before an authoritative record is received. Only executed on Add operations, not on Updates.
• Role Status on Delete: If Create CO Person Role Record is set and the Organizational Identity Source record is deleted (no longer valid), the corresponding CO Person Role will be set to the specified status.

When a Sync Strategy executes, it copies all data provided by the Organizational Identity Source and any defined Group Mappings.

Sync Attributes

The following attributes are supported for sync from an Org Identity:

• Address
• EmailAddress (verified status is honored)
• Identifier
• Name (primary name flags are cleared)
• TelephoneNumber

When a CO Person Role is created, the following attributes are supported for sync from the Org Identity:

• Affiliation
• O (organization)
• OU (department)
• Title
• Valid From
• Valid Through

Connecting Pipelines

Pipelines can be connected to various contexts:

1. Enrollment Flows
2. Organizational Identity Sources
3. To the CO itself (for Default Registry Enrollment)

This is also the order of preference. That is, if an Organizational Identity is created from an Enrollment Flow, and that Enrollment Flow also queries an Organizational Identity Source, and both the Enrollment Flow and Organizational Identity Source are connected to Pipelines, the Organizational Identity will be processed via the Pipeline connected to the Enrollment Flow.

Except when connected to an Enrollment Flow, when a Pipeline creates a new CO Person record, Identifier Assignment is triggered, and when a Pipeline creates or updates a CO Person or CO Person Role record, provisioning is triggered.

Pipelines are executed according to the current configuration, so it is possible for an Organizational Identity to be processed by a different Pipeline than the one it was originally attached to.

Enrollment Flows

Pipelines attached to Enrollment Flows are not currently supported (CO-1380).

Note that Pipelines can be implicitly attached to Enrollment Flows via a second mechanism, Enrollment Sources. If an Organizational Identity is created from an Enrollment Source attached to an Enrollment Flow, and if a Pipeline is attached to the Organizational Identity Source that the Enrollment Source is configured to use, then that Pipeline will be executed (not any Pipeline directly attached to the Enrollment Flow).

Organizational Identity Source Sync

Pipelines can be executed during Organizational Identity Source sync processes, resulting in add, update or deletes being processed.

Default Enrollment

Pipeline execution during Default Registry Enrollment is not currently supported (CO-1381).

Manually Rerunning a Pipeline

An Organizational Identity can be reprocessed through a Pipeline by viewing the appropriate Org Identity (People >> Organizational Identities) and clicking Rerun Pipeline. The Pipeline run will be selected in the same order of preference as defined above.