As part of midPoint containerization for InCommon Trusted Access Platform the midPoint connector for Grouper has been developed. This document describes its installation and use.
Connector overview
There are two mechanisms used for midPoint ↔ Grouper communication:
- synchronous, REST-based i.e. "pull" communication where midPoint asks Grouper to provide data on a given group or groups,
- asynchronous, MQ-based i.e. "push" communication where Grouper asynchronously sends updates about changes to midPoint via AMQP-based solution.
You can use both (this is preferred), or only one of these mechanisms in your deployment. For example, if you have relatively small deployment and your are OK with the nightly Grouper→midPoint synchronization, you can choose synchronous REST communication only.
Installation and use
Binaries
The connector JAR file should be put into icf-connectors
directory in midPoint home directory. Current version of the file can be downloaded from https://github.internet2.edu/docker/midPoint_container/blob/master/demo/grouper/midpoint_server/container_files/mp-home/icf-connectors/connector-grouper-rest-0.6.jar. (TODO: this JAR should be published in more appropriate way, maybe as part of standard midPoint Internet2 dockerization as discussed on weekly Zoom meeting.) This JAR contains REST-based part of the connector.
The MQ-based part of the connector uses functionality that is built into midPoint, so no external JAR is needed. But Grouper-specific Groovy code has to be provided, as described below.
Configuration
The connector is to be used within the context of a midPoint resource object. Let us describe the sample resource object used for midPoint-Grouper integration demo. Its current version can be found in Internet2 GitHub.
<resource oid="1eff65de-5bb6-483d-9edf-8cc2c2ee0233" xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3" xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3" xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3" xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3" xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3" xmlns:icfc="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/connector-schema-3" xmlns:rest="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/bundle/com.evolveum.polygon.connector-grouper-rest/com.evolveum.polygon.connector.grouper.rest.GrouperConnector" xmlns:conf="http://midpoint.evolveum.com/xml/ns/public/connector/builtin-1/bundle/com.evolveum.midpoint.provisioning.ucf.impl.builtin.async/AsyncUpdateConnector" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"> <name>Grouper Resource</name> <connectorRef type="c:ConnectorType"> <filter> <q:equal> <q:path>connectorType</q:path> <q:value>com.evolveum.polygon.connector.grouper.rest.GrouperConnector</q:value> </q:equal> </filter> </connectorRef> <connectorConfiguration> <icfc:configurationProperties> <rest:baseUrl>https://grouper-ws:443</rest:baseUrl> <rest:username>banderson</rest:username> <rest:password>password</rest:password> <rest:ignoreSslValidation>true</rest:ignoreSslValidation> <rest:baseStem>:</rest:baseStem> <rest:groupIncludePattern>app:.*</rest:groupIncludePattern> <rest:groupIncludePattern>test:.*</rest:groupIncludePattern> <rest:groupIncludePattern>ref:.*</rest:groupIncludePattern> <rest:groupExcludePattern>.*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)</rest:groupExcludePattern> <rest:subjectSource>ldap</rest:subjectSource> <rest:testStem>:</rest:testStem> <!-- no testGroup: we cannot be sure that banderson is a member of sysadmingroup when doing the first test --> </icfc:configurationProperties> <icfc:resultsHandlerConfiguration> <icfc:enableNormalizingResultsHandler>false</icfc:enableNormalizingResultsHandler> <icfc:enableFilteredResultsHandler>true</icfc:enableFilteredResultsHandler> <icfc:enableAttributesToGetSearchResultsHandler>false</icfc:enableAttributesToGetSearchResultsHandler> </icfc:resultsHandlerConfiguration> </connectorConfiguration> <additionalConnector> <name>AMQP async update connector</name> <connectorRef type="c:ConnectorType"> <filter> <q:equal> <q:path>connectorType</q:path> <q:value>AsyncUpdateConnector</q:value> </q:equal> </filter> </connectorRef> <connectorConfiguration> <conf:sources> <amqp091> <uri>amqp://mq:5672</uri> <username>guest</username> <password>guest</password> <queue>sampleQueue</queue> </amqp091> </conf:sources> <conf:transformExpression> <script> <code> // ------------------ START OF CONFIGURATION ------------------ parameters = [ groupIncludePattern: [ 'app:.*', 'test:.*', 'ref:.*' ], groupExcludePattern: [ '.*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)' ], relevantSourceId: 'ldap' ] // ------------------ END OF CONFIGURATION ------------------ parameters.put('message', message) grouper.execute('createUcfChange', parameters) </code> </script> </conf:transformExpression> </connectorConfiguration> </additionalConnector> <schemaHandling> <objectType> <kind>entitlement</kind> <intent>group</intent> <objectClass>ri:Group</objectClass> <default>true</default> <attribute> <ref>ri:name</ref> <inbound> <strength>strong</strength> <target> <path>extension/grouperName</path> </target> </inbound> <inbound> <strength>strong</strength> <expression> <script> <code> import com.evolveum.midpoint.schema.util.* import com.evolveum.midpoint.schema.constants.* if (input == null) { null } else { archetypeOid = '5f2b96d2-49b5-4a8a-9601-14457309a69b' // generic-grouper-group archetype switch (input) { case ~/ref:affiliation:.*/: archetypeOid = '56f53812-047d-4b69-83e8-519a73d161e1'; break; // affiliation archetype case ~/ref:dept:.*/: archetypeOid = '1cec5f78-8fba-459b-9547-ef7485009f40'; break; // department archetype case ~/ref:course:.*/: archetypeOid = '3dab9a72-118b-4e40-a138-bb691c335eca'; break; // course archetype case ~/app:mailinglist:.*/: archetypeOid = '1645d1dc-1f7c-4508-b50b-97b501ccdee3'; break; // mailing-list archetype } ObjectTypeUtil.createAssignmentTo(archetypeOid, ObjectTypes.ARCHETYPE, prismContext) } </code> </script> </expression> <target> <path>assignment</path> <set> <predefined>all</predefined> <!-- we tolerate no other assignments --> </set> </target> </inbound> </attribute> <attribute> <ref>ri:member</ref> <fetchStrategy>explicit</fetchStrategy> <storageStrategy>indexOnly</storageStrategy> </attribute> </objectType> </schemaHandling> <synchronization> <objectSynchronization> <enabled>true</enabled> <kind>entitlement</kind> <intent>group</intent> <objectClass>ri:Group</objectClass> <focusType>OrgType</focusType> <correlation> <q:equal> <q:path>extension/grouperName</q:path> <expression> <path>$projection/attributes/name</path> </expression> </q:equal> </correlation> <reaction> <situation>linked</situation> <channel>http://midpoint.evolveum.com/xml/ns/public/provisioning/channels-3#asyncUpdate</channel> <condition> <script> <code>import com.evolveum.midpoint.prism.path.ItemPath import com.evolveum.midpoint.xml.ns._public.common.common_3.ShadowType // member-only updates should _NOT_ be synchronized resourceObjectDelta != null && resourceObjectDelta.isModify() && resourceObjectDelta.modifications.size() == 1 && ItemPath.create(ShadowType.F_ATTRIBUTES, 'member').equivalent(resourceObjectDelta.modifications.iterator().next().path) </code> </script> </condition> <synchronize>false</synchronize> </reaction> <reaction> <situation>linked</situation> <synchronize>true</synchronize> </reaction> <reaction> <situation>deleted</situation> <!-- a separate task will take care of deleted groups --> <!-- we don't even need to unlink the shadow --> <synchronize>true</synchronize> </reaction> <reaction> <situation>unlinked</situation> <action> <handlerUri>http://midpoint.evolveum.com/xml/ns/public/model/action-3#link</handlerUri> </action> </reaction> <reaction> <situation>unmatched</situation> <action> <handlerUri>http://midpoint.evolveum.com/xml/ns/public/model/action-3#addFocus</handlerUri> </action> </reaction> </objectSynchronization> </synchronization> <caching> <cachingStategy>passive</cachingStategy> </caching> </resource>
This resource uses combined REST + MQ connections. The MQ part refers to grouper
Groovy function library that has to be installed (simply by importing the functional-library-grouper.xml file into midPoint repository). TODO this is to be resolved somehow. Maybe it should be part of the standard midPoint containerization as discussed on Jan 28th on the weekly Zoom meeting.
REST connector configuration
Sample configuration for REST connector is here:
<icfc:configurationProperties> <rest:baseUrl>https://grouper-ws:443</rest:baseUrl> <rest:username>banderson</rest:username> <rest:password>password</rest:password> <rest:ignoreSslValidation>true</rest:ignoreSslValidation> <rest:baseStem>:</rest:baseStem> <rest:groupIncludePattern>app:.*</rest:groupIncludePattern> <rest:groupIncludePattern>test:.*</rest:groupIncludePattern> <rest:groupIncludePattern>ref:.*</rest:groupIncludePattern> <rest:groupExcludePattern>.*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)</rest:groupExcludePattern> <rest:subjectSource>ldap</rest:subjectSource> <rest:testStem>:</rest:testStem> <!-- no testGroup: we cannot be sure that banderson is a member of sysadmingroup when doing the first test --> </icfc:configurationProperties>
Let us describe individual items.
Item name | Meaning | Comment |
---|---|---|
baseUrl | URL on which the Grouper REST service can be accessed. | An example: https://localhost:9443. |
username | Name of the user that is used to access the Grouper REST service. | |
password | Password of the user that is used to access the Grouper REST service. | |
ignoreSslValidation | Whether to ignore SSL validation issues when connecting to the Grouper REST service. | Do not use in production. |
baseStem | The stem whose content is to be visible to this connector. | The default is ":" (the whole tree). |
groupIncludePattern | Groups that should be visible to this connector. Specify them using regular expressions like "ref:.*". You can specify multiple values of this item. | If nothing is specified, all groups under root stem are included. |
groupExcludePattern | Groups that should not be visible to this connector. Specify them using regular expressions like ".*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)". You can specify multiple values of this item. | |
subjectSource | The source of subjects that will be visible by this connector. | |
testStem | Stem whose accessibility is checked during Test connection operation (if specified). | |
testGroup | Group whose accessibility is checked during Test connection operation (if specified). |
MQ connector configuration
Sample configuration for MQ connector is here:
<connectorConfiguration> <conf:sources> <amqp091> <uri>amqp://mq:5672</uri> <username>guest</username> <password>guest</password> <queue>sampleQueue</queue> </amqp091> </conf:sources> <conf:transformExpression> <script> <code> // ------------------ START OF CONFIGURATION ------------------ parameters = [ groupIncludePattern: [ 'app:.*', 'test:.*', 'ref:.*' ], groupExcludePattern: [ '.*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)' ], relevantSourceId: 'ldap' ] // ------------------ END OF CONFIGURATION ------------------ parameters.put('message', message) grouper.execute('createUcfChange', parameters) </code> </script> </conf:transformExpression> </connectorConfiguration
Let us describe individual items.
Item name | Meaning | Comment |
---|---|---|
conf:sources | Source(s) for asynchronous messages. These can be e.g. MQ or REST endpoints, although midPoint currently supports only AMQP 0.9.1 or custom (defined e.g. via overlay) sources. | |
amqp091/uri | URI where AMQP 0.9.1 broker resides. | |
amqp091/username | Name of the user that is used to access AMQP 0.9.1 broker. | |
amqp091/password | Password of the user that is used to access AMQP 0.9.1 broker. | |
amqp091/queue | Queue from where change notifications can be obtained. | |
amqp091/virtualHost | AMQP virtual host. | The default value is "/". |
amqp091/prefetch | Number of messages to prefetch. | The default is 5. |
amqp091/connectionHandlingThreads | Number of connection handling threads. (Note that if you'd need real multithreaded asynchronous messages handling, you need to specify also the number of worker threads also for Async Update task. This is to be described elsewhere.) | The default is 10. |
transformExpression/script/code:parameters | Parameters related to the processing of asynchronous messages obtained from (e.g.) AMQP queue. | |
groupIncludePattern | Groups that should be visible to this connector. Specify them using regular expressions like "ref:.*". You can specify multiple values of this item. Should be the same as groupIncludePattern in the REST part. | If nothing is specified, all groups under root stem are included. |
groupExcludePattern | Groups that should not be visible to this connector. Specify them using regular expressions like ".*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)". You can specify multiple values of this item. Should be the same as groupExcludePattern in the REST part. | |
relevantSubjectSource | The source of subjects that will be visible by this connector. Should be the same as subjectSource parameter in the REST part. |
SchemaHandling and Synchronization configuration
The schemaHandling
section describes how data in Grouper are mapped to data in midPoint. In principle, this is out of scope of the Grouper connector as such. However, in order for Grouper-midPoint integration to work, something reasonable should be configured here. Let us describe the configuration we created for midPoint-Grouper integration demo, explaining probable point of extension or customization.
The principle is simple:
- Grouper groups are mapped to midPoint organizations (OrgType). This is set by
synchronization/objectSynchronization/focusType
=OrgType
setting. - "Raw" group name (e.g.
ref:affiliation:alum
) is directly stored asextension/grouperName
property in the respective Org objects. This is set by the first inbound mapping onri:name
attribute. - Everything else is driven by appropriate Org archetype. By default, all groups are mapped to orgs with the
generic-grouper-group
archetype (OID5f2b96d2-49b5-4a8a-9601-14457309a69b
in the demo). But you can customize this in the second inbound mapping onri:name
attribute (see lines 108-113 in the sample configuration).
The description of archetypes and related metaroles is, however, out of scope of this document.
So, if you'd like to categorize your Grouper groups and their midPoint org representations into several categories (like affiliation, department, course, mailing-list in this demo) you would need to create your own archetypes and update name → archetype mapping in the configuration. If you are OK with generic Grouper group support, you need to remove lines 108-113 (the switch statement) from the above sample configuration.
The synchronization section describes how should midPoint react to appearance of objects (or their changes) in the Grouper. The sample configuration provided above instructs midPoint to:
- Map Grouper groups to midPoint organizations (as stated above).
- Group-organization matching is based on group name == organization
extension/grouperName
property match. - When new Grouper group is found (situation = unmatched), midPoint organization should be created (action = addFocus).
- When Grouper group is found that has midPoint organization created but these two are not linked together (situation = unlinked), they should simply be linked (action = link).
- When Grouper group is deleted, midPoint organization should not be deleted immediately (situation = deleted, action is not deleteFocus). Instead, the custom group scavenger task revokes group membership gracefully and then deletes the organization. Again, this is out of scope of this document.
- When Grouper group information is updated (situation = linked), midPoint organization is updated. But, as a special case, if the change information came from MQ, and if the only changed information is the group membership, the complex org update (involving e.g. REST calls to Grouper) is skipped because of performance reasons. This is implemented in lines 149-165.
Overall, the schemaHandling
and synchronization
sections should be left "as they are". The only customization/extension point is the mapping of group name to appropriate Org archetype.