As part of midPoint containerization for InCommon Trusted Access Platform we have developed the midPoint connector for Grouper.
Connector overview
There are two mechanisms used for midPoint ↔ Grouper communication:
- synchronous, REST-based i.e. "pull" communication where midPoint asks Grouper to provide data on a given group or groups,
- asynchronous, MQ-based i.e. "push" communication where Grouper asynchronously sends updates about changes to midPoint via RabbitMQ.
You can use both (this is preferred), or only one of these mechanisms.
Installation and use
Binaries
The connector JAR should be put into icf-connectors directory in midPoint home directory. Current version can be downloaded from https://github.internet2.edu/docker/midPoint_container/blob/master/demo/grouper/midpoint_server/container_files/mp-home/icf-connectors/connector-grouper-rest-0.6.jar. (TODO: this JAR should be published in more appropriate way) This JAR contains REST-based part of the connector.
The MQ-based part of the connector uses functionality that is built into midPoint, so no external JAR is needed. But Grouper-specific Groovy code has to be provided, as described below.
Configuration
The connector is to be used within the context of a midPoint resource object. Let us describe the sample resource object used for midPoint-Grouper integration demo. Its current version can be found in Internet2 GitHub.
Sample Grouper resource object
<resource oid="1eff65de-5bb6-483d-9edf-8cc2c2ee0233"
xmlns="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
xmlns:c="http://midpoint.evolveum.com/xml/ns/public/common/common-3"
xmlns:q="http://prism.evolveum.com/xml/ns/public/query-3"
xmlns:icfs="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/resource-schema-3"
xmlns:ri="http://midpoint.evolveum.com/xml/ns/public/resource/instance-3"
xmlns:icfc="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/connector-schema-3"
xmlns:rest="http://midpoint.evolveum.com/xml/ns/public/connector/icf-1/bundle/com.evolveum.polygon.connector-grouper-rest/com.evolveum.polygon.connector.grouper.rest.GrouperConnector"
xmlns:conf="http://midpoint.evolveum.com/xml/ns/public/connector/builtin-1/bundle/com.evolveum.midpoint.provisioning.ucf.impl.builtin.async/AsyncUpdateConnector"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<name>Grouper Resource</name>
<connectorRef type="c:ConnectorType">
<filter>
<q:equal>
<q:path>connectorType</q:path>
<q:value>com.evolveum.polygon.connector.grouper.rest.GrouperConnector</q:value>
</q:equal>
</filter>
</connectorRef>
<connectorConfiguration>
<icfc:configurationProperties>
<rest:baseUrl>https://grouper-ws:443</rest:baseUrl>
<rest:username>banderson</rest:username>
<rest:password>password</rest:password>
<rest:ignoreSslValidation>true</rest:ignoreSslValidation>
<rest:baseStem>:</rest:baseStem>
<rest:groupIncludePattern>app:.*</rest:groupIncludePattern>
<rest:groupIncludePattern>test:.*</rest:groupIncludePattern>
<rest:groupIncludePattern>ref:.*</rest:groupIncludePattern>
<rest:groupExcludePattern>.*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)</rest:groupExcludePattern>
<rest:subjectSource>ldap</rest:subjectSource>
<rest:testStem>:</rest:testStem>
<!-- no testGroup: we cannot be sure that banderson is a member of sysadmingroup when doing the first test -->
</icfc:configurationProperties>
<icfc:resultsHandlerConfiguration>
<icfc:enableNormalizingResultsHandler>false</icfc:enableNormalizingResultsHandler>
<icfc:enableFilteredResultsHandler>true</icfc:enableFilteredResultsHandler>
<icfc:enableAttributesToGetSearchResultsHandler>false</icfc:enableAttributesToGetSearchResultsHandler>
</icfc:resultsHandlerConfiguration>
</connectorConfiguration>
<additionalConnector>
<name>AMQP async update connector</name>
<connectorRef type="c:ConnectorType">
<filter>
<q:equal>
<q:path>connectorType</q:path>
<q:value>AsyncUpdateConnector</q:value>
</q:equal>
</filter>
</connectorRef>
<connectorConfiguration>
<conf:sources>
<amqp091>
<uri>amqp://mq:5672</uri>
<username>guest</username>
<password>guest</password>
<queue>sampleQueue</queue>
</amqp091>
</conf:sources>
<conf:transformExpression>
<script>
<code>
// ------------------ START OF CONFIGURATION ------------------
parameters = [
groupIncludePattern: [ 'app:.*', 'test:.*', 'ref:.*' ],
groupExcludePattern: [ '.*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)' ],
relevantSourceId: 'ldap'
]
// ------------------ END OF CONFIGURATION ------------------
parameters.put('message', message)
grouper.execute('createUcfChange', parameters)
</code>
</script>
</conf:transformExpression>
</connectorConfiguration>
</additionalConnector>
<schemaHandling>
<objectType>
<kind>entitlement</kind>
<intent>group</intent>
<objectClass>ri:Group</objectClass>
<default>true</default>
<attribute>
<ref>ri:name</ref>
<inbound>
<strength>strong</strength>
<target>
<path>extension/grouperName</path>
</target>
</inbound>
<inbound>
<strength>strong</strength>
<expression>
<script>
<code>
import com.evolveum.midpoint.schema.util.*
import com.evolveum.midpoint.schema.constants.*
if (input == null) {
null
} else {
archetypeOid = '5f2b96d2-49b5-4a8a-9601-14457309a69b' // generic-grouper-group archetype
switch (input) {
case ~/ref:affiliation:.*/: archetypeOid = '56f53812-047d-4b69-83e8-519a73d161e1'; break; // affiliation archetype
case ~/ref:dept:.*/: archetypeOid = '1cec5f78-8fba-459b-9547-ef7485009f40'; break; // department archetype
case ~/ref:course:.*/: archetypeOid = '3dab9a72-118b-4e40-a138-bb691c335eca'; break; // course archetype
case ~/app:mailinglist:.*/: archetypeOid = '1645d1dc-1f7c-4508-b50b-97b501ccdee3'; break; // mailing-list archetype
}
ObjectTypeUtil.createAssignmentTo(archetypeOid, ObjectTypes.ARCHETYPE, prismContext)
}
</code>
</script>
</expression>
<target>
<path>assignment</path>
<set>
<predefined>all</predefined> <!-- we tolerate no other assignments -->
</set>
</target>
</inbound>
</attribute>
<attribute>
<ref>ri:member</ref>
<fetchStrategy>explicit</fetchStrategy>
<storageStrategy>indexOnly</storageStrategy>
</attribute>
</objectType>
</schemaHandling>
<synchronization>
<objectSynchronization>
<enabled>true</enabled>
<kind>entitlement</kind>
<intent>group</intent>
<objectClass>ri:Group</objectClass>
<focusType>OrgType</focusType>
<correlation>
<q:equal>
<q:path>extension/grouperName</q:path>
<expression>
<path>$projection/attributes/name</path>
</expression>
</q:equal>
</correlation>
<reaction>
<situation>linked</situation>
<channel>http://midpoint.evolveum.com/xml/ns/public/provisioning/channels-3#asyncUpdate</channel>
<condition>
<script>
<code>import com.evolveum.midpoint.prism.path.ItemPath
import com.evolveum.midpoint.xml.ns._public.common.common_3.ShadowType
// member-only updates should _NOT_ be synchronized
resourceObjectDelta != null && resourceObjectDelta.isModify() &&
resourceObjectDelta.modifications.size() == 1 &&
ItemPath.create(ShadowType.F_ATTRIBUTES, 'member').equivalent(resourceObjectDelta.modifications.iterator().next().path)
</code>
</script>
</condition>
<synchronize>false</synchronize>
</reaction>
<reaction>
<situation>linked</situation>
<synchronize>true</synchronize>
</reaction>
<reaction>
<situation>deleted</situation>
<!-- a separate task will take care of deleted groups -->
<!-- we don't even need to unlink the shadow -->
<synchronize>true</synchronize>
</reaction>
<reaction>
<situation>unlinked</situation>
<action>
<handlerUri>http://midpoint.evolveum.com/xml/ns/public/model/action-3#link</handlerUri>
</action>
</reaction>
<reaction>
<situation>unmatched</situation>
<action>
<handlerUri>http://midpoint.evolveum.com/xml/ns/public/model/action-3#addFocus</handlerUri>
</action>
</reaction>
</objectSynchronization>
</synchronization>
<caching>
<cachingStategy>passive</cachingStategy>
</caching>
</resource>
This resource uses combined REST + MQ connectors.
REST connector configuration
Sample configuration for REST connector is here:
Sample configuration for Grouper REST connector
<icfc:configurationProperties>
<rest:baseUrl>https://grouper-ws:443</rest:baseUrl>
<rest:username>banderson</rest:username>
<rest:password>password</rest:password>
<rest:ignoreSslValidation>true</rest:ignoreSslValidation>
<rest:baseStem>:</rest:baseStem>
<rest:groupIncludePattern>app:.*</rest:groupIncludePattern>
<rest:groupIncludePattern>test:.*</rest:groupIncludePattern>
<rest:groupIncludePattern>ref:.*</rest:groupIncludePattern>
<rest:groupExcludePattern>.*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)</rest:groupExcludePattern>
<rest:subjectSource>ldap</rest:subjectSource>
<rest:testStem>:</rest:testStem>
<!-- no testGroup: we cannot be sure that banderson is a member of sysadmingroup when doing the first test -->
</icfc:configurationProperties>
Let us describe individual items.
| Item name | Meaning | Comment |
|---|---|---|
| baseUrl | URL on which the Grouper REST service can be accessed. | An example: https://localhost:9443. |
| username | Name of the user that is used to access the Grouper REST service. | |
| password | Password of the user that is used to access the Grouper REST service. | |
| ignoreSslValidation | Whether to ignore SSL validation issues when connecting to the Grouper REST service. | Do not use in production. |
| baseStem | The stem whose content is to be visible to this connector. | The default is ":" (the whole tree). |
| groupIncludePattern | Groups that should be visible to this connector. Specify them using regular expressions like "ref:.*". You can specify multiple values of this item. | If nothing is specified, all groups under root stem are included. |
| groupExcludePattern | Groups that should not be visible to this connector. Specify them using regular expressions like ".*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)". You can specify multiple values of this item. | |
| subjectSource | The source of subjects that will be visible by this connector. | |
| testStem | Stem whose accessibility is checked during Test connection operation (if specified). | |
| testGroup | Group whose accessibility is checked during Test connection operation (if specified). |
MQ connector configuration
Sample configuration for MQ connector is here:
Sample configuration for Grouper MQ connector
<connectorConfiguration>
<conf:sources>
<amqp091>
<uri>amqp://mq:5672</uri>
<username>guest</username>
<password>guest</password>
<queue>sampleQueue</queue>
</amqp091>
</conf:sources>
<conf:transformExpression>
<script>
<code>
// ------------------ START OF CONFIGURATION ------------------
parameters = [
groupIncludePattern: [ 'app:.*', 'test:.*', 'ref:.*' ],
groupExcludePattern: [ '.*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)' ],
relevantSourceId: 'ldap'
]
// ------------------ END OF CONFIGURATION ------------------
parameters.put('message', message)
grouper.execute('createUcfChange', parameters)
</code>
</script>
</conf:transformExpression>
</connectorConfiguration
Let us describe individual items.
| Item name | Meaning | Comment |
|---|---|---|
| conf:sources | Source(s) for asynchronous messages. These can be e.g. MQ or REST endpoints, although midPoint currently supports only AMQP 0.9.1 or custom (defined e.g. via overlay) sources. | |
| amqp091/uri | URI where AMQP 0.9.1 broker resides. | |
| amqp091/username | Name of the user that is used to access AMQP 0.9.1 broker. | |
| amqp091/password | Password of the user that is used to access AMQP 0.9.1 broker. | |
| amqp091/queue | Queue from where change notifications can be obtained. | |
| amqp091/virtualHost | AMQP virtual host. | The default value is "/". |
| amqp091/prefetch | Number of messages to prefetch. | The default is 5. |
| amqp091/connectionHandlingThreads | Number of connection handling threads. | The default is 10. |
| transformExpression/script/code:parameters | Parameters related to the processing of asynchronous messages obtained from (e.g.) AMQP queue. | |
| groupIncludePattern | Groups that should be visible to this connector. Specify them using regular expressions like "ref:.*". You can specify multiple values of this item. Should be the same as groupIncludePattern in the REST part. | If nothing is specified, all groups under root stem are included. |
| groupExcludePattern | Groups that should not be visible to this connector. Specify them using regular expressions like ".*_(includes|excludes|systemOfRecord|systemOfRecordAndIncludes)". You can specify multiple values of this item. Should be the same as groupExcludePattern in the REST part. | |
| relevantSubjectSource | The source of subjects that will be visible by this connector. Should be the same as subjectSource parameter in the REST part. |
SchemaHandling and Synchronization configuration
The schemaHandling section describes how data in Grouper are mapped to data in midPoint. In principle, this is out of scope of the Grouper connector as such. However, in order for Grouper-midPoint integration to work, something reasonable should be configured here. Let us describe the configuration we created for midPoint-Grouper integration demo, explaining probable point of extension or customization.
The principle is simple:
- Grouper groups are mapped to midPoint organizations (OrgType). This is set by synchronization/objectSynchronization/focusType = OrgType setting.
- "Raw" group name (e.g. ref:affiliation:alum) is directly stored as extension/grouperName property in the respective OrgType objects. This is set by the first inbound mapping on ri:name attribute.
- Everything else is driven by Org archetype. By default, all groups are mapped to orgs with the "generic-grouper-group" archetype (OID 5f2b96d2-49b5-4a8a-9601-14457309a69b in the demo). But you can customize this in the second inbound mapping on ri:name attribute (see lines 108-113 in the sample configuration).
The description of archetypes and related metaroles is, however, out of scope of this document.
So, if you'd like to categorize your Grouper groups and their midPoint org representations into several categories (like affiliation, department, course, mailing-list in this demo) you would need to create your own archetypes and update name → archetype mapping in the configuration. If you are OK with generic Grouper group support, you need to remove lines 108-113 (the switch statement) from the above sample configuration.
The synchronization section describes how should midPoint react to appearance of objects (or their changes) in the Grouper. The sample configuration provided above instructs midPoint to:
- Map Grouper groups to midPoint organizations (as stated above).
- Group-organization matching is based on group name == organization extension/grouperName property match.
- When new Grouper group is found (situation = unmatched), midPoint organization should be created (action = addFocus).
- When Grouper group is found that has midPoint organization created but these two are not linked together (situation = unlinked), they should simply be linked (action = link).
- When Grouper group is deleted, midPoint organization should not be deleted immediately (situation = deleted, action is not deleteFocus). Instead, the custom group scavenger task revokes group membership gracefully and then deletes the organization. Again, this is out of scope of this document.
- When Grouper group information is updated (situation = linked), midPoint organization is updated. But, as a special case, if the change information came from MQ, and if the only changed information is the group membership, the complex org update (involving e.g. REST calls to Grouper) is skipped because of performance reasons. This is implemented in lines 149-165.
Overall, the schemaHandling and synchronization sections should be left "as they are". The only customization/extension point is the mapping of group name to appropriate Org archetype.