Grouper Real-Time Provisioning

Getting Started with Real-Time Provisioning

Real-time provisioning is the incremental provisioning of groups, stems, and memberships triggered from the Grouper change log. Incremental provisioning is distinguished from full provisioning in that only a single or subset of an attribute's values are provisioned.

Real-time provisioning is available from the provisioning service provider (psp, formerly known as ldappcng) as of version 2.1.0. It should be possible to install and configure the psp for Grouper versions 1.6 and up.

These instructions assume that Grouper subjects are already provisioned to your ldap directory, and makes use of the vt-ldap based ldap source adapter.

These instructions were written on a Mac, other platforms should be similar.

Requirements

• Grouper API (requires Java 6 and a database, details are here)
• provisioning service provider (psp)
• ldap directory

Install : Grouper Installer

To install Grouper including the API, UI, WS, grouperClient, psp, etc., download and run the Grouper Installer.

As of version 2.1.0, the psp may be installed but not configured via the Grouper Installer.

curl http://www.internet2.edu/grouper/release/2.1.0/grouperInstaller.jar -O
java -jar grouperInstaller.jar

Install : Manual

Install the provisioning service provider by copying jar files and example configuration files from the psp distribution to your Grouper API installation.

Many jars are dependencies of the Shibboleth attribute resolver and may not be necessary in your deployment.

To install manually, download and unpack the psp, then copy jars and configuration files to your Grouper API installation.

Download and Unpack the PSP

Download the PSP here and unpack.

The distribution name is of the form grouper.psp-2.1.0.tar.gz

curl http://www.internet2.edu/grouper/release/2.1.0/grouper.psp-2.1.0.tar.gz -O
tar xzf grouper.psp-2.1.0.tar.gz

Copy Jars

Copy jars located in lib/custom from the psp distribution to the Grouper API installation.

cp -vR grouper.psp-2.1.0/lib/custom/ grouper.apiBinary-2.1.0/lib/custom/

Copy Example Configuration Files

Copy example configuration files located in conf from the psp distribution to the Grouper API installation.

Example psp configuration files are in directories named with the prefix psp-example-*.

cp -vR grouper.psp-2.1.0/conf/ grouper.apiBinary-2.1.0/conf/

Configuration Example : Grouper to LDAP

This configuration example should apply to any ldap directory server.

Configuration Example : Grouper to Tivoli

This configuration example targets an IBM Tivoli Directory Server with requirements from Penn State.

Configuration Example : Grouper to OpenLDAP

This configuration example applies to OpenLDAP.

Configuration Example : Grouper to Active Directory

Configuration Example : LDAP to Grouper

Configuration : SPMLv2

Provisioned Objects

Provisioned objects, in SPMLv2 terms, consist of identifiers, attributes (probably), and references (maybe) to the identifiers of other objects, which are most likely located on the same provisioning target.

Identifiers

Identifiers consist of a string ID, a target ID, and possibly a container ID. We consider a container ID to be similar to an ldap base dn. A container ID is itself an identifier, recursing potentially indefinitely.

Attributes

Name value pairs. Probably multi-valued. Case sensitive names and values. We return values in the same order as they were given to us.

References

A reference refers to the identifier of another object. It consists of two identifiers, the "from object" and the "to object". A node in a directed graph. Directional.

Configuration : Grouper

Configure LDAP Connection in ldap.properties

Configure the default search base DN to match your directory :

edu.vt.middleware.ldap.baseDn = dc=example,dc=edu

Configure authentication and encryption :

edu.vt.middleware.ldap.serviceUser=cn=Manager,dc=example,dc=edu
edu.vt.middleware.ldap.serviceCredential=secret

Configure the default base DN (container) for people and groups :

# The base DN for groups.
edu.internet2.middleware.psp.groupsBaseDn = ou=groups,dc=example,dc=edu
# The base DN for people.
edu.internet2.middleware.psp.peopleBaseDn = ou=people,dc=example,dc=edu

Configure Grouper Stem to be Provisioned

Configure the name of the Grouper stem to be provisioned, by default this is the root stem, which is the empty string.

The Grouper stem to be provisioned is configured in ldap.properties.

# The base Grouper stem to be provisioned.
edu.internet2.middleware.psp.baseStem=

Configure LDAP Subject Source Connection

Configure Grouper to look for subjects in your LDAP directory by using the ldap source adapter.

The Grouper LDAP subject source connection is defined in sources.xml.

<source adapterClass="edu.internet2.middleware.subject.provider.LdapSourceAdapter">
    <id>ldap</id>
    <name>LdapSourceAdapter</name>
    <type>person</type>

    <init-param>
      <param-name>ldapProperties_file</param-name>
      <param-value>ldap.properties</param-value>
    </init-param>

Configure the base DN to match your directory in the various search configuration elements :

 <param-name>base</param-name>
 <param-value>ou=people,dc=example,dc=edu</param-value>

Configure Ldap Connection in the Grouper UI

Copy sources.xml and ldap.properties from the Grouper API to the Grouper UI.

cp grouper.apiBinary-2.1.0/conf/ldap.properties grouper.ui-2.1.0/dist/grouper/WEB-INF/classes/
cp grouper.apiBinary-2.1.0/conf/sources.xml grouper.ui-2.1.0/dist/grouper/WEB-INF/classes/

Copy vt-ldap.jar from the Grouper API to the Grouper UI :

cp grouper.apiBinary-2.1.0/lib/custom/vt-ldap-3.3.4.jar grouper.ui-2.1.0/dist/grouper/WEB-INF/lib

Comment out or remove the psp specific search result handlers in ldap.properties in your Grouper UI installation :

# edu.vt.middleware.ldap.searchResultHandlers=edu.internet2.middleware.psp.ldap.QuotedDnResultHandler,...

Configure LDAP Structure

The LDAP structure and corresponding group RDN source attribute ID are configurable via macro replacement in ldap.properties.

Flat

In a flat structure all groups are provisioned under a single base DN (container ID). A flat group's ldap RDN is its Grouper name or displayName.

Configure the flat LDAP structure and name group RDN source attribute ID in ldap.properties :

edu.internet2.middleware.psp.structure=flat
edu.internet2.middleware.psp.cnSourceAttributeID=name

Bushy

In a bushy structure groups are provisioned hierarchically, with stems as branches (ldap organizationalUnits) in the tree. A bushy group's RDN is its Grouper extension or displayExtension.

Configure the bushy LDAP structure and extension group RDN source attribute ID in ldap.properties :

edu.internet2.middleware.psp.structure=bushy
edu.internet2.middleware.psp.cnSourceAttributeID=extension

Configure Grouper Change Log

The Grouper change log is configured in grouper-loader.properties.

To enable change log provisioning :

changeLog.consumer.psp.class = edu.internet2.middleware.psp.grouper.PspChangeLogConsumer

To schedule when the change log is processed :

changeLog.consumer.psp.quartzCron = 0 * * * * ?

To run full synchronizations periodically (by default every day at 5am) :

changeLog.psp.fullSync.class = edu.internet2.middleware.psp.grouper.PspChangeLogConsumer
changeLog.psp.fullSync.quartzCron = 0 0 5 * * ?

To run a full synchronization job at loader startup :

changeLog.psp.fullSync.runAtStartup = true

The Quartz cron string documentation is here.

Tune Subject API Cache

The Subject API cache is configured in grouper.ehcache.xml.

Adjust maxElementsInMemory to be greater than or equal to the number of subjects.

Adjust timeToIdleSeconds and timeToLiveSeconds ... accordingly ... ?

As of Grouper version 2.1.0, which uses Ehcache 2.4, statistics must be "true" to collect statistics which are logged at DEBUG level.

<!-- Subject resolving caching -->

 <!-- @see   CachingResolver#find(...) -->
 <cache  name="edu.internet2.middleware.grouper.subj.CachingResolver.Find"
         maxElementsInMemory="5000"
         eternal="false"
         timeToIdleSeconds="30"
         timeToLiveSeconds="120"
         overflowToDisk="false"
         statistics="true"
 />

 <!-- @see   CachingResolver#findAll(...) -->
 <cache  name="edu.internet2.middleware.grouper.subj.CachingResolver.FindAll"
         maxElementsInMemory="5000"
         eternal="false"
         timeToIdleSeconds="30"
         timeToLiveSeconds="120"
         overflowToDisk="false"
         statistics="true"
 />

 <!-- @see   CachingResolver#findByIdentifier(...) -->
 <cache  name="edu.internet2.middleware.grouper.subj.CachingResolver.FindByIdentifier"
         maxElementsInMemory="5000"
         eternal="false"
         timeToIdleSeconds="30"
         timeToLiveSeconds="120"
         overflowToDisk="false"
         statistics="true"
 />

 <!-- @see   CachingResolver#findByIdOrIdentifier(...) -->
 <cache  name="edu.internet2.middleware.grouper.subj.CachingResolver.FindByIdOrIdentifier"
         maxElementsInMemory="5000"
         eternal="false"
         timeToIdleSeconds="30"
         timeToLiveSeconds="120"
         overflowToDisk="false"
         statistics="true"
 />

Configure Grouper Logging

You may want to change the Grouper log file appenders in grouper.apiBinary-2.1.0/conf/log4j.properties.

log4j.appender.grouper_error                            = org.apache.log4j.DailyRollingFileAppender
log4j.appender.grouper_error.File                       = ${grouper.home}logs/grouper_error.log
log4j.appender.grouper_error.DatePattern                = '.'yyyy-MM-dd

log4j.appender.grouper_event                            = org.apache.log4j.DailyRollingFileAppender
log4j.appender.grouper_event.File                       = ${grouper.home}logs/grouper_event.log
log4j.appender.grouper_event.DatePattern                = '.'yyyy-MM-dd

Grouper Versions Prior to 2.1.0

1. For versions prior to 2.1.0, there is a bug which will throw a NullPointerException if the following is not present in sources.xml :

<search>
    <searchType>searchSubjectByIdentifierAttributes</searchType>
    <param>
        <param-name>filter</param-name>
        <param-value>
            (&amp;(uid=%TERM%)(objectclass=person))
        </param-value>
    </param>
    <param>
        <param-name>scope</param-name>
        <param-value>SUBTREE_SCOPE</param-value>
    </param>
    <param>
        <param-name>base</param-name>
        <param-value>ou=people,dc=example,dc=edu</param-value>
    </param>
</search>

2. For versions prior to 2.1.0, the location of ldap.properties specified in sources.xml must be an absolute path. For versions 2.1.0 or later, the location of ldap.properties may be an absolute path or in your Java classpath.

For example, Grouper API version 2.0.3 requires an absolute path to ldap.properties in sources.xml :

<init-param>
    <param-name>ldapProperties_file</param-name>
    <param-value>/opt/grouper/2.0.3/grouper.apiBinary-2.0.3/conf/ldap.properties</param-value>
</init-param>

For Grouper UI versions prior to 2.1.0, the path to ldap.properties specified in sources.xml will be different than in the Grouper API since the psp specific search result handlers must be commented out or removed in the Grouper UI :

<init-param>
    <param-name>ldapProperties_file</param-name>
    <param-value>/opt/grouper/2.0.3/grouper.ui-2.0.3/dist/grouper/WEB-INF/classes/ldap.properties</param-value>
</init-param>

3. For Grouper API versions prior to 2.1.0, the ldap source adapter in subject.jar does not provide the method which allows the psp to re-use the same ldap connection as the subject source. You will need to copy lib/grouper/subject.jar from the Grouper 2.1.0 API distribution to your pre-2.1.0 Grouper API installation.

Configure : Provisioning Service Provider

The psp configuration files are :

The configuration files for Grouper ldap subjects are :

Configure PSP : Provisioned Objects, Identifiers, Attributes and References

Configure PSP : Attribute Resolver

Provision Grouper

Before you can provision anything from Grouper to ldap or anywhere else, you will need to create the corresponding objects in Grouper using the UI, API, GSH, WS, loader, import, etc.

Provision Grouper : GSH

To calculate how a group should be provisioned :

bin/gsh.sh -psp -calc edu:group

To diff the current and correct provisioning of a group :

bin/gsh.sh -psp -diff edu:group

To provision or synchronize a group :

bin/gsh.sh -psp -sync edu:group

Provision Grouper : Grouper Change Log

To provision in real-time triggered by the Grouper change log, enable the psp consumer in grouper-loader.properties and run the loader via

bin/gsh.sh -loader

Real-Time Provisioning Beta-Testing

Grouper Subject Sources

Provisioning Targets

Provisioning memberOf

The groups that a member is a member of may be provisioned to the memberOf attribute. Some LDAP implementations, such as Active Directory, automatically maintain the memberOf attribute. OpenLDAP maintains the memberOf attribute automatically via the memberOf overlay. The value of the memberOf attribute is typically a group DN.

Provisioning eduMember

The eduMember objectClass defines the isMemberOf and hasMember attributes, whose values are identifiers which are not DNs.

Provisioning eduCourse

The eduCourse objectClass defines course related attributes.

Provisioning Structure

The group provisioning structure may be either flat or bushy. A flat structure provisions all groups into a single container. A bushy structure provisions groups hierarchically.

For example, the DN of a group with name 'edu:stem:group' in a flat structure looks like : 

dn: cn=edu:stem:group,ou=groups,dc=example,dc=edu

while the DN of a group with name 'edu:stem:group' in a bushy structure looks like :

dn: cn=group,ou=stem,ou=edu,ou=groups,dc=example,dc=edu

Membership Structure

Given groupA with memberA and groupB with memberB :

dn : cn=groupA,ou=groups
member: cn=memberA,ou=people

dn: cn=groupB,ou=groups
member: cn=memberB,ou=people

If groupB is added as a member to groupA, how do you want groupA to be provisioned :

everything :

dn : cn=groupA,ou=groups
member: cn=memberA,ou=people
member: cn=memberB,ou=people
member: cn=groupB,ou=people

immediate :

dn : cn=groupA,ou=groups
member: cn=memberA,ou=people
member: cn=groupB,ou=people

The everything membership structure handles applications which may not support nested groups and represents the nested structure of the group memberships.

The same membership structure applies to memberOf :

everything :

dn: cn=memberB,ou=people
memberOf: cn=groupB,ou=groups
memberOf: cn=groupA,ou=groups

immediate :

dn: cn=memberB,ou=people
memberOf: cn=groupB,ou=groups