Page tree
Skip to end of metadata
Go to start of metadata

Here we experiment with different midPoint-Grouper integration approaches. To be used internally by the project team. Please use laboratory branch as described below.

Downloading and building the project

This version of demo/complex is available on the laboratory branch. So you should use e.g. the following commands to download and build it:

$ git clone -b laboratory
$ cd midPoint_container
$ ./

The project consists of several parts:

  • demo/complex demonstrates an approach that makes midPoint responsible for all the interfacing with source and target systems, and Grouper responsible for maintaining the group membership. This is the same approach as was used in demo/complex from the beginning.
  • demo/complex2 is an alternative design that makes Grouper responsible for getting membership information from source systems.
  • demo/complex2s is the same as complex2 but midPoint-Grouper interface is simplified (hence "2s").

Because these compositions use the same ports, only one of them can be running at the same time.

Starting the project

Please visit the appropriate child page:

  1. Design option 1: All interfacing via midPoint
  2. Design option 2: SoR groups to Grouper
  3. Design option 2s: SoR groups to Grouper, simplified

Description of data processing

In this section we describe the overall processing of the data. It is common for all design options. Differences are dealt with later.


We need to:

  1. fetch data from source systems (represented by a mock of a student information system),
  2. get them into midPoint and Grouper where it is augmented and/or modified,
  3. provision the data to target systems.

Let's have a look at these three areas.

Source systems

As a demonstration of source systems let's use the following (extremely simple) three tables:

Each person has:

  1. zero or one department membership (e.g. BusinessLaw, and so on),
  2. zero or more affiliations (student, facultystaffmember, alumcommunity),
  3. zero or more course enrollments (e.g. CS251MATH101, and so on).

To summarize the data representation in SIS:

personrow in SIS_PERSONS table# uid, surname, givenName, fullName, department, mail
'bgasper', 'Gasper', 'Bill', 'Bill Gasper', 'Business', ''
person's departmentdepartment columnBusiness
person's affiliationrows in SIS_AFFILIATIONS table

# uid, affiliation
'bgasper', 'alum'

person's coursesrows in SIS_COURSES table

# uid, surname, givenName, courseId
'bgasper', 'Gasper', 'Bill', 'CS251'
'bgasper', 'Gasper', 'Bill', 'MATH100'

(Actually, specific SQL representation is quite irrelevant, because SIS tables serve here only as a simplified version of a real academic information system.)

midPoint and Grouper

Via midPoint and Grouper we want do achieve the following:

  1. To modify selected information from SoR by including and/or excluding given persons to/from given groups.
    1. This applies to affiliation information: For example we might want to state that although bgasper is listed under alum he should actually not be in alum but in faculty.
    2. Departmental and course information do not need to be changed in this way.
  2. To create extra groups and manually maintain their members.
  3. To create extra groups that aggregate information from other groups.

In these scenarios we decided to use group-management features of Grouper to implement the above requirements.

Data in midPoint

In midPoint the data is represented like this:

person's departmentuser → org (of subtype department; under Departments org)bgasper → Business
person's refined affiliationuser → org (of subtype affiliation; under Affiliations org)bgasper → Affiliation: faculty
person's coursesuser → org (of subtype course; under Courses org)bgasper → MATH100, CS251
person's mailing list membershipuser → org (of subtype mailing-list; under Mailing lists org)bgasper → Mailing list: chess, Mailing list: idm-fans
person's other membershipuser → org (of subtype generic-group; under Generic groups org)bgasper → test:volunteers, app:cs

An example:


Relation targets (departments, affiliations, courses, mailing lists, other groups) are modeled as midPoint organizations.

TODO some screenshots here

Data in Grouper

personsubject referencing LDAP entryuid=bgasper,ou=People,dc=internet2,dc=edu
person's departmentmembership in ref:dept:XXX groupref:dept:Business
person's affiliation (from SoR)membership in ref:affiliation:XXX_systemOfRecord groupsref:affiliation:alum_systemOfRecord
person's affiliation (refined)membership in ref:affiliation:XXX groupsref:affiliation:faculty
person's coursesmembership in ref:course:XXX groups

ref:course:MATH100, ref:course:CS251

mailing list membershipmembership in app:mailinglist:XXX groupsapp:mailinglist:chess, app:mailinglist:idm-fans
computer science course enrollmentmembership in app:cs group
any other membershipmembership in respective groupstest:volunteers

An example:

Target systems

Target 1: Faculty portal

All users having affiliation of faculty (potentially modified in Grouper) should have a record in faculty portal database, carrying the following information: uidgivenNamefamilyNamefullName, mail.

This resource is temporarily created as a CSV.

Data representation:




person's record

A database table row (temporary a line in CSV)

bgasper,Bill,Gasper,Bill Gasper, (temporary in CSV)

Target 2: Computer science students portal

All computer science students (enrolled in CSxxx courses) should have a record in this system, providing the following information: identifier (i.e. uid), name (i.e. fullName), mail, computer science courses enrolled in.

Data representation:




person's record

A line in CSV file

dlangenberg61,Donna Langenberg,,CS251;CS252

Target 3: Generic mailing list application

This application expects to get the set of pairs of (listNamemail) describing membership of individual mailing lists.

This resource is temporarily created as a CSV, represented as a set of (username,mail,list-of-mailing-lists) triples.

Data representation:




mailing list membership

A line in CSV file (temporarily)


Of course, all this information required by targets 1-3 can be taken directly from LDAP. But we want here to simulate resources that need some extra processing (e.g. Box, Office365, and so on) leading to the use of a specific connector.

Target 4: LDAP

In order to provide information to a lot of other systems we need to maintain LDAP directory where each user has an eduPerson record with the following attributes or relations set (among others)

personeduPerson object with givenNamesncnmail containing corresponding information from sis_persons table i.e. givenNamesurnamefullNamemail, respectivelyuid=bgasper,ou=People,dc=internet2,dc=edu
person's departmentbusinessCategory attributeBusiness
person's affiliation (refined by inclusion/exclusion)group membership (in ou=Affiliations,ou=Groups,dc=internet2,dc=edu groups)cn=faculty,ou=Affiliations,ou=Groups,dc=internet2,dc=edu
person's coursesgroup membership (in ou=Courses,ou=Groups,dc=internet2,dc=edu groups)



person's other Grouper groupsgroup membership (in ou=Generic,ou=Groups,dc=internet2,dc=edu groups)





person's midPoint-managed groupsgroup membership (in ou=midpoint,ou=Groups,dc=internet2,dc=edu groups)cn=sysadmingroup,ou=midpoint,ou=Groups,dc=internet2,dc=edu

An example:

Design options

The two options differ in how group membership is transferred from source systems to Grouper.

In the first one ("All interfacing via midPoint") midPoint is solely responsible for getting the data and providing it in the cleaned form to Grouper:

The idea behind this option is to concentrate all interfacing at a single place: into midPoint, which has strong features in this area.

In the second one ("SoR groups to Grouper") midPoint gets only data about persons. Data about groups are transferred from source systems directly to Grouper:

The idea is to reduce the amount of data going through midPoint: if we ultimately want to have all groups in Grouper, and we only need some of them to be provisioned to target systems, it is not necessary to pull this information through midPoint.

The discussion on these options is at the end of this document (TODO); we might note, however, that both have their own rationale, and the ultimate selection among them depends on particular circumstances.

Discussion of the options


Option 1 requires the representation of "raw" SoR data that is flowing through midPoint via LDAP to Grouper. According to the current requirements this is the case of person's affiliation that is refined in Grouper.

So in option 1 we have the following additional data items:

person's affiliation (from SoR)midPointuser's rawAffiliation extension propertyalum
LDAPeduPersonAffiliation attributealum


Note: Although Option 1 resembles the demo/complex on the master branch, it is a bit different. For example, raw affiliation (taken from SoR) is not represented as midPoint role membership but only as rawAffiliation extension attribute. The membership in generic groups taken from Grouper is represented by midPoint org membership and LDAP group membership, not just by extension property value setting as was in original demo/complex scenario.

  • No labels