Here we experiment with different midPoint-Grouper integration approaches. To be used internally by the project team. Please use laboratory
branch as described below.
Downloading and building the project
This version of demo/complex
is available on the laboratory
branch. So you should use e.g. the following commands to download and build it:
$ git clone -b laboratory https://github.internet2.edu/TIER/midPoint_container.git $ cd midPoint_container $ ./build.sh
The project consists of two parts:
demo/complex
demonstrates an approach that makes midPoint responsible for all the interfacing with source and target systems, and Grouper responsible for maintaining the group membership. This is the same approach as was used indemo/complex
from the beginning.demo/complex2
is an alternative design that makes Grouper responsible for getting membership information from source systems.
Because these compositions use the same ports, only one of them can be running at the same time.
Starting the project
Please visit the appropriate child page:
Description of data processing
In this section we describe the overall processing of the data. It is common for both design options. Differences are dealt with later.
Introduction
We need to:
- fetch data from source systems (represented by a mock of a student information system),
- get them into midPoint and Grouper where it is augmented and/or modified,
- provision the data to target systems.
Let's have a look at these three areas.
Source systems
<== add staged identity onboarding (Keith)
As a demonstration of source systems let's use the following (extremely simple) three tables:
Each person has:
zero or one department membership (e.g.
Business
,Law
, and so on),zero or more affiliations (
student
,faculty
,staff
,member
,alum
,community
),zero or more course enrollments (e.g.
CS251
,MATH101
, and so on).
To summarize the data representation in SIS:
What | How | Example |
---|---|---|
person | row in | # uid, surname, givenName, fullName, department, mail 'bgasper', 'Gasper', 'Bill', 'Bill Gasper', 'Business', 'bgasper@example.edu' |
person's department |
| Business |
person's affiliation | rows in | # uid, affiliation |
person's courses | rows in | # uid, surname, givenName, courseId |
(Actually, specific SQL representation is quite irrelevant, because SIS tables serve here only as a simplified version of a real academic information system.)
midPoint and Grouper
Via midPoint and Grouper we want do achieve the following:
To modify selected information from SoR by including and/or excluding given persons to/from given groups.
This applies to affiliation information: For example we might want to state that although
bgasper
is listed underalum
he should actually not be inalum
but infaculty
.Departmental and course information do not need to be changed in this way.
To create extra groups and manually maintain their members.
To create extra groups that aggregate information from other groups.
In these scenarios we decided to use group-management features of Grouper to implement the above requirements.
Data in midPoint
In midPoint the data is represented like this:
What | How | Example |
---|---|---|
person | user | bgasper |
person's department | user → org (of subtype | bgasper → Business |
person's refined affiliation | user → org (of subtype | bgasper → Affiliation: faculty |
person's courses | user → org (of subtype | bgasper → MATH100, CS251 |
person's mailing list membership | user → org (of subtype | bgasper → Mailing list: chess, Mailing list: idm-fans |
person's other membership | user → org (of subtype | bgasper → test:volunteers, app:cs |
An example:
Relation targets (departments, affiliations, courses, mailing lists, other groups) are modeled as midPoint organizations.
TODO some screenshots here
Data in Grouper
What | How | Example |
---|---|---|
person | subject referencing LDAP entry | uid=bgasper,ou=People,dc=internet2,dc=edu |
person's department | membership in | ref:dept:Business |
person's affiliation (from SoR) | membership in | ref:affiliation:alum_systemOfRecord |
person's affiliation (refined) | membership in | ref:affiliation:faculty |
person's courses | membership in | ref:course:MATH100, ref:course:CS251 |
mailing list membership | membership in | app:mailinglist:chess, app:mailinglist:idm-fans |
computer science course enrollment | membership in | |
any other membership | membership in respective groups | test:volunteers |
An example:
Target systems
Target 1: Faculty portal
All users having affiliation offaculty
(potentially modified in Grouper) should have a record in faculty portal database, carrying the following information:uid
,givenName
,familyName
,fullName
,
This resource is temporarily created as a CSV.
Data representation:
What | How | Example |
---|---|---|
person's record | A database table row (temporary a line in CSV) | bgasper,Bill,Gasper,Bill Gasper,bgasper@example.edu (temporary in CSV) |
Target 2: Computer science students portal
All computer science students (enrolled in CSxxx courses) should have a record in this system, providing the following information:identifier
(i.e.uid
),name
(i.e.fullName
),
Data representation:
What | How | Example |
---|---|---|
person's record | A line in CSV file | dlangenberg61,Donna Langenberg,dlangenberg61@example.edu,CS251;CS252 |
Target 3: Generic mailing list application
This application expects to get the set of pairs of (listName
,
This resource is temporarily created as a CSV, represented as a set of (username,mail,list-of-mailing-lists) triples.
Data representation:
What | How | Example |
---|---|---|
mailing list membership | A line in CSV file (temporarily) | bgasper,bgasper@example.edu,chess;idm-fans |
Of course, all this information required by targets 1-3 can be taken directly from LDAP. But we want here to simulate resources that need some extra processing (e.g. Box, Office365, and so on) leading to the use of a specific connector.
Target 4: LDAP
In order to provide information to a lot of other systems we need to maintain LDAP directory where each user has an eduPerson
record with the following attributes or relations set (among others)
What | How | Example |
---|---|---|
person |
| uid=bgasper,ou=People,dc=internet2,dc=edu |
person's department |
| Business |
person's affiliation (refined by inclusion/exclusion) | group membership (in | cn=faculty,ou=Affiliations,ou=Groups,dc=internet2,dc=edu |
person's courses | group membership (in | cn=MATH100,ou=Courses,ou=Groups,dc=internet2,dc=edu cn=CS251,ou=Courses,ou=Groups,dc=internet2,dc=edu |
person's other Grouper groups | group membership (in | cn=app:cs,ou=Generic,ou=Groups,dc=internet2,dc=edu cn=app:mailinglist:chess,ou=Generic,ou=Groups,dc=internet2,dc=edu cn=app:mailinglist:idm-fans,ou=Generic,ou=Groups,dc=internet2,dc=edu cn=test:volunteers,ou=Generic,ou=Groups,dc=internet2,dc=edu |
person's midPoint-managed groups | group membership (in | cn=sysadmingroup,ou=midpoint,ou=Groups,dc=internet2,dc=edu |
An example:
Design options
The two options differ in how group membership is transferred from source systems to Grouper.
In the first one ("All interfacing via midPoint") midPoint is solely responsible for getting the data and providing it in the cleaned form to Grouper:
The idea behind this option is to concentrate all interfacing at a single place: into midPoint, which has strong features in this area.
In the second one ("SoR groups to Grouper") midPoint gets only data about persons. Data about groups are transferred from source systems directly to Grouper:
The idea is to reduce the amount of data going through midPoint: if we ultimately want to have all groups in Grouper, and we only need some of them to be provisioned to target systems, it is not necessary to pull this information through midPoint.
The discussion on these options is at the end of this document (TODO); we might note, however, that both have their own rationale, and the ultimate selection among them depends on particular circumstances.
Discussion of the options
TODO
Option 1 requires the representation of "raw" SoR data that is flowing through midPoint via LDAP to Grouper. According to the current requirements this is the case of person's affiliation that is refined in Grouper.
So in option 1 we have the following additional data items:
What | Where | How | Example |
---|---|---|---|
person's affiliation (from SoR) | midPoint | user's | alum |
LDAP |
| alum |
TODO
Note: Although Option 1 resembles thedemo/complex
on themaster
branch, it is a bit different. For example, raw affiliation (taken from SoR) is not represented as midPoint role membership but only asrawAffiliation
extension attribute. The membership in generic groups taken from Grouper is represented by midPoint org membership and LDAP group membership, not just by extension property value setting as was in originaldemo/complex
scenario.