Overall architecture

The overall architecture of this demonstration composition is the following:

Generally, source systems provide data about majority of identities at the institution: students, faculty and other employees; potentially also about external persons. Those systems are typically student information system(s), human resource management system(s), and so on. They are primary sources of identity information.

Target systems, then, are primary consumers of identity data. These could be on-premises or cloud applications or services - like, for example, Office 365, G Suite (Google Apps), Slack, Jira, box.com, Coupa, Drupal, Liferay, GitLab, many kinds of ERP/CRM systems like SAP or Siebel; even student or HR information systems themselves; also individual servers, email platforms, other directories, and so on.

Source and target systems are shown in magenta color in the above schema.

Red components deal with managing user identities. These are (in this particular case) LDAP directory, midPoint, and Grouper.

  1. LDAP directory serves as the central directory of identities. It is used by both (some of) target systems and identity management components themselves.
  2. Because of its flexible interfacing features, midPoint takes care about collecting (some of) data from source systems, provisioning them into the central LDAP directory and to target systems. It also carries out complex processing of the data via mappings (i.e. transformations). Optionally, it allows designated users to augment/modify/review these data e.g. by modifying users' attributes not covered by source systems, or by assigning users specific roles, organizational membership, services or resources. If needed, it can also help with governance, risk management and compliance via features like approvals, recertification, reporting, auditing, policy rules, and so on.
  3. Grouper is - in this particular demo case - the primary tool for managing access policies and affiliations. Using its inclusion/exclusion mechanism it allows designated users to modify membership of groups defined in source systems. It also allows creation of ad-hoc groups that may or may not utilize pre-existing group membership.

Other components of the solution are:

  1. RabbitMQ is used to achieve near real-time synchronization from Grouper to midPoint. Grouper publishes changes to AMQP and midPoint updates itself based on those AMQP messages.
  2. Shibboleth Identity Provider (IdP) is used to achieve single-sign on inside and/or outside the institution. Specifically, both Grouper and midPoint users are authenticated using it.

Integration of other Internet2-provided components, e.g. COmanage, is currently out of scope of this demo.

Docker representation

Containers

The Docker containers for demo/grouper composition look like this:


Each box in the picture is one container. Name of the container is shown in parentheses in box header, e.g. sourcesmidpoint_datamidpoint_servergrouper_data, and so on.

Some components of the demo are present just to demonstrate the architecture (like containers representing source systems), others are present as suggestions for real deployments: midpoint_datamidpoint_servergrouper_datagrouper_uigrouper_daemon, mqdirectory, idp.

Let us describe individual containers now. (After composition is set up, the real container names are in the form of grouper_ABC_1 e.g. grouper_sources_1 or grouper_midpoint_server_1.)

ContainerContentDescription
sourcesMariaDB server

This serves as an example of source systems. MariaDB here hosts SIS_PERSONSSIS_AFFILIATIONS and SIS_COURSES tables that contain data about approximately 100 persons, their departmental membership, affiliations and course enrollment. The data are the same as present in TIER Grouper sample, only the form is different (here SQL, in Grouper sample LDAP+SQL).

midpoint_servermidPoint softwareThis container hosts midPoint itself: its GUI, connectors to data sources/targets, background tasks, and REST/SOAP services. It is currently 4.1-SNAPSHOT version (not the released 4.0.x because of some Internet2-specific enhancements that have been done). The container itself is quite stateless: midPoint repository is in separate container (see below) and the midPoint home directory with connector files, custom schema, and log files is in complex_midpoint_home volume.
midpoint_dataMariaDB serverThis container provides the repository for midPoint. It is separate, so it can be replaced as needed: either with a container hosting any other supported database, or with a non-dockerized repository (on-premises or in cloud).
grouper_uiGrouper softwareHere the Grouper UI executes.
grouper_wsGrouper softwareHere the Grouper WS (Web Services) executes.
grouper_daemonGrouper softwareHere the Grouper loader jobs execute.
grouper_dataMariaDB serverThis is the repository for Grouper. In a similar way as for midPoint, it is separate so that it can be replaced as needed.
directory389ds serverThis is the central LDAP directory. It is used by Grouper as a primary source of subjects and "imported" group membership (from source systems and optionally from midPoint). It is also used by Shibboleth IdP as source of authentication information.
mqRabbitMQThis provides near real-time synchronization between Grouper and midPoint and optionally target systems.
idpShibboleth IdPHere Shibboleth IdP executes, providing authentication service for Grouper and (in the future) also for midPoint.

Composition of these Docker containers is described in docker-compose.yml file.

Communication

The containers publish the following TCP ports. (Port mapped to localhost denotes the mapping of container port to the host port where it can be reached from the outside.)

ContainerPort numberPort mapped to localhostDescription
grouper_sources_1330613306Port used to connect to the sources MariaDB
grouper_midpoint_server_14438443HTTPS port used to connect to midPoint application
80-HTTP port used to connect to midPoint application
grouper_midpoint_data_1330633306Port used to connect to the default MariaDB repository
grouper_grouper_ui_14434443HTTPS port used to connect to Grouper UI
80-HTTP port used to connect to Grouper UI
grouper_grouper_ws_14439443HTTPS port used to connect to Grouper Web Service
80-HTTP port used to connect to Grouper Web Service
grouper_grouper_daemon_1443-HTTPS port used to connect to Grouper daemon (if needed)
80-HTTP port used to connect to Grouper daemon (if needed)
grouper_data_133063306Port used to connect to the Grouper MariaDB repository
grouper_directory_1389389LDAP connection to the directory
grouper_mq1567215672Port used to access RabbitMQ
grouper_idp443443HTTPS port to be used to connect to the Shibboleth IdP

(Some other, less important ports that are internally exposed are not mentioned here.)

Docker volumes

The following volumes are created to persist data and other relevant files.

Volume nameDescriptionUsed by container
grouper_source_dataVolume hosting MariaDB database used by the sources containergrouper_sources_1
grouper_source_mysqlVolume hosting /var/lib/mysql directory for the sourcesgrouper_sources_1
grouper_midpoint_homeThe midPoint home directory. Contains schema extensions, logs, custom libraries, custom ConnId connectors, and so on.grouper_midpoint_server_1
grouper_midpoint_dataVolume hosting MariaDB database used by midPointgrouper_midpoint_data_1
grouper_midpoint_mysqlVolume hosting /var/lib/mysql directory
grouper_grouper_dataVolume hosting MariaDB database used by Groupergrouper_grouper_data
grouper_ldapVolume hosting the LDAP databasegrouper_directory
grouper_mqVolume hosting the messaging i.e. the /var/lib/rabbitmq directorygrouper_mq

Configuring the composition

The following configuration properties are supported. Please refer to the main documentation page for their explanation.

PropertyDefault value
ENVdemo
USERTOKEN
REPO_DATABASE_TYPEmariadb
REPO_JDBC_URLdefault
REPO_HOSTmidpoint_data
REPO_PORTdefault
REPO_DATABASEregistry
REPO_USERregistry_user
REPO_MISSING_SCHEMA_ACTIONcreate
REPO_UPGRADEABLE_SCHEMA_ACTIONstop
REPO_SCHEMA_VERSION_IF_MISSING
REPO_SCHEMA_VARIANT
MP_MEM_MAX2048m
MP_MEM_INIT1024m
MP_JAVA_OPTS
SSO_HEADERuid
TIER_BEACON_OPT_OUT
TIMEZONEUTC

You can tailor these to your needs.

The following Docker secrets are used:

SecretLocation
mp_database_password.txtconfigs-and-secrets/midpoint/application/database_password.txt
mp_keystore_password.txtconfigs-and-secrets/midpoint/application/keystore_password.txt
mp_host-key.pemconfigs-and-secrets/midpoint/httpd/host-key.pem
mp_sp-key.pemconfigs-and-secrets/midpoint/shibboleth/sp-key.pem

The following configuration files are used:

Target fileSource location
/etc/pki/tls/certs/host-cert.pemconfigs-and-secrets/midpoint/httpd/host-cert.pem
/etc/pki/tls/certs/cachain.pemconfigs-and-secrets/midpoint/httpd/host-cert.pem
/etc/shibboleth/idp-metadata.xmlconfigs-and-secrets/midpoint/shibboleth/idp-metadata.xml
/etc/shibboleth/shibboleth2.xmlconfigs-and-secrets/midpoint/shibboleth/shibboleth2.xml
/etc/shibboleth/sp-cert.pemconfigs-and-secrets/midpoint/shibboleth/sp-cert.pem

You can modify or replace these files as needed.

In addition to these, there are environment variables, Docker secrets and configuration files used to configure other components like Grouper or Shibboleth IdP. For their setup please consult relevant products' documentation.

  • No labels