Overall architecture
The overall architecture of this demonstration composition is the following:
Generally, source systems provide data about majority of identities at the institution: students, faculty and other employees; potentially also about external persons. Those systems are typically student information system(s), human resource management system(s), and so on. They are primary sources of identity information.
Target systems, then, are primary consumers of identity data. These could be on-premises or cloud applications or services - like, for example, Office 365, G Suite (Google Apps), Slack, Jira, box.com, Coupa, Drupal, Liferay, GitLab, many kinds of ERP/CRM systems like SAP or Siebel; even student or HR information systems themselves; also individual servers, email platforms, other directories, and so on.
Source and target systems are shown in magenta color in the above schema.
Red components deal with managing user identities. These are (in this particular case) LDAP directory, midPoint, and Grouper.
- LDAP directory serves as the central directory of identities. It is used by both (some of) target systems and identity management components themselves.
- Because of its flexible interfacing features, midPoint takes care about collecting (some of) data from source systems, provisioning them into the central LDAP directory and to target systems. It also carries out complex processing of the data via mappings (i.e. transformations). Optionally, it allows designated users to augment/modify/review these data e.g. by modifying users' attributes not covered by source systems, or by assigning users specific roles, organizational membership, services or resources. If needed, it can also help with governance, risk management and compliance via features like approvals, recertification, reporting, auditing, policy rules, and so on.
- Grouper is - in this particular demo case - the primary tool for managing access policies and affiliations. Using its inclusion/exclusion mechanism it allows designated users to modify membership of groups defined in source systems. It also allows creation of ad-hoc groups that may or may not utilize pre-existing group membership.
Other components of the solution are:
- RabbitMQ is used to achieve near real-time synchronization from Grouper to midPoint. Grouper publishes changes to AMQP and midPoint updates itself based on those AMQP messages.
- Shibboleth Identity Provider (IdP) is used to achieve single-sign on inside and/or outside the institution. Specifically, both Grouper and midPoint users are authenticated using it.
Integration of other Internet2-provided components, e.g. COmanage, is currently out of scope of this demo.
Docker representation
Containers
The Docker containers for demo/grouper
composition look like this:
Each box in the picture is one container. Name of the container is shown in parentheses in box header, e.g. sources
, midpoint_data
, midpoint_server
, grouper_data
, and so on.
Some components of the demo are present just to demonstrate the architecture (like containers representing source systems), others are present as suggestions for real deployments: midpoint_data
, midpoint_server
, grouper_data
, grouper_ui
, grouper_daemon
, mq
, directory
, idp
.
Let us describe individual containers now. (After composition is set up, the real container names are in the form of grouper_ABC_1
e.g. grouper_sources_1
or grouper_midpoint_server_1
.)
Container | Content | Description |
---|---|---|
sources | MariaDB server | This serves as an example of source systems. MariaDB here hosts |
midpoint_server | midPoint software | This container hosts midPoint itself: its GUI, connectors to data sources/targets, background tasks, and REST/SOAP services. It is currently 4.1-SNAPSHOT version (not the released 4.0.x because of some Internet2-specific enhancements that have been done). The container itself is quite stateless: midPoint repository is in separate container (see below) and the midPoint home directory with connector files, custom schema, and log files is in complex_midpoint_home volume. |
midpoint_data | MariaDB server | This container provides the repository for midPoint. It is separate, so it can be replaced as needed: either with a container hosting any other supported database, or with a non-dockerized repository (on-premises or in cloud). |
grouper_ui | Grouper software | Here the Grouper UI executes. |
grouper_ws | Grouper software | Here the Grouper WS (Web Services) executes. |
grouper_daemon | Grouper software | Here the Grouper loader jobs execute. |
grouper_data | MariaDB server | This is the repository for Grouper. In a similar way as for midPoint, it is separate so that it can be replaced as needed. |
directory | 389ds server | This is the central LDAP directory. It is used by Grouper as a primary source of subjects and "imported" group membership (from source systems and optionally from midPoint). It is also used by Shibboleth IdP as source of authentication information. |
mq | RabbitMQ | This provides near real-time synchronization between Grouper and midPoint and optionally target systems. |
idp | Shibboleth IdP | Here Shibboleth IdP executes, providing authentication service for Grouper and (in the future) also for midPoint. |
Composition of these Docker containers is described in docker-compose.yml
file.
Communication
The containers publish the following TCP ports. (Port mapped to localhost denotes the mapping of container port to the host port where it can be reached from the outside.)
Container | Port number | Port mapped to localhost | Description |
---|---|---|---|
grouper_sources_1 | 3306 | 13306 | Port used to connect to the sources MariaDB |
grouper_midpoint_server_1 | 443 | 8443 | HTTPS port used to connect to midPoint application |
80 | - | HTTP port used to connect to midPoint application | |
grouper_midpoint_data_1 | 3306 | 33306 | Port used to connect to the default MariaDB repository |
grouper_grouper_ui_1 | 443 | 4443 | HTTPS port used to connect to Grouper UI |
80 | - | HTTP port used to connect to Grouper UI | |
grouper_grouper_ws_1 | 443 | 9443 | HTTPS port used to connect to Grouper Web Service |
80 | - | HTTP port used to connect to Grouper Web Service | |
grouper_grouper_daemon_1 | 443 | - | HTTPS port used to connect to Grouper daemon (if needed) |
80 | - | HTTP port used to connect to Grouper daemon (if needed) | |
grouper_data_1 | 3306 | 3306 | Port used to connect to the Grouper MariaDB repository |
grouper_directory_1 | 389 | 389 | LDAP connection to the directory |
grouper_mq | 15672 | 15672 | Port used to access RabbitMQ |
grouper_idp | 443 | 443 | HTTPS port to be used to connect to the Shibboleth IdP |
(Some other, less important ports that are internally exposed are not mentioned here.)
Docker volumes
The following volumes are created to persist data and other relevant files.
Volume name | Description | Used by container |
---|---|---|
grouper_source_data | Volume hosting MariaDB database used by the sources container | grouper_sources_1 |
grouper_source_mysql | Volume hosting /var/lib/mysql directory for the sources | grouper_sources_1 |
grouper_midpoint_home | The midPoint home directory. Contains schema extensions, logs, custom libraries, custom ConnId connectors, and so on. | grouper_midpoint_server_1 |
grouper_midpoint_data | Volume hosting MariaDB database used by midPoint | grouper_midpoint_data_1 |
grouper_midpoint_mysql | Volume hosting /var/lib/mysql directory | |
grouper_grouper_data | Volume hosting MariaDB database used by Grouper | grouper_grouper_data |
grouper_ldap | Volume hosting the LDAP database | grouper_directory |
grouper_mq | Volume hosting the messaging i.e. the /var/lib/rabbitmq directory | grouper_mq |
Configuring the composition
The following configuration properties are supported. Please refer to the main documentation page for their explanation.
Property | Default value |
---|---|
ENV | demo |
USERTOKEN | |
REPO_DATABASE_TYPE | mariadb |
REPO_JDBC_URL | default |
REPO_HOST | midpoint_data |
REPO_PORT | default |
REPO_DATABASE | registry |
REPO_USER | registry_user |
REPO_MISSING_SCHEMA_ACTION | create |
REPO_UPGRADEABLE_SCHEMA_ACTION | stop |
REPO_SCHEMA_VERSION_IF_MISSING | |
REPO_SCHEMA_VARIANT | |
MP_MEM_MAX | 2048m |
MP_MEM_INIT | 1024m |
MP_JAVA_OPTS | |
SSO_HEADER | uid |
TIER_BEACON_OPT_OUT | |
TIMEZONE | UTC |
You can tailor these to your needs.
The following Docker secrets are used:
Secret | Location |
---|---|
mp_database_password.txt | configs-and-secrets/midpoint/application/database_password.txt |
mp_keystore_password.txt | configs-and-secrets/midpoint/application/keystore_password.txt |
mp_host-key.pem | configs-and-secrets/midpoint/httpd/host-key.pem |
mp_sp-key.pem | configs-and-secrets/midpoint/shibboleth/sp-key.pem |
The following configuration files are used:
Target file | Source location |
---|---|
/etc/pki/tls/certs/host-cert.pem | configs-and-secrets/midpoint/httpd/host-cert.pem |
/etc/pki/tls/certs/cachain.pem | configs-and-secrets/midpoint/httpd/host-cert.pem |
/etc/shibboleth/idp-metadata.xml | configs-and-secrets/midpoint/shibboleth/idp-metadata.xml |
/etc/shibboleth/shibboleth2.xml | configs-and-secrets/midpoint/shibboleth/shibboleth2.xml |
/etc/shibboleth/sp-cert.pem | configs-and-secrets/midpoint/shibboleth/sp-cert.pem |
You can modify or replace these files as needed.
In addition to these, there are environment variables, Docker secrets and configuration files used to configure other components like Grouper or Shibboleth IdP. For their setup please consult relevant products' documentation.