In the TIER/midPoint_container GitHub project there are artifacts needed to build and deploy dockerized version of midPoint suitable to use within the TIER IdM environment.

This is the status of the work:

...

1. Source: standard maintained Centos 7 docker image
2. (Under Discussion) potential use of Centos 7 image from Dockerhub that includes what is needed to use systemd as init (instead of supervisord).  We may enable this option if obtaining/implementing the logging changes we need to supervisord are hard - - https://hub.docker.com/r/centos/systemd/
3. When build pipelines are published for production, they must include a yum update step.

...

1. If a relational database is provided within a container, MARIADB will be used.
2. In general, database support is normally handled externally by the user or via a TIER-maintained MARIADB container.

Solution comments: The midPoint repository can be attached to the midPoint server in a flexible way. It can be either deployed in an (alternative) Docker container, or be provided externally either on premises or in the cloud.

...

7. Container Configuration

a) Standard Data

...

1. Containers may receive configuration data via the environment as described below for Secret Data (7.b)
2. Configuration data may be mounted into the container from external storage
3. Configuration data may be "burned" into the container while it is being built.
4. There are many trade-offs between ii and iii, some environments will choose to enable the end user to build their containers using either method.

...

7. Container Configuration

b) Secret Data

...

1. The preferred mechanism to support data that must be protected (e.g., passwords, keys, etc.) is Swarm-mode Docker Secrets.
2. Docker secrets are read-only to the application.
3. Secret Processing - Docker Secrets are processed using one of the two mechanisms described below
   
   1. Secrets/Pointers-to-Secrets are passed in the environment using the syntax described below.  Either a single value may be supplied or, with the _FILE suffix, a file pointing to a docker swarm secret location 
      1. COMPONENT_DATABASE_PASSWORD=foobar
      2. COMPONENT_DATABASE_PASSWORD_FILE=/run/secrets/my_password_file
         
      3. Container startup scripts
         
         1. Start-up scripts process the environment and do whatever setup is needed to make secrets usable in the application.
         2. If the environment contains both a _FILE and name-only variables, the _FILE form is to be used.
            
      4. Documentation/comments for each attribute is required.
   2. A naming convention is developed for all application files that will exist in /run/secrets.  Scripting within the container appropriately processes these files, linking them to the application components as appropriate.  Documentation/comments re: the naming convention and files is required.

...

1. Containers designed for compatibility/ease of use with Docker SWARM mode using Docker Stack Deploy and Compose files.
2. Work to not preclude the use of other orchestration frameworks.
3. Secrets are automatically mounted in /run/secrets by docker stack deploy using a compose file.

...

1. All logs from all elements within a container are written to stdout
2. Goal: easily parsable records; future work is likely to include json formatted logs
3. Lines (records) within each log file start with the following format

   1. Component Name (e.g., Shibboleth IdP, Grouper Loader, etc.)

   2. Native logfile name (e.g., Catalina.out, shibd.log, etc.)

   3. Environment (e.g., Prod, Dev, Test)

   4. A user supplied token via the environment

   5. The text of the logfile line, without modification.
4. Records within a line are separated by the semi-colon character.  Semicolons are not permitted in the first four fields and must be removed if present.
5. Spaces also need to be removed from the (c) Environment and (d) User Supplied fields of each record. If anyone remembers why we need ro remove these spaces, please comment here. 

6. Example Records

   1. supervisord;console;testing;Build:1.2.3;2018-04-02 18:27:30,778 CRIT Set uid to user 0

   2. tomcat;catalina.out;testing;Build:1.2.3;2018-04-02 18:27:32,915 [main] INFO  org.apache.coyote.http11.Http11NioProtocol- Initializing ProtocolHandler ["https-jsse-nio-443"]

   3. Timestamps in logs must default to UTC.  Documentation should exist to assist users with changing this default to a local timezone.  The default of UTC instead of EST or PST seems logical given that many future campus deployments will include components deployed in multiple timezones for redundancy.

...

Introduction

This page shows how to get started with a Docker image for the midPoint component of the InCommon Trusted Access Platform.

Anchor
prerequisites prerequisites

Prerequisites

In order to set up and run this container and associated demonstrations, you need a Linux machine with a reasonably recent Docker and docker-compose installation. The most advanced demo/grouper optionally uses an LDAP browser, e.g. Apache Directory Studio that itself requires Java.

This container and demos were tested on Ubuntu 18.04.1 LTS with

• Docker 18.06.1-ce,
• docker-compose 1.17.1 (and 1.22.0),
• libxml2-utils package (in order to have xmllint command available),
• Apache Directory Studio 2.0.0.v20170904-M13 with OpenJDK 8.

The LDAP browser is really optional. It is used only to check that LDAP objects are created correctly. You can safely proceed without it.

It seems that the usual way of installation via e.g. apt-get does not always guarantee sufficiently recent versions of Docker and docker-compose. To have the latest stable versions you can use procedures described here:

• https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-docker-ce
• https://docs.docker.com/compose/install/

Also, make sure appropriate ports are available on the host machine. They are listed in the documentation to individual demonstrations; usually they are 8443 and 3306, sometimes 389, 443, or 5432. The demo/grouper needs even more free ports, please see the description.

Getting started (simple demo)

The image cannot be "run" by itself as it requires a midPoint repository - i.e. an SQL database - to execute. So the easiest way to start dockerized midPoint is to use one of the provided demonstrations. The most logical choice for just getting started with midPoint is demo/simple.

$ git clone https://github.internet2.edu/Docker/midPoint_container.git
$ cd midPoint_container/demo/simple
$ docker-compose up

Building own images

The above commands download TAP midpoint image from Internet2's enterprise github. Alternatively, you can build this image yourself. Here is how:

$ git clone https://github.internet2.edu/Docker/midPoint_container.git
$ cd midPoint_container
$ ./build.sh
$ cd demo/simple
$ docker-compose up

(Note the build.sh has a -n switch that skips downloading the midPoint distribution archive, saving some time during repeated builds.)

After starting

After docker-compose up command successfully finishes you should see something like this on the console:

midpoint_server_1  | midpoint;midpoint.log;demo;;2018-09-20 16:25:22,191 [] [main] INFO (org.springframework.boot.web.embedded.tomcat.TomcatWebServer): Tomcat started on port(s): 8080 (http) 9090 (http) with context path '/midpoint'
midpoint_server_1  | midpoint;midpoint.log;demo;;2018-09-20 16:25:22,209 [] [main] INFO (com.evolveum.midpoint.web.boot.MidPointSpringApplication): Started MidPointSpringApplication in 60.512 seconds (JVM running for 61.688)

Now you can log into midPoint using https://localhost:8443/midpoint URL, with an user of administrator and a password of 5ecr3t. If you are using midPoint 4.8.1 or greater please check the documentation on how to set an initial password. 

Beyond the simple demo

Besides the midPoint image itself, and the simple demo discussed above, the project contains demonstrations that integrate midPoint with selected other components

Configuring the container (general information)

The lowest level of configuration of the midPoint container is during its inclusion into a Docker composition. There is the full set of environment properties and other configurable items (e.g. Docker secrets and configs) available.

During the composition some of the environment properties can be made accessible from the outside. This depends strictly on the compositor. The demonstrations here show some of the options.

How to set environment variables after composition is done

After the composition is done, you can set the environment variables like this:

$ export ENV="test" USERTOKEN="4.0.1" MP_MEM="4096m"
$ docker-compose up

Or like this:

$ env ENV="test" USERTOKEN="4.0.1" MP_MEM="4096m" docker-compose up

How to set Docker secrets and configs

The way of accessing secrets and configs is specific to the composition. In our demonstrations these are stored in the configs-and-secrets directory. They are provided to midPoint containers in appropriate ways. (Currently, secrets are passed as Docker secrets, configs are mounted as volumes. This might be changed in the future.) For detailed information on individual items please see the following sections.

Configuring specific container features

In this section we describe how to configure and use specific features of this midPoint dockerization.

Repository

Documentation

Logging feature

Logging is configured by setting the following environment variables: either from the command line or from docker-compose.yml (see commented-out examples in the provided file).

...

According to the specification, semicolons in these fields are eliminated (replaced by underscores). The same is done for spaces in ENV and USERTOKEN.

...

Repository configuration is done via the following environment variables.

Environment variable	Meaning	Default value
REPO_DATABASE_TYPE	Type of the database. Supported values are mariadb, mysql, postgresql, sqlserver, oracle. It is possible to use H2 as well but H2 is inappropriate for production use.	mariadb
REPO_JDBC_URL	URL of the database.	MariaDB: jdbc:mariadb://$REPO_HOST:$REPO_PORT/$REPO_DATABASE?characterEncoding=utf8 MySQL: jdbc:mysql://$REPO_HOST:$REPO_PORT/$REPO_DATABASE?characterEncoding=utf8 PostgreSQL: jdbc:postgresql://$REPO_HOST:$REPO_PORT/$REPO_DATABASE SQL Server: jdbc:sqlserver://$REPO_HOST:$REPO_PORT;database=$REPO_DATABASE Oracle: jdbc:oracle:thin:@$REPO_HOST:$REPO_PORT/xe
REPO_HOST	Host of the database. Used to construct the URL.	midpoint-_data
REPO_PORT	Port of the database. Used to construct the URL.	3306, 5432, 1433, 1521 for MariaDB/MySQL, PostgreSQL, SQL Server and Oracle, respectively
REPO_DATABASE	Specific database to connect to. Used to construct the URL.	midpointregistry
REPO_USER	User under which the connection to the database is made.	rootregistry_user
REPO_PASSWORD_FILE	File (e.g. holding a docker secret) that contains the password for the db connection.	/run/secrets/mmp_database_password.txt

Docker secrets

As of v3.9devel-578-gb20f43e (September 10th, 2018), each configuration parameter can be supplied either as a string value or as a file reference. This is to allow using Docker secrets to provide values for sensitive parameters.

Currently, there are two standard places when references to Docker secrets might be used:

...

For automatic schema creation and upgrade options please see Schema creation and updating section in midPoint documentation.

Note that in order to connect to the database you have to provide the password. For security reasons, we use the indirect way through file access. So, typically you provide the following Docker secret:

Of course, you can provide the password file in any other way, assuming you correctly set REPO_PASSWORD_FILE environment variable.

Logging

Logging is configured by setting the following environment variables:

According to the specification, semicolons and spaces in these fields are eliminated. We decided to replace them by underscores.

Authentication

This midPoint dockerization supports two authentication mechanisms.

Authentication configuration is done using the following environment variables.

Note that besides these variables you have to provide the following files. They are necessary for the Shibboleth service provider module.

And the following Docker secrets are to be provided:

Other

Other aspects can be configured using the following variables and Docker secrets or configs.

Other files that are necessary for this midPoint container to function are:

And the following Docker secrets are to be provided:

Guiding requirements for this project: TIER Docker Container Specification

This is how these file references are used:

If a configuration parameter name (e.g. midpoint.keystore.keystorePassword) is suffixed by _FILE, i.e. forming midpoint.keystore.keystorePassword_FILE, then the value of the configuration parameter is taken from the specified file. As with the regular parameters, the file pointer can be defined either in config.xml or in the command line, using -D Java option. So, for example, mapping of REPO_PASSWORD_FILE and KEYSTORE_PASSWORD_FILE environment variables is done by the following lines in midPoint Dockerfile:

ENV REPO_PASSWORD_FILE /run/secrets/m_database_password.txt
ENV KEYSTORE_PASSWORD_FILE /run/secrets/m_keystore_password.txt

(...)

# Execution
CMD java -Xmx$MEM -Xms2048M -Dfile.encoding=UTF8 \
(...)
       -Dmidpoint.repository.jdbcPassword_FILE=$REPO_PASSWORD_FILE \
       -Dmidpoint.keystore.keyStorePassword_FILE=$KEYSTORE_PASSWORD_FILE \
(...)
       -jar $MP_DIR/lib/midpoint.war

If needed, other sensitive information can be provided to midPoint in similar way. In particular, constants are expected to be used here, typical example being resource passwords.

Shibboleth integration

...