The major obstacle to getting the Grouper docker container to run in the OpenShift environment is that the process user runs as a userid with few permissions, and which isn't known at build time. This means that the Grouper container processes that normally run as either tomcat or root must be runnable under this low-permission context. These issues can be solved by building a derived container that includes a combination of file permission changes plus minor changes to the library scripts in /usr/local/bin. Since version 2.5.40, file permissions in the Grouper container have been adjusted to better support OpenShift out of the box. In addition, this version adds a new environment variable GROUPER_OPENSHIFT, which triggers the setting of other variables needed in OpenShift – GROUPER_CHOWN_DIRS=false, GROUPER_USE_PIPES=false, GROUPER_GSH_CHECK_USER=false, and GROUPER_RUN_PROCESSES_AS_USERS=false.
One trait of the OpenShift container (and Docker containers in general) is that if a user does not have a primary group, it will run with the
root group. This can help in the early stages of development. By building an image from a Dockerfile including a
USER <random id> statement, you can start it up locally as a simple Docker image and look for errors, without needing to do frequent OpenShift deployments.
In OpenShift, the Grouper UI or WS only needs to run Tomcat as its only process, on port 8080. Shibboleth can be run as a separate sidecar container, proxying both the login and SSL.
Configuration (since 2.5.40)
For configuration, it works well to set (a) properties that vary between environments in database configuration (Miscellaneous > Configuration in the UI); (b) properties the same in all environments as either property files in the subimage, or in the database; (c) the minimal set of properties needed for startup as environment secrets; and (d) properties specific to OpenShift as normal deployment env variables. Although secrets could be bind mounted in /run/secrets/grouper_* files, the ease of secret setup in OpenShift makes the setup in (c) preferred. Minimally, the morphString key and the database password are the only two properties that need to be set up this way. However, it may simplify maintenance if the database url and username are also included, since they are likely to be maintained at the same time.
Create an opaque secret with these values in it, and then add it to the deployment configuration as Environment from Secret.
Other environment variables necessary or optional for OpenShift are:
- TZ: America/New_York (set to local time zone, otherwise the container may default to UTC)
- GROUPER_OPENSHIFT: true (if you only want to run your image in OpenShift you can add to the Dockerfile, but if not add, it can be inserted here)
- GROUPER_RUN_TOMCAT_NOT_SUPERVISOR: true (this should be added as part of the GROUPER_OPENSHIFT switches in a future version)
- GC_MAX_METASPACE_SIZE: 256 (helps with startup memory)
- GROUPER_MAX_MEMORY: 1792m (the default is 1500m which actually may be enough; the critical factor in JVM heap sizing is that the container's memory seems to need at least 700M to avoid out of memory crashes at the OS level; in our case the container limit was 2560Mi, and a value of 1792m gave sufficient spare memory to the OS to prevent crashes)
JVM_OPTS: -XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap -XX:MaxRAMFraction=1 -Xms64M (the extra options are supposed to help the JVM have awareness of the actual VM memory it can use)
The startup command in theory should just be "ui" or "ws". But OpenShift doesn't pick up the entrypoint value of "entrypoint.sh" in the Grouper standard image for some reason, so it needs to be included. Thus, the startup command would be "/usr/local/bin/entrypoint.sh ui". If your OpenShift UI doesn't have a place to configure the start command, it can be added by editing the deployment config yaml:
The readiness check can utilize the diagnostic status page. This can be useful not only to detect pod availability, but also helps in WS to force a preload of object data to speed up the first external call. I could not get the "HTTP GET" health check to work. But a curl command also works:
Note that when building an image from a Dockerfile, it will inspect the code, see that its base image is i2incommon/grouper, and create an image in the registry for the specified version. Once the buildConfig has been created with the base image, subsequent builds will ignore the FROM version in the Dockerfile even if it changes, preferring the one set up in the buildConfig. Thus, when upgrading versions of Grouper, both the FROM base image in the Dockerfile, and the base image defined in the buildConfig need to be modified.
Changes to startup scripts (prior to 2.5.40)
The following changes to the Grouper library scripts in /usr/local/bin fix some issues with temporary files, running
rm as a non-owner, and the non-existent userid. Until the changes are made to the base Grouper container, they should be copied from the Grouper container Git project, patched locally, and then included in the new Docker build.
Dockerfile (prior to 2.5.40)
The Docker subimage includes some extra environment variables, imports the modified library scripts, and sets the extra file permissions needed to do file preparation and run the TomEE process. Note that this also includes a USER directive to set the user to an invalid userid. This is for quick testing in the local environment, and should be removed when building the image in OpenShift.
Running a container
Besides the environment variables baked into the image, there are also some additional variables that are better left to the Docker run command, since they differ depending on which Grouper component is run. The
ws, etc. commands that are normally used to start a container can't be used, because they add additional environment variables that force Apache and Shibboleth to be run as services, which we don't need in OpenShift.
UI (leave out
ui in container run command):
WS (leave out
ws for container):
daemon to invoke)
gsh to invoke)
UI + WS: Because this is running only TomEE without Apache, there is only one context ("grouper", or defined by GROUPER_TOMCAT_CONTEXT) running. Thus, both the UI and WS will be accessed under the single context. If you need the UI running under /groupre and WS running under grouper-ws, further modifications (as a future exercise) would be needed.