Grouper container documentation

The Grouper project currently supports Grouper v2.5+ running only in the i2incommon container (including of course subcontainers).  If you want to take pieces out and run in an unsupported way feel free to do that on your own.  If you have questions about how to do something with the container that you think you cannot do and think you need to do surgery, please discuss it with the Grouper team first, there is likely a way to do it with the existing container.   We are open to changing how the container is structured in v2.6, lets discuss and improve together.

• Grouper container - changes in 2.6/2.7
• Grouper Container running as non-root
• Grouper container v2.6 change JVM
• Grouper customize SSL certificate
• Grouper dev and container strategy for v2.5
• Grouper installer task to build the container v2.5
• Grouper v2.5 container apply patch
• Grouper v2.5 container docker-compose example
• Grouper v2.5 container SSL trust management
• Grouper v2.5 container unit tests
• Grouper v2.5 customize container config files
• Grouper v2.5 customize UI authentication Shibboleth
• Grouper v2.5 customize UI security headers
• Running Grouper in a different container
• Running Grouper not in a container

This is how the Grouper container works in v2.5

Directory structure

alpha order...

        <Logger name="edu.a.b.c" level="debug" additivity="false">
            <AppenderRef ref="grouper_error"/>
        </Logger>

        <RollingFile name="some_appender_name" fileName="/opt/grouper/logs/someFile.log" 
                filePattern="/opt/grouper/logs/someFile.log.%d{yyyy-MM-dd}" >
            <PatternLayout pattern="someFile.log;${ENV};${USERTOKEN};${layout}"/>
            <Policies>
                <TimeBasedTriggeringPolicy interval="1"/>
            </Policies>
            <DefaultRolloverStrategy max="30" />
        </RollingFile>

  

Ports and configs

RUN yum install -y openssh-server
RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key -N ''
RUN ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key -N ''
EXPOSE 22

[program:sshd]
command=/usr/sbin/sshd -D
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0

#!/bin/sh

grouperScriptHooks_setupFilesPost() {
  cp -v /opt/tier-support/supervisord.conf /opt/tier-support/supervisord.conf.origGrouper
  cat /opt/tier-support/supervisord_sshd.conf >> /opt/tier-support/supervisord.conf
  # assumes you have an env var PASS from your password manager
  echo "root:${PASS}" | chpasswd
}
export -f grouperScriptHooks_setupFilesPost

Grouper Container params

There are a few arguments you can pass to the container, and env vars...   Note the command if specified (optional) will set env vars before the env vars.  So you could call the container with "ui" but then specify that -e GROUPER_RUN_SHIB_SP='false'   (e.g. if you run CAS)

docker run --detach --mount type=bind,src=/opt/grouperInstaller/logs,dst=/opt/grouper/logs --mount type=bind,src=/opt/grouperInstaller/slashRoot,dst=/opt/grouper/slashRoot --name gsh i2incommon/grouper:2
.5.XX /opt/grouper/grouperWebapp/WEB-INF/bin/gsh.sh /opt/grouper/grouperWebapp/WEB-INF/bin/createGrouperSystemPasswordUi.gsh

RUN /usr/local/bin/changeUid.sh tomcat 1870 \
  && /usr/local/bin/changeGid.sh tomcat 1870 \
  && /opt/container_files/docker-build-bin/containerDockerfileInstallPermissions.sh tomcat root

docker run -u 1870 ...

1. (in container) ps -ef | grep tom

2. (in container) sudo -u tomcat /usr/lib/jvm/java-1.8.0-amazon-corretto/bin/jmap -heap <pid>      (see max heap, should be approx what you expect)

3. (in container and on host if applicable) free -m

4. (in container) set | grep MEM

5. (in container) top

[tomcat@3af22d6c689f4e879e25071bacf99a3c-2291675529 root]$ set | grep MEM
GROUPER_MAX_MEMORY=3000m

[root@3af22d6c689f4e879e25071bacf99a3c-2291675529 ~]# free -m
              total        used        free      shared  buff/cache   available
Mem:           7763        3557         555           0        3650        3949
Swap:             0           0           0

# e.g. postgres (a):        jdbc:postgresql://localhost:5432/database
# e.g. postgres (b):        jdbc:postgresql://localhost:5432/database?currentSchema=mySchema
# e.g. mysql:           jdbc:mysql://localhost:3306/grouper?useSSL=false
# e.g. oracle:          jdbc:oracle:thin:@server.school.edu:1521:sid
# e.g. p6spy (log sql): [use the URL that your DB requires]

RUN chown tomcat:root  /opt/grouper/ /etc/httpd/conf/ /home/tomcat/ /opt/tomcat/ /usr/local/bin /etc/httpd/conf.d/ /opt/tier-support/ /usr/lib/jvm/java/jre/lib/security/cacerts \    
&& chown -R tomcat:root $(find /opt/grouper/ /etc/httpd/conf/ /home/tomcat/ /opt/tomcat/ /usr/local/bin /etc/httpd/conf.d/ /opt/tier-support/ ! -user tomcat -o ! -group root -print) \    
&& chmod g+rwx /opt/grouper/ /etc/httpd/conf/ /home/tomcat/ /opt/tomcat/ /usr/local/bin /etc/httpd/conf.d/ /opt/tier-support/ /usr/lib/jvm/java/jre/lib/security/cacerts \    
&& chmod -R g+rwx $(find /opt/grouper/ /etc/httpd/conf/ /home/tomcat/ /opt/tomcat/ /usr/local/bin /etc/httpd/conf.d/ /opt/tier-support/ ! -perm -g+rwx )  \    
&& chmod +x /opt/grouper/*.sh 

Note: If customizing a particular component's logging, it is recommended that the file be source from the image (docker container cp) or from the image's source repository.

shibd shibd.log dev build-2 2018-03-27 20:42:22 INFO Shibboleth.Listener : listener service starting
grouper-api grouper_event.log dev build-2 2018-03-27 21:10:00,046: [DefaultQuartzScheduler_Worker-1] INFO  EventLog.info(156) -  - [fdbb0099fe9e46e5be4371eb11250d39,'GrouperSystem','application'] session: start (0ms)
tomcat console dev build-2 Grouper starting up: version: 2.3.0, build date: null, env: <no label configured>

RUN . /usr/local/bin/library.sh && setupFilesForComponent_playwrightInstallOsLibsHelper

Grouper startup logs (v2.5.34+)

When the Grouper container starts up, it does a lot of work to manipulate the config files.  The logs are printed the container logs.  Note, this prints the bash file and function which calls the command, and prints the exit code (0 is usually success).  When it gets to "Starting supervisor" then the container should start.

grouperContainer; INFO: (library.sh) Start loading library.sh
grouperContainer; INFO: (library.sh) End loading library.sh
grouperContainer; INFO: (libraryPrep.sh-prep_conf) Start setting up initial pipes
grouperContainer; INFO: (librarySetupPipe.sh-setupPipe) Setup pipe: /tmp/logpipe
grouperContainer; INFO: (librarySetupPipe.sh-setupPipe) Setup pipe: /tmp/logsuperd
grouperContainer; INFO: (librarySetupPipe.sh-setupPipe) Setup pipe: /tmp/loggrouper
grouperContainer; INFO: (libraryPrep.sh-prep_conf) End setting up initial pipes
grouperContainer; INFO: (entrypoint.sh) Executing quickstart
grouperContainer; INFO: (libraryPrep.sh-prep_finish) End prep
grouperContainer; INFO: (librarySetupFiles.sh-setupFiles_storeEnvVars) Start store env vars in /opt/grouper/grouperEnv.sh
grouperContainer; INFO: (librarySetupFiles.sh-setupFiles_storeEnvVars) End store env vars in /opt/grouper/grouperEnv.sh
grouperContainer; INFO: (librarySetupFilesForProcess.sh-setupFilesForProcess_supervisor) Clear out supervisor.conf
grouperContainer; INFO: (librarySetupFilesApache.sh-setupFilesApache_supervisor) Appending supervisord-httpd.conf to supervisord.conf
grouperContainer; INFO: (librarySetupFilesApache.sh-setupFilesApache_selfSignedCert) cp /opt/tier-support/ssl-enabled.conf /etc/httpd/conf.d/ , result: 0
grouperContainer; INFO: (librarySetupFilesApache.sh-setupFilesApache_ports) Replace apache ssl port in ssl-enabled.conf', result: 0
patching file /etc/httpd/conf/httpd.conf
grouperContainer; INFO: (librarySetupFilesApache.sh-setupFilesApache_indexes) Patch httpd.conf to turn off indexes 'patch /etc/httpd/conf/httpd.conf /etc/httpd/conf.d/httpd.conf.noindexes.patch' result=0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_loggingSlf4j) rm /opt/tomee/lib/slf4j-api*.jar , result: 0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_loggingSlf4j) rm /opt/tomee/lib/slf4j-jdk*.jar , result: 0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_loggingSlf4j) cp /opt/grouper/grouperWebapp/WEB-INF/lib/slf4j-api-*.jar /opt/tomee/lib , result: 0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_loggingSlf4j) cp /opt/grouper/grouperWebapp/WEB-INF/lib/slf4j-jdk*.jar /opt/tomee/lib , result: 0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_loggingSlf4j) rm /opt/grouper/grouperWebapp/WEB-INF/lib/slf4j-jdk*.jar , result: 0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_turnOnAjp) cp /opt/tomee/conf/server.xml /opt/tomee/conf/server.xml.currentOriginalInContainer , result: 0
patching file /opt/tomee/conf/server.xml
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_turnOnAjp) Patch server.xml to turn on ajp, result: 0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_supervisor) Append supervisord-tomee.conf to supervisord.conf
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_context) Replace context cookies in grouper.xml, result: 0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_context) Replace tomcat context in grouper.xml, result: 0
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_context) Set contexts in grouper-www.conf and other files, results: 0 0 0 0 0 0 0 0 0
patching file /opt/tomee/conf/server.xml
grouperContainer; INFO: (librarySetupFilesTomcat.sh-setupFilesTomcat_accessLogs) Patch server.xml to not log access, result: 0
grouperContainer; INFO: (librarySetupFilesForProcess.sh-setupFilesForProcess_shib) mv /etc/httpd/conf.d/shib.conf /etc/httpd/conf.d/shib.conf.dontuse , result: 0
grouperContainer; INFO: (librarySetupFilesForProcess.sh-setupFilesForProcess_supervisorFinal) Running processes as users in supervisord.conf, result: 0
grouperContainer; INFO: (librarySetupFilesForComponent.sh-setupFilesForComponent_ws) cp -r /opt/grouper/grouperWebapp/WEB-INF/libWs/* /opt/grouper/grouperWebapp/WEB-INF/lib/ , result: 0
grouperContainer; INFO: (librarySetupFilesForComponent.sh-setupFilesForComponent_scim) cp -r /opt/grouper/grouperWebapp/WEB-INF/libScim/* /opt/grouper/grouperWebapp/WEB-INF/lib/ , result: 0
grouperContainer; INFO: (librarySetupFilesForComponent.sh-setupFilesForComponent_ui) cp -r /opt/grouper/grouperWebapp/WEB-INF/libUiAndDaemon/* /opt/grouper/grouperWebapp/WEB-INF/lib/ , result: 0
grouperContainer; INFO: (librarySetupFilesForComponent.sh-setupFilesForComponent_quickstart) edit grouper.hibernate.base.properties with UI GrouperSystem password for quick start, result: 0
grouperContainer; INFO: (librarySetupFilesForComponent.sh-setupFilesForComponent_quickstart) edit grouper.hibernate.base.properties with WS GrouperSystem password for quick start, result: 0
grouperContainer; INFO: (librarySetupFiles.sh-setupFiles_loggingPrefix) Changing log prefix to grouper in log4j.properties, result: 0
grouperContainer; INFO: (librarySetupFiles.sh-setupFiles_chownDirs) chown -R tomcat:root /opt/grouper/grouperWebapp, result: 0
grouperContainer; INFO: (librarySetupFiles.sh-setupFiles_chownDirs) chown -R tomcat:root /opt/tomee, result: 0
grouperContainer; INFO: (libraryRunCommand.sh-runCommand) Start setting up remaining pipes
grouperContainer; INFO: (librarySetupPipe.sh-setupPipe) Setup pipe: /tmp/loghttpd
grouperContainer; INFO: (librarySetupPipe.sh-setupPipe) Setup pipe: /tmp/logtomcat
grouperContainer; INFO: (libraryRunCommand.sh-runCommand) End setting up remainder pipes
grouperContainer; INFO: (libraryRunCommand.sh-runCommand) Starting supervisor

Tomcat contexts (v2.5.28+)

There is some complexity in tomcat contexts for a few reasons

1. Only one webapp should be running at a time, do not put multiple context configs in /opt/tomcat/conf/Catalina/localhost
   1. If running multiple modules (e.g. ws/ui/daemon), those run in the same webapp
2. If there is one module running, there should be a specific context config so Grouper knows the context (e.g. for SOAP WSDL), and can customize it (e.g. turn off cookies for WS/Scim)
3. The apache grouper config should dynamically adjust based on which modules are running (in v5 there is no apache)
4. If you dont care about SOAP or cookies and just want everything in the same /grouper tomcat context, just set this env var: GROUPER_USE_GROUPER_CONTEXT=true

To customize the grouper.xml tomcat config:

1. You can overlay the /opt/tomcat/conf/Catalina/localhost/grouper.xml (note, its tomee pre v4)
2. If only WS is running, then grouper.xml will be renamed grouper-ws.xml unless overridden with GROUPERWS_URL_CONTEXT
3. If only SCIM is running, then grouper.xml will be renamed grouper-ws-scim.xml unless overridden with GROUPERSCIM_URL_CONTEXT
4. Otherwise grouper.xml will remain
5. If grouper.xml remains, it will be renamed if overridden with GROUPER_TOMCAT_CONTEXT
   
   
   

   
   

To customize the /etc/httpd/conf.d/grouper-www.conf: (removed in v5)

1. This is the base file now, use this as a template for overlays (look in container for most recent version)
   
   

    Click here to expand...

   Timeout __GROUPER_APACHE_AJP_TIMEOUT_SECONDS__
   ProxyTimeout __GROUPER_APACHE_AJP_TIMEOUT_SECONDS__
   ProxyBadHeader Ignore
   
   # the variable for _ _GROUPER_APACHE_AJP_TIMEOUT_SECONDS_ _ will be replaced to default for one hour on startup env var $GROUPER_APACHE_AJP_TIMEOUT_SECONDS
   # the variable for _ _THE_AJP_URL_ _ (no spaces) will be replaced with something like: ajp://localhost:port/grouper   on startup
   # the variable for _ _GROUPER_PROXY_PASS_ _ (no spaces) will be replaced with comment or blank on startup if running grouper url
   # the variable for _ _GROUPERWS_PROXY_PASS_ _ (no spaces) will be replaced with comment or blank on startup if running grouper-ws url
   # the variable for _ _GROUPERSCIM_PROXY_PASS_ _ (no spaces) will be replaced with comment of blank on startup if running grouper-ws-scim url
   # the variable for _ _GROUPER_TOMCAT_CONTEXT_ _ (no spaces) will be replaced with the env var $GROUPER_TOMCAT_CONTEXT
   # the variable for _ _GROUPER_URL_CONTEXT_ _ (no spaces) will be replaced with the env var $GROUPER_URL_CONTEXT
   # the variable for _ _GROUPERWS_URL_CONTEXT_ _ (no spaces) will be replaced with the env var $GROUPERWS_URL_CONTEXT
   # the variable for _ _GROUPERSCIM_URL_CONTEXT_ _ (no spaces) will be replaced with the env var $GROUPERSCIM_URL_CONTEXT
   __GROUPER_PROXY_PASS__ProxyPass /__GROUPER_URL_CONTEXT__ ajp://localhost:8009/__GROUPER_TOMCAT_CONTEXT__ timeout=__GROUPER_APACHE_AJP_TIMEOUT_SECONDS__
   __GROUPERWS_PROXY_PASS__ProxyPass /__GROUPERWS_URL_CONTEXT__ ajp://localhost:8009/__GROUPER_TOMCAT_CONTEXT__ timeout=__GROUPER_APACHE_AJP_TIMEOUT_SECONDS__
   __GROUPERSCIM_PROXY_PASS__ProxyPass /__GROUPERSCIM_URL_CONTEXT__ ajp://localhost:8009/__GROUPER_TOMCAT_CONTEXT__ timeout=__GROUPER_APACHE_AJP_TIMEOUT_SECONDS__
   
   __GROUPER_PROXY_PASS__RewriteEngine on
   __GROUPER_PROXY_PASS__RewriteRule "^/$" "/__GROUPER_URL_CONTEXT__/" [R]
   
   
   
   

2. Variables available (removed in v5)
   
   

   Grouper apache config variable	Value	Condition
   __THE_AJP_URL__	ajp://localhost:8009/grouper	If the UI is running, or multiple modules (e.g. SCIM and WS)
   __THE_AJP_URL__	ajp://localhost:8009/grouper-ws	If the WS is running and not UI or SCIM
   __THE_AJP_URL__	ajp://localhost:8009/grouper-ws-scim	If SCIM is running and not WS or SCIM
   __GROUPER_PROXY_PASS__	<blank>	If the UI is running
   __GROUPER_PROXY_PASS__	#   (comment)	If the UI is not running
   __GROUPERWS_PROXY_PASS__	<blank>	If the WS is running
   __GROUPERWS_PROXY_PASS__	#   (comment)	If the WS is not running
   __GROUPERSCIM_PROXY_PASS__	<blank>	If SCIM is running
   __GROUPERSCIM_PROXY_PASS__	#   (comment)	If SCIM is not running

3. Use those variables in your overlay.  You should be able to have one overlay for all your environments, though maybe you want separate subimages for each env

Dockerfile

If you create dirs or copy things to the webapp, you should set the owner at the end of your Dockerfile (or whatever user/group you run tomcat as)

RUN /opt/container_files/docker-build-bin/containerDockerfileInstallPermissions.sh tomcat root

Container startup workflow (v2.5.28+)

Custom overridable hooks in green

There are three main phases:

1. Setup logging
2. Setup environment variables.  Note: generally these are only set if not already set so they can be customized
3. Setup files
4. Run Processes

Versions

• Tag in github docker is: 2.5.X where X is an integer that increases for each build
• There is a listing of each version in the Grouper 2.5 release notes, with an indication on if it is stable or not
• Every container will have "yum update" done before the container is released.  As needed tomcat will be updated

Jstack

If you want to know what is hung or what is currently using CPU, you need to run jstack.  You need to be able to shell in to your container, and the OS needs to allow jstack (lwp).

Know what node to be on, if it is a daemon node, and you have multiple, look in the daemon logs on the UI, see the running job, and see which node to go to.

Shell into your terminal and be the same user as tomcat/tomee.

PS and get the process ID of tomee/tomcat

ps -ef | grep catalina

Run jstack:

jstack 71

Misc

• If you use Kubernetes, use an ARG for the "ui" or "ws" and not a COMMAND
• All of your containers must have the correct time and consistent timezones
• HTTP Strict Transport Security (HSTS) is enabled on the Apache HTTP Server. (apache removed in v5)
• morphStrings functionality in Grouper is supported. It is recommended that the various morphString files be associated with the containers as Docker Secrets. Set the configuration file properties to use `/run/secrets/secretname`.
• /run/secrets 

  • If you have a secret available in /run/secrets/grouper_fileName.pass, it will be linked from  /opt/grouper/grouperWebapp/WEB-INF/classes/fileName.pass   (change fileName.pass name as needed)

  • If you have a secret available in /run/secrets/shib_fileName.pass, it will be linked from  /etc/shibboleth/fileName.pass   (change fileName.pass name as needed)

  • If you have a secret available in /run/secrets/httpd_fileName.pass, it will be linked from  /etc/httpd/conf.d/fileName.pass   (change fileName.pass name as needed)

  • If you have a secret named /run/secrets/host-key.pem, it will be linked from /etc/pki/tls/private/host-key.pem
• Configure Grouper UI and WS authentication

• Take a heap snapshot
  
  

  [root@43c6b6e000b3 ~]# sudo -u tomcat  /usr/lib/jvm/java-1.8.0-amazon-corretto/bin/jmap -dump:file=/tmp/memory.dump <pid>

• See the env vars of a process, get the pid fist with ps-ef, then, run as the user (in this case tomcat) and substitute the pid in
  
  

  [root@73e1e62a5fa1 bin]# sudo -u tomcat cat /proc/45/environ | tr '\0' '\n'

Jars

• If you want a jar in all JVMS (ui/ws/daemon/gsh/scim), add it to /opt/grouper/grouperWebapp/WEB-INF/lib
• If you want a jar in ui/daemon/gsh only, add it to /opt/grouper/grouperWebapp/WEB-INF/libUiAndDaemon

• If you are replacing a jar (e.g. an existing driver), you need to remove it first from your dockerfile by wildcard, or overlay a blank file (risky since filenames can change)
  
  

  RUN rm -rf /opt/grouper/grouperWebapp/WEB-INF/lib/mysql-connector-java*.jar

• dfs