The Grouper project currently supports Grouper v2.5+ running only in the i2incommon container (including of course subcontainers). If you want to take pieces out and run in an unsupported way feel free to do that on your own. If you have questions about how to do something with the container that you think you cannot do and think you need to do surgery, please discuss it with us first, I bet there is a way to do it with the existing container. We are open to changing how the container is structured in v2.6, lets discuss and improve together.
This is how the Grouper container works in v2.5
Directory structure
alpha order...
Path | Description |
---|---|
/etc/httpd/conf.d/grouper-www.conf | Apache config for Grouper. Includes Shibboleth directive if using Shibboleth |
/etc/shibboleth/shibboleth2.xml | Shibboleth config |
/opt/grouper/ | Grouper base dir |
/opt/grouper/grouperWebapp/ | J2EE webapp dir for grouper |
/opt/grouper/grouperWebapp/WEB-INF/classes/ | Grouper config files |
/opt/grouper/grouperWebapp/WEB-INF/classes/grouperText/grouper.text.en.us.properties | Externalized text file |
/opt/grouper/grouperWebapp/WEB-INF/ddlScripts | DDL scripts that are run automatically or not, are written here |
/opt/grouper/grouperWebapp/WEB-INF/libUiAndDaemon/ | Jars used in UI and daemon only. Place custom change log consumers here |
/opt/grouper/grouperWebapp/WEB-INF/lib/ | Jars used in all services (UI/WS/daemon/scim). Replace database drivers or add jars for all services |
/opt/grouper/grouperWebapp/WEB-INF/web.xml | web.xml for grouper might need security settings for authentication (e.g. tomcat LDAP authn for WS) |
/opt/grouper/logs | If you are externalizing logs to a "mount", this is the suggested standard location |
/opt/grouper/slashRoot | Any files or folders in here will be overlaid on / (root dir). This is useful for lower maturity levels If you have a dir in your host, lets say "/foo/bar/slashRoot", and you mount that to the container |
/opt/hsqldb | HSQLDB directory if using an embedded database (for quick starts or demos) |
/opt/tomee | Tomee/Tomcat app server |
/opt/tomee/bin | Startup and shutdown scripts for tomee |
/opt/tomee/conf/server.xml | Tomee/Tomcat server.xml might need settings for authentication (e.g. tomcat LDAP authn for WS) |
/usr/lib/jvm/java-1.8.0-amazon-corretto | JAVA_HOME |
/usr/lib/jvm/java-1.8.0-amazon-corretto/jre/lib/security/cacerts | cacerts for java |
Ports and configs
Port | Service | Description | Config file |
---|---|---|---|
443 | Apache | Apache listens to requests and reverse proxies to tomcat 8009. The apache URL path: /grouper → tomcat /grouper The apache URL path: /grouper-ws → tomcat /grouper The apache URL path: /grouper -scim → tomcat /grouper | /etc/httpd/conf.d/grouper-www.conf |
8009 | Tomee AJP | Tomcat listens here to get reverse proxied requests from apache (or another web server. Note, if you use an external apache you can link that up with the internal apache or expose the 8009 (in a secured way!) | /opt/tomee/conf/server.xml |
maps the /opt/grouper/grouperWebapp directory with | /opt/tomee/conf/Catalina/localhost/grouper.xml | ||
8005/8080/8443 | Tomee | These arent really used unless you configure and use them | /opt/tomee/conf/server.xml |
9001 | Hsqldb | Embedded database port (for quick starts and demos) |
Grouper Container params
There are a few arguments you can pass to the container, and env vars... Note the command if specified (optional) will set env vars before the env vars. So you could call the container with "ui" but then specify that -e GROUPER_RUN_SHIB_SP='false' (e.g. if you run CAS)
Argument | Description |
---|---|
ui | will set env vars: |
ws | will set env vars: GROUPER_WS='true' GROUPER_RUN_APACHE='true' GROUPER_RUN_TOMEE='true' |
scim | will set env vars: GROUPER_SCIM='true' GROUPER_RUN_APACHE='true' GROUPER_RUN_TOMEE='true' |
daemon | will set env vars: GROUPER_DAEMON='true' GROUPER_RUN_TOMEE='true' |
bin/gsh <gshScriptFileName> -or- /opt/grouper/grouperWebapp/WEB-INF/bin/gsh.sh <gshScriptFileName> | will just run gsh commands from docker command line e.g. docker run --detach --mount type=bind,src=/opt/grouperInstaller/logs,dst=/opt/grouper/logs --mount type=bind,src=/opt/grouperInstaller/slashRoot,dst=/opt/grouper/slashRoot --name gsh i2incommon/grouper:2 .5.XX /opt/grouper/grouperWebapp/WEB-INF/bin/gsh.sh /opt/grouper/grouperWebapp/WEB-INF/bin/createGrouperSystemPasswordUi.gsh |
ui-ws | will set env vars (if not overridden): GROUPER_UI='true' GROUPER_WS='true' GROUPER_RUN_APACHE='true' GROUPER_RUN_SHIB_SP='true' GROUPER_RUN_TOMEE='true' |
quickstart (v2.5.27+) | will set env vars (if not overridden): |
<no command> | do nothing, so GSH can be used in bash in container, or pass in ENV vars to run something not with command above |
-e GROUPER_UI=true | env var will tell grouper to allow ui calls via grouper.hibernate.base.properties grouper.is.ui.elConfig = ${java.lang.System.getenv().get('GROUPER_UI')} |
-e GROUPER_WS=true | env var will tell grouper to allow ws calls via grouper.hibernate.base.properties grouper.is.ws.elConfig = ${java.lang.System.getenv().get('GROUPER_WS')} |
-e GROUPER_SCIM=true | env var will tell grouper to allow ws calls via grouper.hibernate.base.properties grouper.is.scim.elConfig = ${java.lang.System.getenv().get('GROUPER_SCIM')} |
-e GROUPER_DAEMON=true | env var will tell grouper to kick of daemon thread in tomee grouper.is.daemon.elConfig = ${java.lang.System.getenv().get('GROUPER_DAEMON')} |
-e GROUPER_QUICKSTART=true (v2.5.28+) | env var will setup quickstart components |
-e GROUPER_RUN_APACHE=true | env var will tell supervisor to kick off apache in container. Note, apache is not needed for Grouper. You could hook up an external web server to tomee or run from tomee itself (not recommended) |
-e GROUPER_RUN_SHIB_SP=true (RUN_SHIB_SP up to v2.5.27) | env var will tell supervisor to kick off shib sp in container. Note if you dont use shib this is not needed. Note: you can also run shib outside the grouper container (e.g. in another container or from reverse proxy) Note: if RUN_SHIB_SP is false, it will take the shib apache directive out of grouper-www.conf |
-e GROUPER_RUN_TOMEE=true (RUN_TOMEE up to v2.5.27) | env var will tell supervisor to kick off tomee. Note you must have this to true if you are doing anything but a GSH env. The WS/UI/scim/daemon must run tomee in container. |
-e GROUPER_RUN_HSQLDB=true (v2.5.27+) | env var will tell supervisor to start hsqldb, storing data files to /opt/hsqldb and listening in container on port 9001 |
-e GROUPER_SELF_SIGNED_CERT=true (SELF_SIGNED_CERT up to v2.5.27) | will overlay /etc/httpd/conf.d/ssl-enabled.conf so that apache uses a self-signed cert for quick starts |
-e GROUPER_APACHE_SSL_PORT=443 (v2.5.28+) | will change the ssl port of apache. default is 443. |
-e GROUPER_USE_SSL=false (v2.5.28+) | if you do not want apache listening on 443 |
-e GROUPER_MAX_MEMORY='3g' | set memory of java to 3 gigs. recommended 2 or 3 gig for WS and UI, and 12gig for daemon # ps -ef | grep tom (get pid) # sudo -u tomcat /usr/lib/jvm/java-1.8.0-amazon-corretto/bin/jmap -heap <pid> (see max heap, should be approx what you expect) |
-e GROUPER_EXTRA_CATALINA_OPTS='-XX:+PrintGCDetails' | add additional JVM options |
-e CATALINA_OPTS='whatever' | Generally you should not set this, unless you want to override all the default tomee Grouper customizations |
-e GROUPER_AUTO_DDL_UPTOVERSION='v2.5.*' (v2.5.27+) | If you want Grouper to automatically install and update the database DDL when it starts up, and dont go to another minor version, anything for v2.5.* You can instead configure this in the grouper.hibernate.properties config file with key: registry.auto.ddl.upToVersion |
-e GROUPER_UI_CONFIGURATION_EDITOR_SOURCEIPADDRESSES=0.0.0.0/0 | Allow the configuration editor in the UI only from this IP address. Put in a cidr, or comma separated cidrs. Or open up and trust your authn/MFA and set to 0.0.0.0/0 You can instead configure this in the grouper-ui.properties config file with key: grouperUi.configurationEditor.sourceIpAddresses |
-e GROUPER_START_DELAY_SECONDS=10 (v.2.5.27+) | If you want Grouper to delay on startup, e.g. if waiting for the database to start You can instead configure this in the grouper.hibernate.properties config file with key: grouper.start.delay.seconds |
-e GROUPER_UI_GROUPER_AUTH=true (v.2.5.27+) | If you want to use built-in Grouper authentication for the UI before you integrate Grouper with your SSO You can instead configure this in the grouper.hibernate.properties config file with key: grouper.is.ui.basicAuthn |
-e GROUPER_WS_GROUPER_AUTH=true (v.2.5.27+) | If you want to use built-in Grouper authentication for the WS You can instead configure this in the grouper.hibernate.properties config file with key: grouper.is.ws.basicAuthn |
-e GROUPER_WS_TOMCAT_AUTHN=true (v.2.5.27+) | Will setup the /opt/grouper/grouperWebapp/WEB-INF/web.xml and /opt/tomee/conf/server.xml to use tomcat authentication for web services. Note you should consider using Grouper LDAP or built in authentication instead. |
-e GROUPER_SCIM_GROUPER_AUTH=true (v.2.5.27+) | If you want to use built-in Grouper authentication for SCIM You can instead configure this in the grouper.hibernate.properties config file with key: grouper.is.scim.basicAuthn |
-e GROUPER_MORPHSTRING_ENCRYPT_KEY_FILE=/a/b/c (v.2.5.28+) | Location of morphString encryption key You can instead configure this in the morphString.properties config file with key: encrypt.key |
-e GROUPER_MORPHSTRING_ENCRYPT_KEY=myUnsecureKey (v.2.5.27+) | morphString encryption key. Note, passwords in environment variables or Docker commands are security risks You can instead configure this in the morphString.properties config file with key: encrypt.key |
-e GROUPER_DATABASE_URL_FILE=/a/b/c (v.2.5.28+) | Location of the database jdbc url. # e.g. postgres (a): jdbc:postgresql://localhost:5432/database # e.g. postgres (b): jdbc:postgresql://localhost:5432/database?currentSchema=mySchema # e.g. mysql: jdbc:mysql://localhost:3306/grouper?useSSL=false # e.g. oracle: jdbc:oracle:thin:@server.school.edu:1521:sid # e.g. hsqldb (a): jdbc:hsqldb:dist/run/grouper;create=true # e.g. hsqldb (b): jdbc:hsqldb:hsql://localhost:9001/grouper # e.g. p6spy (log sql): [use the URL that your DB requires] You can instead configure this in the grouper.hibernate.properties config file with key: hibernate.connection.url |
-e GROUPER_DATABASE_URL='jdbc:postgresql://localhost:5432/database' (v.2.5.27+) | Database URL (if not provided from file) You can instead configure this in the grouper.hibernate.properties config file with key: hibernate.connection.url |
-e GROUPER_DATABASE_USERNAME_FILE=/a/b/c (v.2.5.28+) | Database username from file You can instead configure this in the grouper.hibernate.properties config file with key: hibernate.connection.username |
-e GROUPER_DATABASE_USERNAME=grouperSchema | Database username (if not provided by file) You can instead configure this in the grouper.hibernate.properties config file with key: hibernate.connection.username |
-e GROUPER_DATABASE_PASSWORD_FILE=/a/b/c (v.2.5.28+) | Database password, should be encrypted in file You can instead configure this in the grouper.hibernate.properties config file with key: hibernate.connection.password |
-e GROUPER_DATABASE_PASSWORD=myUnsecurePass (v.2.5.27+) | Database password (if not provided by file). Note, passwords in environment variables or Docker commands are security risks You can instead configure this in the grouper.hibernate.properties config file with key: hibernate.connection.password |
-e GROUPERSYSTEM_QUICKSTART_PASS=myUnsecurePass (v.2.5.27+) | If you are running the quickstart command on the container, and you set this env var, and you are doing grouper built in authentication in the UI and/or WS, then this password will log in GrouperSystem in the UI and/or WS. Note, this is a HUGE security problem if this is available in a production system. GrouperSystem should not be able to log in to Grouper via WS or UI. |
-e GROUPER_CHOWN_DIRS=false (v.2.5.27+) | If you do not want the container to chown dirs that it needs owned by certain users. If you are making a subimage, if you can RUN chown -R tomcat:tomcat /opt/grouper/grouperWebapp Then the startup of the image will be quicker (whether or not you pass in this variable). Docker subimages can COPY as root which negatively affects Grouper. |
-e GROUPER_LOG_TO_HOST=true (v.2.5.27+) | If you do not want Grouper to log to container pipes (for Maturity level 0) and you want Grouper to log to files and mount the /opt/grouper/logs dir to a host directory |
-e GROUPER_USE_GROUPER_CONTEXT=true | Just use /grouper as the tomcat context no matter what (see "tomcat contexts" below) |
-e GROUPERUI_LOGOUT_REDIRECTTOURL=/some/path (v2.5.28+) | Set to grouper-ui.properties: grouperUi.logout.redirectToUrl This will be set to /Shibboleth.sso/Logout if the shib env var is set |
-e GROUPER_LOG_PREFIX=grouper | Log prefix. By default it is "grouper-ui' for ui-only container. grouper-ws for ws-only. grouper-scim for scim-only. grouper-daemon for daemon-only. Or "grouper" if not set to something else. |
-e GROUPER_RUN_TOMCAT_NOT_SUPERVISOR=true (v2.5.28+) | Will run the tomee process as the only process in the container, not supervisor. Note, this is advanced, and should be run as the tomcat user |
Tomcat contexts
There is some complexity in tomcat contexts for a few reasons
- Only one webapp should be running at a time, do not put multiple context configs in /opt/tomcat/conf/Catalina/localhost
- If running multiple modules (e.g. ws/ui/daemon), those run in the same webapp
- If there is one module running, there should be a specific context config so Grouper knows the context (e.g. for SOAP WSDL), and can customize it (e.g. turn off cookies for WS/Scim)
- The apache grouper config should dynamically adjust based on which modules are running
- If you dont care about SOAP or cookies and just want everything in the same /grouper tomcat context, just set this env var: GROUPER_USE_GROUPER_CONTEXT=true
To customize the grouper*.xml tomcat config:
- You can overlay the /opt/tomee/conf/Catalina/localhost/grouper.xml, grouper-ws.xml, or grouper-ws-scim.xml
- Know that two will be deleted when Grouper starts based on which modules are running
- If only WS is running, then grouper-ws.xml will remain
- If only SCIM is running, then only grouper-ws-scim.xml will remain
- Otherwise grouper.xml will remain
To customize the /etc/httpd/conf.d/grouper-www.conf:
This is the base file now, use this as a template for overlays (look in container for most recent version)
Variables available
Grouper apache config variable Value Condition __THE_AJP_URL__ ajp://localhost:8009/grouper If the UI is running, or multiple modules (e.g. SCIM and WS) __THE_AJP_URL__ ajp://localhost:8009/grouper-ws If the WS is running and not UI or SCIM __THE_AJP_URL__ ajp://localhost:8009/grouper-ws-scim If SCIM is running and not WS or SCIM __GROUPER_PROXY_PASS__ <blank> If the UI is running __GROUPER_PROXY_PASS__ # (comment)
If the UI is not running __GROUPERWS_PROXY_PASS__ <blank> If the WS is running __GROUPERWS_PROXY_PASS__ # (comment)
If the WS is not running __GROUPERSCIM_PROXY_PASS__ <blank> If SCIM is running __GROUPERSCIM_PROXY_PASS__ # (comment)
If SCIM is not running - Use those variables in your overlay. You should be able to have one overlay for all your environments, though maybe you want separate subimages for each env
Dockerfile
If you create dirs or copy things to the webapp, you should set the owner at the end of your Dockerfile
RUN chown -R tomcat:tomcat /opt/grouper/grouperWebapp
Container startup workflow (v2.5.28+)
Custom overridable hooks in green
There are three main phases:
- Setup logging
- Setup environment variables. Note: generally these are only set if not already set so they can be customized
- Setup files
- Run Processes
Step | File | Description |
---|---|---|
Setup logging | ||
Start | /usr/local/bin/entrypoint.sh | Kick off the container |
prepConf() | /usr/local/bin/libraryPrep.sh | Setup logging, needs to be first |
Setup environment variables | ||
grouperPrepConfPost() | /usr/local/bin/grouperCustomShellHooks.sh | Custom hook called at the beginning of the container startup after logging is setup |
Built-in command | /usr/local/bin/daemon | Continue the workflow (below) by prepping the component, finish prep, setup files, and start supervisor |
prep command: prepUI() prepWS() etc | /usr/local/bin/setupPrep.sh | Setup each component, setting env variables |
prep "only" command: | /usr/local/bin/setupPrepOnly.sh | Set variables when certain components are the only thing runnings, can optimize e.g. if in UI only, or WS only E.g. set logging prefix and webapp context |
grouperPrepComponentPost() | /usr/local/bin/grouperCustomShellHooks.sh | Custom hook called after the component has been prepped |
Setup files | ||
setupFiles() | /usr/local/bin/librarySetupFiles.sh | Main logic that moves around files, generates files, massages files, substitutes files |
grouperSetupFilesPost() | /usr/local/bin/grouperCustomShellHooks.sh | Custom hook called after the setupFiles functions is called, right before the process starts |
Run processes |
Versions
- Tag in github docker is: 2.5.X where X is an integer that increases for each build
- There is a listing of each version in the Grouper 2.5 release notes, with an indication on if it is stable or not
- Every container will have "yum update" done before the container is released. As needed tomee will be updated
Misc
- All of your containers must have the correct time and consistent timezones
- HTTP Strict Transport Security (HSTS) is enabled on the Apache HTTP Server.
- morphStrings functionality in Grouper is supported. It is recommended that the various morphString files be associated with the containers as Docker Secrets. Set the configuration file properties to use `/run/secrets/secretname`.
- /run/secrets
If you have a secret available in /run/secrets/grouper_fileName.pass, it will be linked from /opt/grouper/grouperWebapp/WEB-INF/classes/fileName.pass (change fileName.pass name as needed)
If you have a secret available in /run/secrets/shib_fileName.pass, it will be linked from /etc/shibboleth/fileName.pass (change fileName.pass name as needed)
If you have a secret available in /run/secrets/httpd_fileName.pass, it will be linked from /etc/httpd/conf.d/fileName.pass (change fileName.pass name as needed)
- If you have a secret named /run/secrets/host-key.pem, it will be linked from /etc/pki/tls/private/host-key.pem
- Configure Grouper UI and WS authentication
Take a heap snapshot
[root@43c6b6e000b3 ~]# sudo -u tomcat /usr/lib/jvm/java-1.8.0-amazon-corretto/bin/jmap -dump:file=/tmp/memory.dump <pid>
See the env vars of a process, get the pid fist with ps-ef, then, run as the user (in this case tomcat) and substitute the pid in
[root@73e1e62a5fa1 bin]# sudo -u tomcat cat /proc/45/environ | tr '\0' '\n'
Jars
- If you want a jar in all JVMS (ui/ws/daemon/gsh/scim), add it to /opt/grouper/grouperWebapp/WEB-INF/lib
- If you want a jar in ui/daemon/gsh only, add it to /opt/grouper/grouperWebapp/WEB-INF/libUiAndDaemon
If you are replacing a jar (e.g. an existing driver), you need to remove it first from your dockerfile by wildcard, or overlay a blank file (risky since filenames can change)
RUN rm -rf /opt/grouper/grouperWebapp/WEB-INF/lib/mysql-connector-java*.jar
- dfs