...
- Proof of concept running daemon/ui/ws in one directory
- Build the Grouper webapp with WS and UI in one webapp.
- Put all libs in there, all configs, and merge the web.xml's
- Run the UI and WS
- Run the daemon, it should work, but just make sure
- Add the scim stuff to that one directory. Can tomcat read it for WS/UI/daemon, and tomee read it for SCIM? Maybe need dynamic web.xml (only loads scim servlet if env var is set or something)?
- Remove web.xml, do basic auth in filter, look at kerberos WsGrouperKerberosAuthentication
- Simplify builds of projects
- No more tarballs
- Remove jar only of installer (will be tarball)
- Merge installer with client?
- Disable installer if in container for security reasons?
- Each project in git should have all dependent jars and a simple (similar) pom.xml to build the project and optionally the tarball
- Maven will be used to build jars and publish to maven
- Remove extraneous stuff (almost all stuff) from build.xml
- Installer when installing should not need to run any builds from maven, it should just download files from maven and copy files around
- UI and WS should have dynamic web.xml configuration
- put a config item in grouper hibernate properties (since its not in database), with three switches
- grouper.is.ui = true|false
- grouper.is.ui.basicAuthn = true|false
- grouper.is.ws = true|false
- grouper.is.ws.basicAuthn = true|false
- grouper.is.scim = true|false
- grouper.is.scim.basicAuthn = true|false
- grouper.is.daemon = true|false
- Blank out the web.xml in UI and WS except for the listener or whatever minimal is required. Both should be the exact same
- Based on that config in grouper hibernate properties, can the web.xml be dynamically configured to add the servlets and filters and security constraints based on config?
- The basic auth security constraint can be added from grouper-ui.properties or grouper-ws.properties, and be selective for if ui or ws
- Make sure Grouper fails in any UI call (in existing filter?) if grouper.is.ui is false
- Make sure Grouper fails in any WS call (in existing filter?) if grouper.is.ws is false
- Have the daemon fail (in startup method) if grouper.is.daemon is false
- Have a grouper.properties switch that defaults to true (throw exceptions), that says "fail if servlet class not found", dont load classes (try/catch)
- put a config item in grouper hibernate properties (since its not in database), with three switches
- T
he UI/WS/scim should check the url to make sure its ok (based on configuration which defaults to be strict)configuration to see if this feature is on, default to truee.g. /grouper/servicesRest and /grouper/v2 and /grouper/services should be an error (/grouper is configurable in the grouper-ui config)e.g. /grouper-ws/grouperUi and /grouper-ws/v2 should be an error (/grouperWs is configurable in the grouper-ws config)e.g. /grouper-ws-scim/grouperUi and /grouper-ws-scim/services and /grouper-ws-scim/servicesRest should be an error (/grouper-ws-scim is configurable in the grouper-ws config)
Apache will do this and invalid requests wont be forwardedCode Block ReverseProxy 443 /grouper/grouperUi -> 8080 /grouper/grouperUi ReverseProxy 443 /grouper/status -> 8080 /grouper/status ReverseProxy 443 /grouper-ws/status -> 8080 /grouper/status ReverseProxy 443 /grouper-ws/servicesRest -> 8080 /grouper/servicesRest ReverseProxy 443 /grouper-ws/services -> 8080 /grouper/services ReverseProxy 443 /grouper-ws-scim/v2 -> 8080 /grouper/v2 ReverseProxy 443 /grouper-ws-scim/status -> 8080 /grouper/status
TODO (after 2.5), allow daemon nodes to have HTTP status servlet
- The daemon (if the daemon switch is on), should run inside of tomee, not only from: gsh.sh -loader
- Note, the loader should still be able to be kicked off from gsh.sh -loader, though we wont do that in the container
- In the J2EE config, see if hibernate properties is loader, and if so, run GrouperLoader.main(new String[]{}), in a new thread
- Update the installer (discuss with Chris first) to build container (grouper devs or chris hubing)
- It should ask similar questions, but without redundancies (few questions)
- Assume everything is checked out from git
- Find version in local filesystem (figure that out)
- Download grouper and 3rd party jars from maven (use ant-maven plugin or maven command line to get maven jars from an ant script)
- Build these uber webapp
- Container changes
- Remove the morph config file, and read morph secret from grouper hibernate properties instead. Note it can also be in the grouper.client.properties (as it is now). Cant be in multiple places if same (Hyzer)
- The grouper hibernate properties is the one file not in the database, and has everything needed to start grouper (Hyzer)
- Be able to run the container by mounting the conf directory from a mounted external directory (Hubing)
- This is a directory on the server outside of the container (Hubing)
- The default container should have configs that will work without mounting externally (Hubing)
- Should have 8 variables for morph secret, database user/url/pass, and if ui/ws/daemon, log directory base (can be mounted outside)
- The container should take empty directory /slashRoot and copy the contents onto "/", and the slashRoot should be able to be mounted externally (Hubing)
- Maturity level 0 is just start container with some switches (NA)
- Level 1 is mount external conf/ dir and go from there (log4j should be able to log to externally mounted directory) (Hyzer)
- Do a POC on RHEL 8 with podman: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html-single/building_running_and_managing_containers/index#running-containers-without-docker_building-running-and-managing-containers (Later)
- Remove the morph config file, and read morph secret from grouper hibernate properties instead. Note it can also be in the grouper.client.properties (as it is now). Cant be in multiple places if same (Hyzer)
- Installer mode for installing container
- Ask questions similar to current install. Mode is installContainer just like buildContainer in GrouperInstaller.java.
- Suggest latest container. Get the version out of META-INF and if not from there, get it from the property file.
- Detect if docker installed and if yes, just print out that docker is installed and if not, print the instructions to setup docker. they are at: Install the Grouper 2.4 container with maturity level 0
- ask user to hit enter when done. On enter, confirm that docker is installed.
- If RHEL 8 (or whatever), use "podman" as a substitute (Later)
For the following, print the first three commands and users should run them if docker not already installed and running. To see if it's running, run `/sbin/service docker status`. try/catch everything and keep going. For the fourth command, grouperInstaller should just run it. throw exception if failed but keep going.
[root
@ip
-
172
-
30
-
0
-
157
~]# /sbin/service docker start
Redirecting to /bin/systemctl start docker.service
[root
@ip
-
172
-
30
-
0
-
157
~]# systemctl enable docker
[root
@ip
-
172
-
30
-
0
-
157
~]# docker pull tier/grouper:
2.5.x
- Ask user where the external mounted directory is and the default should be /opt/grouperMount
- Ask about database
Make a grouper.hibernate.properties in the path of wherever the external mounted directory is </opt/grouperMount>/conf/grouper.hibernate.properties
Code Block ######################################## ## DB settings ######################################## # e.g. mysql: jdbc:mysql://localhost:3306/grouper?useSSL=false # e.g. p6spy (log sql): [use the URL that your DB requires] # e.g. oracle: jdbc:oracle:thin:@server.school.edu:1521:sid # e.g. hsqldb (a): jdbc:hsqldb:dist/run/grouper;create=true # e.g. hsqldb (b): jdbc:hsqldb:hsql://localhost:9001/grouper # e.g. postgres: jdbc:postgresql://localhost:5432/database # e.g. mssql: jdbc:sqlserver://localhost:3280;databaseName=grouper hibernate.connection.url = jdbc:hsqldb:hsql://localhost:9001/grouper hibernate.connection.username = sa # If you are using an empty password, depending upon your version of # Java and Ant you may need to specify a password of "". # Note: you can keep passwords external and encrypted: https://bugs.internet2.edu/jira/browse/GRP-122 hibernate.connection.password = <MORPH_ENCRYPTED_PASSWORD> ####################################### grouper.is.ui=${java.lang.System.getenv().get('GROUPER_UI')} grouper.is.ws=${java.lang.System.getenv().get('GROUPER_WS')} grouper.is.scim==${java.lang.System.getenv().get('GROUPER_SCIM')} grouper.is.daemon==${java.lang.System.getenv().get('GROUPER_DAEMON')} grouper.is.ui.basicAuthn=true grouper.is.ws.basicAuthn=true grouper.is.scim.basicAuthn=true
- Ask for a morphString password and make a morphString.properties: </opt/grouperMount>/conf/morphString.properties
Ask about initting it (do you want to init it?) (add this in the README.txt), installer doing this on the user's behalf)
Code Block docker run --detach \ --mount type=bind,src=</opt/grouperMount>/conf,dst=/opt/grouper/conf \ --mount type=bind,src=</opt/grouperMount>/logs,dst=/opt/grouper/logs \ --mount type=bind,src=</opt/grouperMount>/slashRoot=/opt/grouper/slashRoot \ --name gsh \ --entrypoint /opt/grouper/grouperWebapp/WEB-INF/bin/gsh \ tier/grouper:2.5.x \ -registry -check -runscript -noprompt
- Ask about GrouperSystem password and then save it in the grouper_password table via that Authentication class we wrote. Refer to
addGrouperSystemWsGroup in GrouperInstaller.java. Make gsh script file which is inside the container in /opt/grouperMount/slashRoot/opt/grouper/grouperWebapp/WEB-INF/bin/createGrouperSystemPassword.gsh.
Code Block docker run --detach \ --mount type=bind,src=</opt/grouperMount>/conf,dst=/opt/grouper/conf \ --mount type=bind,src=</opt/grouperMount>/logs,dst=/opt/grouper/logs \ --mount type=bind,src=</opt/grouperMount>/slashRoot=/opt/grouper/slashRoot \ --name gsh \ --entrypoint /opt/grouper/grouperWebapp/WEB-INF/bin/gsh \ tier/grouper:2.5.x \ /opt/grouper/grouperWebapp/WEB-INF/bin/createGrouperSystemPassword.gsh
- Ask about external apache (Later)
- Ask if want Dockerfile changes (maturity level 2?) (Later)
There are a few arguments you can pass to the container, and env vars... Note the command if specified (optional) will set env vars before the env vars. So you could call the container with "ui" but then specify that -e RUN_SHIB_SP='false' (e.g. if you run CAS)
Argument Description ui will set env vars:
GROUPER_UI='true'
RUN_APACHE='true'
RUN_SHIB_SP='true'
RUN_TOMEE='true'ws will set env vars:
GROUPER_WS='true'
RUN_APACHE='true'
RUN_TOMEE='true'scim will set env vars:
GROUPER_SCIM='true'
RUN_APACHE='true'
RUN_TOMEE='true'daemon will set env vars:
GROUPER_DAEMON='true'
RUN_TOMEE='true'bin/gsh will just run gsh commands from docker command line ui-ws will set env vars:
GROUPER_UI='true'
GROUPER_WS='true'
RUN_APACHE='true'
RUN_SHIB_SP='true'
RUN_TOMEE='true'<no command> run loader? maybe do nothing instead, so GSH can be used in bash in container -e GROUPER_UI=true env var will tell grouper to allow ui calls via grouper.hibernate.base.properties
grouper.is.ui.elConfig = ${java.lang.System.getenv().get('GROUPER_UI')}-e GROUPER_WS=true env var will tell grouper to allow ws calls via grouper.hibernate.base.properties
grouper.is.ws.elConfig = ${java.lang.System.getenv().get('GROUPER_WS')}-e GROUPER_SCIM=true env var will tell grouper to allow ws calls via grouper.hibernate.base.properties
grouper.is.scim.elConfig = ${java.lang.System.getenv().get('GROUPER_SCIM')}-e GROUPER_DAEMON=true env var will tell grouper to kick of daemon thread in tomee
grouper.is.daemon.elConfig = ${java.lang.System.getenv().get('GROUPER_DAEMON')}-e RUN_APACHE=true env var will tell supervisor to kick off apache in container. Note, apache is not needed
for Grouper. You could hook up an external web server to tomee or run from tomee itself (not recommended)
-e RUN_SHIB_SP=true env var will tell supervisor to kick off shib sp in container. Note if you dont use shib this is not needed.
Note: you can also run shib outside the grouper container (e.g. in another container or from reverse proxy)
-e RUN_TOMEE=true env var will tell supervisor to kick off tomee. Note you must have this to true if you are doing anything
but a GSH env. The WS/UI/scim/daemon must run tomee in container.
Start the container
Code Block docker run --detach --publish 443:443 \ --mount type=bind,src=</opt/grouperMount>/conf,dst=/opt/grouper/conf \ --mount type=bind,src=</opt/grouperMount>/logs,dst=/opt/grouper/logs \ --mount type=bind,src=</opt/grouperMount>/slashRoot=/opt/grouper/slashRoot \ --restart always --name grouper \ tier/grouper:2.5.x \ -e GROUPER_UI='true' -e GROUPER_WS='true' -e GROUPER_DAEMON='true' -e GROUPER_SCIM='true' \ -e RUN_APACHE='true' -e RUN_SHIB_SP='false' -e RUN_TOMEE='true'
- Make a README.txt with some commands for and put it in /opt/grouperMount. Refer here: Install the Grouper 2.4 container with maturity level 0
Starting container
- Stopping container
- SSH'ing into it
- Where the ports/conf/logs are
- How to upgrade (i.e. go to site that lists TAP containers, update the command based on version, follow upgrade steps to go up major version)
- Chad to make maven jar versions of build number: grouper-2.5.8.jar
- After build of uber webapp, we should keep lists of files and filesizes so we can know what overlays are there... keep in a properties file in /WEB-INF/classes
- Add a UI screen for disagnostics that reports on file overlays...
- Remove the GrouperCheckConfig checking of jars
...