How To Setup a Grouper Development Environment

This how-to describes how to set up a Grouper development environment so that you can run, test, and debug the Grouper project.

The example commands and screenshots are from MacOS and may vary slightly for different environments. However, the overall process should be similar on any modern operating system and development tool chain.

Prerequisites

Git for source code version control

1. Install Git

   1. $ brew install git

   2. https://github.com/fabriziocucci/git-bash-for-mac

   3. Or Install from package https://git-scm.com/downloads

2. Github Desktop is also handy

Java - Grouper runs on Java

1. Install OpenJDK 8 or above. Grouper runs on Java.

   1. https://aws.amazon.com/corretto/

Apache Tomcat - Grouper runs in Tomcat

1. Download and unpack Tomcat 8.5.x
1. https://tomcat.apache.org/download-80.cgi

Docker Desktop

1. Install Docker Desktop. We’ll use this to run our development database.
1. https://www.docker.com/products/docker-desktop

Eclipse - Grouper development happens in Eclipse

1. Install Eclipse IDE for Enterprise Java Developers or similar IDE
1. https://www.eclipse.org/downloads/packages/

Clone the Grouper Source Repository

The Grouper code repository is available on https://github.com/Internet2/grouper

1. git clone --depth 1 https://github.com/Internet2/grouper.git grouper-2.5

It is recommend not to switch between 2.4 and 2.5 branches in the same directory and workspace when developing since it makes keeping development environment straight difficult. Instead keep 2.4 and 2.5 work in separate local repos each with their own eclipse workspace.

To developer on 2.4

1. git clone --depth 1 https://github.com/Internet2/grouper.git grouper-2.4
2. git checkout GROUPER_2_4_BRANCH

Import Projects into Eclipse

Start a new eclipse workspace and import grouper modules as individual projects. Project will import as Maven projects and automatically download the required Maven dependencies. The example commands below assume the git repository was cloned to the local directory '/Users/thompsow/src/grouper-2.5/'.

1. Import grouperClient
1. Eclipse -> File -> Open Projects for File System or Archive
2. Import source: /Users/thompsow/src/grouper-2.5/grouper-misc/grouperClient
2. Import the grouper
   1. Eclipse -> File -> Open Projects for File System or Archive
   2. Import source: /Users/thompsow/src/grouper-2.5/grouper
3. Import grouper-ui
1. Eclipse -> File -> Open Projects for File System or Archive
2. Import source: /Users/thompsow/src/grouper/grouper-ui
4. Import grouper-ws
   1. Eclipse -> File -> Open Projects for File System or Archive
   2. Import source: /Users/thompsow/src/grouper/grouper-ws/grouper-ws

All the project should now be open and compiled.

Link source and conf to grouper-ui project

We want to be able to run and debug the Grouper UI from the grouper-ui/webapp folder, so that we can work on webapp artifacts (JSPs, etc), and at the same time update Java code in the grouper project and other code locations. To do this we will update the Java Build Path output folder so that compiled classes and other artifacts go to the right directories under grouper-ui/webapp. We will also add some dependent source and library folders to the grouper-ui Java Build Path.

1. grouper-ui -> File -> Properties -> Java Build Path -> Source tab
1. Set ‘Default output folder:’ to: grouper-ui/webapp/WEB-INF/classes (you may have to create this directory)
2. Add dependent source and configuration folders to the Java Build Path
   1. Click ‘Link Source…’
   2. Click ‘Variables…’
   3. Click ‘New…’
   1. Name: GROUPER_REPO
   2. Click ‘Folder…’ and select the root repo folder
   4. Select GROUPER_REPO
   5. Click ‘Extend…’
   6. Select ‘grouper/conf’
   7. Folder name: grouper-conf
   1. Click ‘Link Source…’
   2. Click ‘Variables…’
   3. Select GROUPER_REPO
   4. Click ‘Extend…’
   5. Select ‘grouper/src/grouper’
   6. Folder name: grouper-src
   1. Click ‘Link Source…’
   2. Click ‘Variables…’
   3. Select GROUPER_REPO
   4. Click ‘Extend…’
   5. Select ‘grouper-misc/grouperClient/src/java
   6. Folder name: grouperClient-src
   1. Click ‘Link Source…’
   2. Click ‘Variables…’
   3. Select GROUPER_REPO
   4. Click ‘Extend…’
   5. Select ‘grouper-misc/grouperClient/conf’
   6. Folder name: grouperClient-conf
1. Grouper Configuration
2. Grouper source
3. GrouperClient source
4. GrouperClient configuration
3. Make sure all folders except ‘grouper-ui/java/test’ are using the default output folder
4. Make sure grouper-ui/conf has ‘Excluded:’ set to (None)
5. Remove srcPoc and misc source folders (these are not needed)

The grouper-ui Java Build Path should now look something like this:

Add runtime jars to grouper-ui Java Build Path

We need to pull dependent jar files into grouper-ui/webapp/WEB-INF/lib directory. We’ll do this by adding three more “source” folders to the grouper-ui Java Build Path.

1. grouper-ui -> File -> Properties -> Java Build Path
   1. Link Source…
   1. Linked folder location: GROUPER_REPO/grouper/lib/grouper
   2. Folder name: grouper-lib
   2. Configure grouper-lib source to output to path relative to ‘grouper-ui’
      1. Check “Allow output folders for source folders
      2. Click “Output folder” on grouper-lib
      3. Select ‘Specific output folder (path relative to ‘grouper-ui’)
         1. create/select ‘webapp/WEB-INF/lib’
            
1. Add grouper jars
2. Add jdbc sample jars
   1. Link Source…
      1. Linked folder location: /Users/thompsow/src/grouper/grouper/lib/jdbcSamples
      2. Folder name: jdbcSamples-lib
      3. Configure grouper-lib source to output to path relative to ‘grouper-ui’
         1. webapp/WEB-INF/lib
3. Add grouper-ui jars
   1. Add Folder…
      1. Select java/lib
      2. Check “Allow output folder for source folders
      3. Configure grouper-lib source to output to path relative to ‘grouper-ui’
         1. webapp/WEB-INF/lib
      4. Add javax.servlet.jsp-api.{version}.jar to Exclusion patterns
         

The grouper-ui Java Build Path should now look something like this:

Configure Eclipse Code Formatter

1. Eclipse -> Preferences -> Java Code Style -> Formatter
2. Import… (navigate and select ../grouper/grouper/misc/eclipse/fastFormat.xml)
3. Apply and close

Development Database

Multiple databases are supported including Oracle, mySQL, and PostgreSQL. We’ll use PostgreSQL for this how-to. The steps for other databases would be similar.

Start the development database

We will run postgres with a mounted external volume to preserve data between docker container restarts.

1. Create directory ‘$HOME/docker/volumes/postgres’
2. Run ‘docker run --name grouperdb -e POSTGRES_PASSWORD=grouper -e POSTGRES_USER=grouper -d -p 5432:5432 -v $HOME/docker/volumes/postgres:/var/lib/postgresql/data postgres’

Connect to development database in Eclipse

1. Window -> Perspective -> Open Perspective -> Other -> Database Development -> Open
2. Right click ‘Database Connections’ and create PostgreSQL connection profile

3. Add the postgess JDBC driver found in under ../grouper/grouper-ui/webapp/WEB-INF/lib/

4. Enter the connection details and then click Test Connection

Configure minimum properties files for development

Create and configure the grouper-hibernate.properties for postgres

1. cd ../grouper/grouper/conf
2. cp ../misc/grouper.hibernate.example.properties grouper.hibernate.properties  
3. Edit ../grouper/grouper/conf/grouper.hibernate.properties

Create and configure morphString.properties

1. $ cp ../misc/morphString.example.properties morphString.properties  
2. Edit ../grouper/grouper/conf/morphString.properties
3. encrypt.key = not_a_random_key

Create properties files

1. cd ../grouper/grouper/conf
2. $ cp log4j.example.properties log4j.properties
3. $ cp ../misc/grouper.example.properties grouper.properties
4. $ cp ../misc/grouper.cache.example.properties grouper.cache.properties
5. $ cp ../misc/subject.example.properties subject.properties
6. ...argh have to copy all the *.example.properties to *.properties how to handle this?
7. $ cp grouperText/grouper.textNg.en.us.base.properties grouperText/grouper.text.en.us.properties

Bootstrap the Grouper Database

For development purposes, we’ll bootstrap the Grouper database, add sample subjects, and reset the database using a few Java classes. To initialize the Grouper database we will run the GrouperShell class with arguments “-registry -runscript -noprompt”.

Run GrouperShell from Eclipse to initialize the Grouper database:

1. Right click GrouperShell in the grouper project explorer
2. Select ‘Run as’ and then ‘Run Configurations…’
1. Name: GrouperShell -registry -runscript -noprompt
3. (x) = Arguments tab
1. Program arguments:
1. -registry -runscript -noprompt
4. Classpath tab
1. Highlight User Entries and click ‘Advanced…’
2. Add the ‘conf’ folder to the classpath for the properties files
5. Click ‘Apply’
6. Click ‘Run’

Run GrouperShell from Eclipse to check the Grouper database:

1. Right click GrouperShell in grouper project explorer
2. Select ‘Run as’ and then ‘Run Configurations…’
1. Name: GrouperShell -registry -check -noprompt
3. (x) = Arguments tab
1. Program arguments:
1. -registry -check -noprompt
4. Classpath tab
1. Highlight User Entries and click ‘Advanced…’
2. Add the ‘conf’ folder to the classpath
5. Click ‘Apply’
6. Click ‘Run’

Run RegistryReset with ‘justAddSubjects’ as an argument to add sample subjects:

1. Right click RegistryRest in grouper project explorer
2. Select ‘Run as’ and then ‘Run Configurations…’
1. Name: RegistryReset justAddSubjects
3. (x) = Arguments tab
1. Program arguments:
   1. justAddSubjects
4. Classpath tab
5. Highlight User Entries and click ‘Advanced…’
   1. Add the ‘conf’ folder to the classpath
   2. Click ‘Apply’
6. Click ‘Run’

Query the subjects table from the Eclipse Data Source Explorer to see the added subjects:

Run grouper-ui in Eclipse with Tomcat

Now that we have a Grouper database and some test subjects, the next step is to add the grouper-ui/webapp directory to the Eclipse Tomcat launcher so we can run and debug the grouper-ui.

Add Tomcat server to Eclipse:

1. Eclipse -> J2EE Perspective -> Servers Tab
2. Click “No servers are available. Click this link to create a new server…”
   1. Select 'Tomcat v8.5'
   2. Select your tomcat installation directory
   3. Select your JRE

Add grouper-ui web module to Tomcat Server

1. Double click on Tomcat v8.5 at localhost [Stopped, Republish] to access configuration panel
2. Click on ‘Modules’ tab, click ‘Add External Web Module…”

Configure Server Location

1. Select ‘Use Tomcat installation (takes control Tomcat installation)

Configure Tomcat Server Working Directory to direct Grouper logs

1. Servers tab -> Overview -> click on “Open launch configuration”
2. (x)= Arguments tab
3. Working directory:
1. Choose Other
2. Click Workspace…
1. Select grouper-ui/webapp/WEB-INF

Grouper logs will now show up under ../grouper-ui/webapp/WEB-INF/logs

Create Grouper Users in tomcat-user.xml

Add GrouperSystem and test.subject.0 through 9 to tomcat-user.xml

1. Add the following to ../apache-tomcat-8.5.50/conf/tomcat-users.xml 

<role rolename="grouper_user"/>
<user username="GrouperSystem" password="" roles="grouper_user"/>
<user username="test.subject.0" password="" roles="grouper_user"/>
<user username="test.subject.1" password="" roles="grouper_user"/>
<user username="test.subject.2" password="" roles="grouper_user"/>
<user username="test.subject.3" password="" roles="grouper_user"/>
<user username="test.subject.4" password="" roles="grouper_user"/>
<user username="test.subject.5" password="" roles="grouper_user"/>
<user username="test.subject.6" password="" roles="grouper_user"/>
<user username="test.subject.7" password="" roles="grouper_user"/>
<user username="test.subject.8" password="" roles="grouper_user"/>
<user username="test.subject.9" password="" roles="grouper_user"/>

Now start Tomcat from Eclipse by selecting the server under the Servers tab and clicking the green ‘Run’ button.