Child pages
  • Grouper Hosted on a Cloud Server
Skip to end of metadata
Go to start of metadata

This is an exercise to see how an Amazon EC2 cloud server can host Grouper. Amazon has inexpensive (~$500/year) dedicated virtual servers that can be provisioned quickly. The ability to run Grouper on one of these servers could help save money for institutions or even make it feasible for other institutions that might not have the server space or resources to manage their own Grouper server. There are high availability options and clustering, though this study assumes the default server configuration.

There are issues with running Grouper on a cloud server. Grouper might contain sensitive (HIPAA or FERPA protected) data, and it might be against your institution's policy to outsource hardware for such data. Also, if the server is located offsite (in Amazon's network), or if you need secure connections (e.g., for your person source), you will need to figure out how to make that happen. E.g. with mysql over SSL or LDAP over SSL or secure oracle connections.

In any event, if you decide you want to run Grouper on cloud infrastructure, here are instructions to make it happen.

Note: you should google all the software to get the latest and greatest since the versions here will be stale soon.

Note: this documentation was posted by a developer, not a sysadmin, there are better ways to secure this software, this is one way to do it, but not necessarily the recommended way. (smile)

Provision your server

  • Sign up for amazon EC2
  • Wait until you are approved (up to 12 hours?)
  • Provision a basic redhat server: Basic Fedora Core 8 Minimal Fedora Core 8, 32-bit architecture, and Amazon EC2 AMI Tools
  • Save the private key in a safe place (e.g. truecrypt volume)
  • Download putty
  • Convert the private key to a putty key
  • Login to the server (public DNS address), login as root, use the putty key for authentication
  • Add a user to run apps:
# useradd appadmin
  •  

Install Apache

  • Download files:
# cd /opt
# mkdir software
# cd software/
# wget http://mirror.candidhosting.com/pub/apache/httpd/httpd-2.2.14.tar.gz

  • Note, there are better ways to secure apache, this is an easy way to get started.  You will probably want to lock it down properly
  • Compile
[httpd-2.2.14]# yum install gcc
[httpd-2.2.14]# yum install *openldap* -y
[httpd-2.2.14]# yum install *zlib* -y                 [note: this installs extraneous packages I think...]
[httpd-2.2.14]# yum install openssl-devel -y
[httpd-2.2.14]# ./configure --prefix=/opt/apache --enable-ssl --enable-cgi --enable-so --enable-shared --enable-mods-shared=all --with-ldap --enable-authnz-ldap --enable-ldap
[httpd-2.2.14]# make
[httpd-2.2.14]# make install
[opt]# chown -R appadmin.appadmin /opt/apache
  • Register domain and SSL (I used godady, it was $12 for the SSL and $2 for the domain for one year)
Go to godaddy.com
Register the SSL and domain name: ec2testhyzer.info (make sure it is not auto renew if this is temporary)
Go to godaddy total dns control, and set the a record to the ec2 ip address
Manage SSL certificates, use the credit you just bought
Manage certificate, request certificate
   (go back to server, make a csr)
[opt]# mkdir cert
[opt]# cd cert
[cert]# openssl genrsa -out ec2testhyzer_info.key 2048
[cert]# openssl req -new -key ec2testhyzer_info.key -out ec2testhyzer_info.csr
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [GB]:US
State or Province Name (full name) [Berkshire]:PA
Locality Name (eg, city) [Newbury]:Media
Organization Name (eg, company) [My Company Ltd]:ec2testhyzer.info
Organizational Unit Name (eg, section) []:ec2testhyzer.info
Common Name (eg, your name or your server's hostname) []:ec2testhyzer.info
Email Address []:dont@email.me

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:
[cert]#

copy the contents of the CSR file and paste to godaddy
generate the certificate
download the certificate
upload to ec2 with psftp
unzip it

psftp> cd /opt/cert
psftp> put c:\temp\ec2testhyzer.info.zip

[cert]# unzip ec2testhyzer.info.zip


[apache]# cd /opt/apache/conf
[conf]# yum install emacs
[conf]# emacs httpd.conf

User appadmin
Group appadmin

Include conf/extra/httpd-ssl.conf

[conf]# emacs extra/httpd-ssl.conf

ServerName ec2testhyzer.info

SSLCertificateFile "/opt/cert/ec2testhyzer.info.crt"

SSLCertificateKeyFile "/opt/cert/ec2testhyzer_info.key"

SSLCertificateChainFile "/opt/cert/gd_bundle.crt"

[init.d]# emacs /etc/init.d/httpd

#!/bin/sh
#
# Startup script for Apache
#
# chkconfig: - 85 15
# description: Apache
# processname:
# pidfile:
# config:
# Tomcat
# description: Starts and stops Apache
# See how we were called.

apachectl=/opt/apache/bin/apachectl
RETVAL=0


case "$1" in
  start)
        $apachectl start
        ;;
  stop|restart|graceful|help|configtest|fullstatus|start)
        $apachectl $@
        RETVAL=$?
        ;;
  *)
        echo "Usage: $0 {start|stop|restart|configtest|help|fullstatus|start|graceful}"
        exit 1
esac

[init.d]# chkconfig --add httpd
[init.d]# chkconfig --levels 345 httpd on
[init.d]# chmod +x httpd
[init.d]# service httpd start

Test in a browser: https://ec2testhyzer.info/



Java

Download:
[init.d]# cd /opt/software/

NOTE: this next link wont work since it is session specific.  Google the java jdk and get the link

[software]# wget http://cds.sun.com/is-bin/INTERSHOP.enfinity/WFS/CDS-CDS_Developer-Site/en_US/-/USD/VerifyItem-Start/jdk-6u18-linux-i586.bin?BundledLineItemUUID=qcRIBe.lWGwAAAEmb2kRH41T&OrderID=mL9IBe.lgiQAAAEmZGkRH41T&ProductID=6XdIBe.pudAAAAElYStRSbJV&FileName=/jdk-6u18-linux-i586.bin
[software]# mv jdk-6u18-linux-i586.bin\?AuthParam\=1265091604_6c594bd3bb701613a5c20291632bd64a\&TicketId\=B%2Fw4lRWBSFNCTRNFM19TlQ%2Fm\&GroupName\=CDS\&FilePath\=%2FESD6%2FJSCDL%2Fjdk%2F6u18-b07%2Fjdk-6u18-linux-i586.bin\&File\=jdk-6u18-linux-i586.bin jdk-6u18-linux-i586.bin
[software]# chmod +x jdk-6u18-linux-i586.bin
[software]# ./jdk-6u18-linux-i586.bin
[software]# mv jdk1.6.0_18 ..
[software]# cd ..
[opt]# ln -s jdk1.6.0_18 java
[opt]# chown -R appadmin.appadmin jdk1.6.0_18


Ant

 [opt]# cd software/
[software]# wget http://archive.apache.org/dist/ant/binaries/apache-ant-1.7.1-bin.tar.gz
[software]# tar xzvf apache-ant-1.7.1-bin.tar.gz
[software]# mv apache-ant-1.7.1 ..
[software]# cd ..
[opt]# ln -s apache-ant-1.7.1 ant
[opt]# chown -R appadmin.appadmin apache-ant-1.7.1

[opt]# emacs /etc/profile


export JAVA_HOME=/opt/java
export ANT_HOME=/opt/ant

PATH=$JAVA_HOME/bin:$ANT_HOME/bin:$PATH

Tomcat

 [opt]# cd software/
[software]# wget http://ftp.wayne.edu/apache/tomcat/tomcat-6/v6.0.24/bin/apache-tomcat-6.0.24.tar.gz
[software]# tar xzvf apache-tomcat-6.0.24.tar.gz
[software]# mv apache-tomcat-6.0.24 ..
[software]# cd ..
[opt]# ln -s apache-tomcat-6.0.24 tomcat6
[opt]# chown -R appadmin.appadmin apache-tomcat-6.0.24
[opt]# ln -s java javaTomcat6
[opt]# cd /etc/init.d/
[init.d]# emacs tomcat6

#!/bin/sh
#
# Startup script for the Tomcat Server
#
# chkconfig: - 86 14
# description: Tomcat
# processname:
# pidfile:
# config:
# Tomcat
# description: Starts and stops the Tomcat
# See how we were called.

export CATALINA_BASE="/opt/tomcat6"
export TOMCAT_NAME="tomcat6"
export JAVA_HOME="/opt/javaTomcat6"
export JAVA_OPTS="-server -Xms5M -Xmx400M -XX:MaxPermSize=120M"

export TOMCAT_HOME="/opt/tomcat6"
export CATALINA_HOME="/opt/tomcat6"

export PATH="$PATH:$JAVA_HOME/bin:$GROOVY_HOME/bin"
export TOMCAT_USER="appadmin"

echo

cd $CATALINA_BASE

case "$1" in
  start)
    echo -n "Starting $TOMCAT_NAME Tomcat services: "
    echo
    if [ "$UID" -ne "0" ]; then
      $TOMCAT_HOME/bin/startup.sh
    else
      sudo -u $TOMCAT_USER $TOMCAT_HOME/bin/startup.sh
    fi
    echo
    ;;
  stop)
    echo -n "Shutting down $TOMCAT_NAME Tomcat services: "
    echo
    if [ "$UID" -ne "0" ]; then
      $TOMCAT_HOME/bin/shutdown.sh
    else
      sudo -u $TOMCAT_USER $TOMCAT_HOME/bin/shutdown.sh
    fi

    #loop until the program is dead
    waitingForExit=true
    iterator=1

    while [ "$waitingForExit" == "true" ]; do
      processId=`ps -ef | grep "$JAVA_HOME/bin/java" | grep -v "grep" | cut -c10-15`
      if [ -n "$processId" ]; then
        echo Waiting for exit...
      else
        waitingForExit=false
      fi

      if [ "$iterator" -gt "10" ]; then
        waitingForExit=false
      fi

      let "iterator += 1"
      sleep 1
    done


    #doesnt work since it kills are javas
    #killall -e $JAVA_HOME/bin/java
    killAgain=true
    while [ "$killAgain" == "true" ]; do
      processId=`ps -ef | grep "$JAVA_HOME/bin/java" | grep -v "grep" | cut -c10-15`
      if [ -n "$processId" ]; then
        echo "Killing the tomcat abruptly"

        if [ "$UID" -ne "0" ]; then
          kill -9 $processId
        else
          sudo -u $TOMCAT_USER kill -9 $processId
        fi

        sleep 1
      else
        killAgain=false
      fi
    done

    echo
    ;;
  status)
    echo | ps -ef | grep $JAVA_HOME | grep -v "grep" | grep $TOMCAT_USER
    ;;
  restart)
    echo -n "Restarting Tomcat $TOMCAT_NAME services: "
    $0 stop
    sleep 5
    $0 start
    echo "done."
    ;;
  *)
    echo "Usage: $0 {start|stop|restart|status}"
    exit 1
esac

[init.d]# chkconfig --add tomcat6
[init.d]# chkconfig --levels 345 tomcat6 on
[init.d]# chmod +x tomcat6
Test starting
\[appadmin tomcat6\]$ /sbin/service tomcat6 start
\[appadmin tomcat6\]$ netstat \-an \| grep 8080
\[appadmin tomcat6\]$ /sbin/service tomcat6 stop
\[appadmin tomcat6\]$ cat /opt/tomcat6/logs/catalina.out

Edit the <Connectors to have uri encoding of utf8
<Connector  URIEncoding="UTF-8"

MySQL

[software]# wget http://dev.mysql.com/get/Downloads/MySQL-5.1/MySQL-server-5.1.43-1.glibc23.i386.rpm/from/http://mysql.he.net/
[software]# rpm -Uvh MySQL-server-5.1.43-1.glibc23.i386.rpm
[software]# wget http://dev.mysql.com/get/Downloads/MySQL-5.1/MySQL-client-5.1.43-1.glibc23.i386.rpm/from/http://mysql.llarian.net/
[software]# rpm -Uvh MySQL-client-5.1.43-1.glibc23.i386.rpm
[software]# /usr/bin/mysqladmin -u root password 'somesecret'
[software]# /usr/bin/mysqladmin -u root -h domU-12-31-39-03-CC-16 password 'somesecret'
[software]# netstat -an | grep 3306
[software]# /sbin/service mysql stop
[software]# /sbin/service mysql start
[software]# mysql -uroot -p
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.1.43 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> create database grouper;
Query OK, 1 row affected (0.00 sec)

mysql> create user 'grouper'@'localhost' identified by 'somesecret';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all on grouper.* to 'grouper'@'localhost';
Query OK, 0 rows affected (0.00 sec)

mysql>


Note: you can tunnel the 3306 port from ec2 to local through putty, and connect with sqlyog or another mysql gui UI


Grouper API

[software]# wget http://www.internet2.edu/grouper/release/1.5.1/grouper.binary.1.5.1.tar.gz
[opt]# cd ..
[software]# mkdir grouper
[software]# cd grouper
[grouper]# mkdir 1.5.1
[grouper]# cd 1.5.1
[1.5.1]# tar xzvf /opt/software/grouper.binary.1.5.1.tar.gz
[1.5.1]# cd /opt/grouper/1.5.1/grouper.binary.1.5.1/conf
[conf]# emacs grouper.hibernate.properties

hibernate.dialect               = org.hibernate.dialect.MySQL5Dialect

hibernate.connection.driver_class = com.mysql.jdbc.Driver

hibernate.connection.url = jdbc:mysql://localhost:3306/grouper

hibernate.connection.username         = grouper

hibernate.connection.password         = someSecret

[conf]# cd ../bin
[bin]# ./gsh.sh -registry -runscript
[bin]# ./gsh.sh

[opt]# chown -R appadmin.appadmin /opt/grouper

Mod JK (or mod proxy ajp below)

[software]# wget http://download.nextag.com/apache/tomcat/tomcat-connectors/jk/source/jk-1.2.28/tomcat-connectors-1.2.28-src.tar.gz
[software]# tar xzvf tomcat-connectors-1.2.28-src.tar.gz
[software]# cd tomcat-connectors-1.2.28-src/native
[native]# ./configure --with-apxs=/opt/apache/bin/apxs
[native]# make
[native]# make install

[appadmin modules]$ emacs /opt/apache/conf/httpd.conf

LoadModule jk_module modules/mod_jk.so

Include /opt/apache/conf/extra/httpd-jk.conf

[appadmin modules]$ emacs /opt/apache/conf/extra/httpd-jk.conf

JkWorkersFile /opt/apache/conf/workers.properties
JkLogFile /opt/apache/logs/mod_jk.log
JkLogLevel error
JkLogStampFormat "[%a %b %d %H:%M:%S %Y] "
JkOptions +ForwardKeySize +ForwardURICompat -ForwardDirectories
JkRequestLogFormat "%w %V %T"

[appadmin modules]$ emacs /opt/apache/conf/extra/httpd-grouper.conf

JkMount /grouper/* tomcat6

[appadmin modules]$ emacs /opt/apache/conf/extra/httpd-ssl.conf

  Include /opt/apache/conf/extra/httpd-grouper.conf

</VirtualHost>

[appadmin modules]# /sbin/service httpd restart
[appadmin modules]# /sbin/service tomcat6 restart



Mod Proxy AJP (or mod jk above)

Note, this was already install in apache
[appadmin@i2midev1 logs]$ more /etc/httpd/conf.d/proxy_ajp.conf

LoadModule proxy_ajp_module modules/mod_proxy_ajp.so

ProxyPass /grouper_v1_6/ ajp://localhost:8009/grouper_v1_6/

#############################
Make sure request.tomcatAuthentication="false" is in the tomcat server.xml

   <Connector port="8009" protocol="AJP/1.3" redirectPort="8443" request.tomcatAuthentication="false" />


Shibboleth

  • Create the file /etc/yum.repos.d/shibboleth.repo with the following:
    [security_shibboleth]
    name=Shibboleth (CentOS_5)
    type=rpm-md
    baseurl=http://download.opensuse.org/repositories/security:/shibboleth/CentOS_5/
    gpgcheck=1
    gpgkey=http://download.opensuse.org/repositories/security:/shibboleth/CentOS_5/repodata/repomd.xml.key
    enabled=1
    
  • Install Shibboleth SP
    yum install shibboleth.x86_64
    
  • Download the InCommon signing key
    wget -P /etc/shibboleth/ https://wayf.incommonfederation.org/bridge/certs/incommon.pem
    
  • Update Apache configuration to require authentication. For instance, update /etc/httpd/conf.d/shib.conf with the following:
    <Location />
      AuthType shibboleth
      ShibRequestSetting requireSession 1
      require valid-user
    </Location>
    
  • In the /etc/shibboleth/shibboleth2.xml file, set the entityID, use a WAYF or DS, and configure the trusted metadata files.
        <ApplicationDefaults id="default" policyId="default"
            entityID="https://grouperdemo.internet2.edu/shibboleth"
            REMOTE_USER="eppn persistent-id targeted-id"
            signing="false" encryption="false">
    
                <!-- Set isDefault to false here. -->
                <SessionInitiator type="Chaining" Location="/Login" isDefault="false" id="Intranet"
                        relayState="cookie" entityID="https://idp.example.org/shibboleth">
                    <SessionInitiator type="SAML2" acsIndex="1" template="bindingTemplate.html"/>
                    <SessionInitiator type="Shib1" acsIndex="5"/>
                </SessionInitiator>
    
                <!-- Set isDefault to true here and set the URL for the DS -->
                <SessionInitiator type="Chaining" Location="/DS" isDefault="true" id="DS" relayState="cookie">
                    <SessionInitiator type="SAML2" acsIndex="1" template="bindingTemplate.html"/>
                    <SessionInitiator type="Shib1" acsIndex="5"/>
                    <SessionInitiator type="SAMLDS" URL="https://wayf.incommonfederation.org/DS/WAYF"/>
                </SessionInitiator>
    
    
           <!-- Set the URL for the metadata file along with the certificate used to validate the metadata signature -->
           <MetadataProvider type="Chaining">
                <MetadataProvider type="XML" uri="http://wayf.incommonfederation.org/InCommon/InCommon-metadata.xml"
                     backingFilePath="InCommon-metadata.xml" reloadInterval="7200">
                   <MetadataFilter type="RequireValidUntil" maxValidityInterval="2419200"/>
                   <MetadataFilter type="Signature" certificate="incommon.pem"/>
                </MetadataProvider>
            </MetadataProvider>
    
  • Restart Apache and Shibboleth
    /etc/init.d/httpd restart
    /etc/init.d/shibd restart
    
  • Note: logs are in /var/log/shibboleth on the demo machine

GrouperUI

[bin]# cd software
[software]# wget http://www.internet2.edu/grouper/release/1.5.1/grouper-ui-1.5.1.tar.gz
[software]# cd /opt/grouper/1.5.1/
[1.5.1]# tar xzvf /opt/software/grouper-ui-1.5.1.tar.gz
[software]# cd grouper-ui-1.5.1/
[grouper-ui-1.5.1]# ant
exit
[grouper-ui-1.5.1]# emacs build.properties

grouper.folder=/opt/grouper/1.5.1/grouper.binary.1.5.1

[grouper-ui-1.5.1]# ant
war

[grouper-ui-1.5.1]# sudo su - appadmin
[appadmin]$ cd /opt/tomcat6/webapps/
[webapps]$ cp /opt/grouper/1.5.1/grouper-ui-1.5.1/dist/grouper.war /opt/tomcat6/webapps/
[webapps]$ emacs /opt/tomcat6/conf/tomcat-users.xml

  <role rolename="grouper_user"/>
  <user username="GrouperSystem" password="GrouperSystem1" roles="grouper_user"/>

Test with a browser: https://ec2testhyzer.info/grouper/, login as GrouperSystem / someSecret

[opt]# chown -R appadmin.appadmin /opt/grouper

Grouper WS

[software]# wget http://www.internet2.edu/grouper/release/1.5.1/grouper-ws-1.5.1.tar.gz
[software]# cd ../grouper/1.5.1/
[1.5.1]# tar xzvf /opt/software/grouper-ws-1.5.1.tar.gz
[1.5.1]# cd grouper-ws-1.5.1/grouper-ws
[grouper-ws]# ant
  will fail
[grouper-ws]# emacs build.properties

grouper.dir=/opt/grouper/1.5.1/grouper.binary.1.5.1

[grouper-ws]# ant
[grouper-ws]# su - appadmin
[appadmin opt]$ cp /opt/grouper/1.5.1/grouper-ws-1.5.1/grouper-ws/build/dist/grouper-ws.war /opt/tomcat6/webapps/

[appadmin opt]$ emacs /opt/apache/conf/extra/httpd-grouper.conf

JkMount /grouper-ws/* tomcat6

[grouper-ws]# /sbin/service httpd graceful

Add a test group

[grouper-ws]# cd /opt/grouper/1.5.1/grouper.binary.1.5.1/bin
[bin]# ./gsh.sh
Using GROUPER_HOME: /opt/grouper/1.5.1/grouper.binary.1.5.1/bin/..
Using GROUPER_CONF: /opt/grouper/1.5.1/grouper.binary.1.5.1/bin/../conf
Using JAVA: /opt/java/bin/java
using MEMORY: 64m-512m
Grouper starting up: version: 1.5.1, build date: 2010/01/22 09:58:41, env: <no label configured>
grouper.properties read from: /opt/grouper/1.5.1/grouper.binary.1.5.1/conf/grouper.properties
Grouper current directory is: /opt/grouper/1.5.1/grouper.binary.1.5.1/bin
log4j.properties read from:   /opt/grouper/1.5.1/grouper.binary.1.5.1/conf/log4j.properties
Grouper is logging to file:   /opt/grouper/1.5.1/grouper.binary.1.5.1/bin/../logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties
grouper.hibernate.properties: /opt/grouper/1.5.1/grouper.binary.1.5.1/conf/grouper.hibernate.properties
grouper.hibernate.properties: grouper@jdbc:mysql://localhost:3306/grouper
sources.xml read from:        /opt/grouper/1.5.1/grouper.binary.1.5.1/conf/sources.xml
sources.xml groupersource id: g:gsa
sources.xml jdbc source id:   jdbc: GrouperJdbcConnectionProvider
Type help() for instructions
gsh 0% addRootStem("test", "test");
stem: name='test' displayName='test' uuid='80e89fc568504d09adc99087f6d44e5f'
gsh 1% addGroup("test", "testGroup", "testGroup");
group: name='test:testGroup' displayName='test:testGroup' uuid='eb79ba0fe6dd4f849782b2823fae1bb1'
gsh 2% quit

[bin]#

Test the web services:

https://ec2testhyzer.info/grouper-ws/servicesRest/v1_5_000/groups?wsLiteObjectType=WsRestFindGroupsLiteRequest&groupName=test:testGroup&queryFilterType=FIND_BY_GROUP_NAME_APPROXIMATE

Login as: GrouperSystem / someSecret

[opt]# chown -R appadmin.appadmin /opt/grouper

Grouper client

[software]# wget http://www.internet2.edu/grouper/release/1.5.1/grouperClient.binary-1.5.1.tar.gz
[software]# cd /opt/grouper/1.5.1/
[1.5.1]# tar xzvf /opt/software/grouperClient.binary-1.5.1.tar.gz
[1.5.1]# cd grouperClient.binary-1.5.1/
[grouperClient.binary-1.5.1]# emacs grouper.client.properties


grouperClient.webService.url = https://ec2testhyzer.info/grouper-ws/servicesRest

grouperClient.webService.login = GrouperSystem

grouperClient.webService.password = someSecret

[grouperClient.binary-1.5.1]# java -jar grouperClient.jar --operation=findGroupsWs --queryFilterType=FIND_BY_GROUP_NAME_APPROXIMATE --groupName=test:testGroup
Index 0: name: test:testGroup, displayName: test:testGroup
[grouperClient.binary-1.5.1]#
[opt]# chown -R appadmin.appadmin /opt/grouper

Grouper loader

[conf]# emacs /opt/grouper/1.5.1/grouper.binary.1.5.1/conf/grouper-loader.properties

loader.autoadd.typesAttributes = true

[conf]# cd /opt/grouper/1.5.1/grouper.binary.1.5.1/bin
[bin]# gsh.sh
gsh 0% exit                (that loaded the types and attributes)

run this mysql:

[software]# mysql -uroot -p grouper
Enter password:
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.1.43 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> CREATE TABLE loader_example ( subject_id VARCHAR(100), source_id varchar(100) );

INSERT INTO loader_example (subject_id, source_id) VALUES ('GrouperSystem', 'g:isa');
COMMIT;

Open the UI: https://ec2testhyzer.info/grouper/

Go to group: test:testGroup

Edit it, add type: grouperLoader, edit attributes

grouperLoaderDbName: grouper
grouperLoaderScheduleType: CRON

(every minute, you wouldnt normally do this, just for testing)
grouperLoaderQuartzCron: 0 * * * * ?

grouperLoaderQuery: select subject_id, source_id as subject_source_id from loader_example
grouperLoaderType: SQL_SIMPLE

[opt]# chown -R appadmin.appadmin /opt/grouper

Kick off the loader as a background job:

[appadmin bin]$ cd /opt/grouper/1.5.1/grouper.binary.1.5.1/bin
[appadmin bin]$ nohup ./gsh.sh -loader &

Now, you can look in grouper_loader_log table, and see log entries every minute.
Also, you can remove the record from loader_example table, see the membership get removed
(in log table and UI, at the top of the minute), and you can add it back,
and see it get added back in at top of minute.

Configure OpenLDAP

  • copy ldappc schema
    # cp /opt/grouper/1.5.1/grouper.binary.1.5.1/misc/ldap/ldappc.schema /etc/openldap/schema/
  • configure slapd.conf
    include         /etc/openldap/schema/core.schema
    include         /etc/openldap/schema/cosine.schema
    include         /etc/openldap/schema/inetorgperson.schema
    include         /etc/openldap/schema/nis.schema
    include         /etc/openldap/schema/misc.schema
    include         /etc/openldap/schema/ldappc.schema
    
    pidfile         /var/run/openldap/slapd.pid
    argsfile        /var/run/openldap/slapd.args
    
    database        bdb
    suffix          "dc=grouper,dc=edu"
    rootdn          "cn=Manager,dc=grouper,dc=edu"
    rootpw          password
    
    directory       /var/lib/ldap
    
    index objectClass                       eq,pres
    index ou,cn,mail,surname,givenname      eq,pres,sub
    index uid,memberUid                     eq,pres,sub
    index member,isMemberOf,hasMember       eq
  • configure data directory
    # mkdir /var/lib/ldap
    # chown ldap /var/lib/ldap
    # chmod go-rwx ldap
    # cp /etc/openldap/DB_CONFIG.example /var/lib/ldap/DB_CONFIG
  • start slapd
    # /etc/init.d/ldap start
  • add OUs and subjects
    dn: dc=grouper,dc=edu
    objectclass: top
    objectclass: domain
    dc: grouper
    
    dn: ou=groups,dc=grouper,dc=edu
    objectclass: top
    objectclass: organizationalUnit
    ou: groups
    
    dn: ou=people,dc=grouper,dc=edu
    objectclass: top
    objectclass: organizationalUnit
    description: people
    ou: people
    
    dn: cn=test.subject.0,ou=people,dc=grouper,dc=edu
    objectclass: top
    objectclass: person
    objectclass: organizationalPerson
    cn: test.subject.0
    description: test.subject.0
    sn: subject.0
    
    dn: cn=test.subject.1,ou=people,dc=grouper,dc=edu
    objectclass: top
    objectclass: person
    objectclass: organizationalPerson
    cn: test.subject.1
    description: test.subject.1
    sn: subject.1
    
    # ldapadd -x -D cn=Manager,dc=grouper,dc=edu -w password -f file.ldif
  • test
    # ldapsearch -x -D"cn=Manager,dc=grouper,dc=edu" -w password objectclass=*
  • configure JNDI source in sources.xml
    <source adapterClass="edu.internet2.middleware.grouper.subj.GrouperJndiSourceAdapter">
        <id>example</id>
        <name>Example Edu</name>
        <type>person</type>
        <init-param>
          <param-name>INITIAL_CONTEXT_FACTORY</param-name>
          <param-value>com.sun.jndi.ldap.LdapCtxFactory</param-value>
        </init-param>
        <init-param>
          <param-name>PROVIDER_URL</param-name>
          <param-value>ldap://localhost:389</param-value>
        </init-param>
        <init-param>
          <param-name>SECURITY_AUTHENTICATION</param-name>
          <param-value>simple</param-value>
        </init-param>
        <init-param>
          <param-name>SECURITY_PRINCIPAL</param-name>
          <param-value>cn=Manager,dc=grouper,dc=edu</param-value>
        </init-param>
        <init-param>
          <param-name>SECURITY_CREDENTIALS</param-name>
          <param-value>password</param-value>
        </init-param>
         <init-param>
          <param-name>SubjectID_AttributeType</param-name>
          <param-value>cn</param-value>
        </init-param>
        <init-param>
          <param-name>Name_AttributeType</param-name>
          <param-value>cn</param-value>
        </init-param>
        <init-param>
          <param-name>Description_AttributeType</param-name>
          <param-value>description</param-value>
        </init-param>
    
        /// Scope Values can be: OBJECT_SCOPE, ONELEVEL_SCOPE, SUBTREE_SCOPE
        /// For filter use
    
        <search>
            <searchType>searchSubject</searchType>
            <param>
                <param-name>filter</param-name>
                <param-value>
                    (&amp; (cn=%TERM%) (objectclass=person))
                </param-value>
            </param>
            <param>
                <param-name>scope</param-name>
                <param-value>
                    SUBTREE_SCOPE
                </param-value>
            </param>
            <param>
                <param-name>base</param-name>
                <param-value>
                    ou=people,dc=grouper,dc=edu
                </param-value>
            </param>
    
        </search>
        <search>
            <searchType>searchSubjectByIdentifier</searchType>
            <param>
                <param-name>filter</param-name>
                <param-value>
                    (&amp; (cn=%TERM%) (objectclass=person))
                </param-value>
            </param>
            <param>
                <param-name>scope</param-name>
                <param-value>
                    SUBTREE_SCOPE
                </param-value>
            </param>
            <param>
                <param-name>base</param-name>
                <param-value>
                    ou=people,dc=grouper,dc=edu
                </param-value>
            </param>
        </search>
    
        <search>
           <searchType>search</searchType>
             <param>
                <param-name>filter</param-name>
                <param-value>
                    (&amp; (cn=%TERM%)(objectclass=person))
                </param-value>
            </param>
            <param>
                <param-name>scope</param-name>
                <param-value>
                    SUBTREE_SCOPE
                </param-value>
            </param>
             <param>
                <param-name>base</param-name>
                <param-value>
                    ou=people,dc=grouper,dc=edu
                </param-value>
            </param>
        </search>
        ///Attributes you would like to display when doing a search
        <attribute>sn</attribute>
        <attribute>cn</attribute>
        <attribute>description</attribute>
    
      </source>
    
  • create group and add members
    [grouper.binary.1.5.1]# bin/gsh.sh
    
    gsh 0% addGroup("test","testGroup2","testGroup2")
    group: name='test:testGroup2' displayName='test:testGroup2' uuid='...'
    gsh 1% addMember("test:testGroup2","test.subject.0")
    true
    gsh 1% addMember("test:testGroup2","test.subject.1")
    true

LDAPPC

  • ldappc.properties
    [grouper.binary.1.5.1]# cp conf/ldappc.example.properties conf/ldappc.properties
    edu.vt.middleware.ldap.ldapUrl=ldap://127.0.0.1:389
    edu.vt.middleware.ldap.base=dc=grouper,dc=edu
    edu.vt.middleware.ldap.authtype=simple
    edu.vt.middleware.ldap.serviceUser=cn=Manager,dc=grouper,dc=edu
    edu.vt.middleware.ldap.serviceCredential=password
    edu.vt.middleware.ldap.searchResultHandlers=edu.internet2.middleware.ldappc.util.QuotedDnResultHandler,edu.vt.middleware.ldap.handler.FqdnSearchResultHandler
  • ldappc.xml
    [grouper.binary.1.5.1]# cp conf/ldappc.example.openldap.xml  conf/ldappc.xml
<?xml version="1.0" encoding="utf-8"?>

<ldappc>
  <grouper>
    <group-queries>

      <attribute-matching-queries>
        <attribute-list>
          <attribute name="name" value="test:testGroup2" />
        </attribute-list>
      </attribute-matching-queries>

    </group-queries>

    <groups
      structure="flat"
      root-dn="ou=groups,${edu.vt.middleware.ldap.base}"
      ldap-object-class="groupOfNames"
      ldap-rdn-attribute="cn"
      grouper-attribute="name">

      <group-members-dn-list list-object-class="groupOfNames" list-attribute="member" list-empty-value="" />

      <group-members-name-list list-object-class="eduMember" list-attribute="hasMember">
        <source-subject-name-mapping>
          <source-subject-name-map source="example" subject-attribute="cn" />
          <source-subject-name-map source="g:gsa" subject-attribute="name" />
        </source-subject-name-mapping>
      </group-members-name-list>

      <group-attribute-mapping ldap-object-class="groupOfNames">
        <group-attribute-map group-attribute="description" ldap-attribute="description" />
      </group-attribute-mapping>

    </groups>

    <memberships>
      <member-groups-list list-object-class="eduMember" list-attribute="isMemberOf" naming-attribute="name" />
    </memberships>

  </grouper>

  <source-subject-identifiers>
    <source-subject-identifier source="example" subject-attribute="cn">
      <ldap-search
        base="ou=people,${edu.vt.middleware.ldap.base}"
        scope="subtree_scope"
        filter="(cn={0})" />
    </source-subject-identifier>
  </source-subject-identifiers>

</ldappc>
  • provision groups and memberships
    [grouper.binary.1.5.1]# bin/gsh.sh -ldappc -groups -memberships
  • enjoy
    ldapsearch -x -D"cn=Manager,dc=grouper,dc=edu" -w password objectclass=*
    
    # extended LDIF
    #
    # LDAPv3
    # base <> with scope subtree
    # filter: objectclass=*
    # requesting: ALL
    #
    
    # grouper.edu
    dn: dc=grouper,dc=edu
    objectClass: top
    objectClass: domain
    dc: grouper
    
    # groups, grouper.edu
    dn: ou=groups,dc=grouper,dc=edu
    objectClass: top
    objectClass: organizationalUnit
    ou: groups
    
    # people, grouper.edu
    dn: ou=people,dc=grouper,dc=edu
    objectClass: top
    objectClass: organizationalUnit
    description: people
    ou: people
    
    # test.subject.0, people, grouper.edu
    dn: cn=test.subject.0,ou=people,dc=grouper,dc=edu
    objectClass: top
    objectClass: person
    objectClass: organizationalPerson
    objectClass: eduMember
    cn: test.subject.0
    description: test.subject.0
    sn: subject.0
    isMemberOf: test:testGroup2
    
    # test.subject.1, people, grouper.edu
    dn: cn=test.subject.1,ou=people,dc=grouper,dc=edu
    objectClass: top
    objectClass: person
    objectClass: organizationalPerson
    objectClass: eduMember
    cn: test.subject.1
    description: test.subject.1
    sn: subject.1
    isMemberOf: test:testGroup2
    
    # test:testGroup2, groups, grouper.edu
    dn: cn=test:testGroup2,ou=groups,dc=grouper,dc=edu
    description: testGroup2
    objectClass: groupOfNames
    objectClass: eduMember
    member: cn=test.subject.1,ou=people,dc=grouper,dc=edu
    member: cn=test.subject.0,ou=people,dc=grouper,dc=edu
    cn: test:testGroup2
    hasMember: test.subject.0
    hasMember: test.subject.1
    
    # search result
    search: 2
    result: 0 Success
    
    # numResponses: 7
    # numEntries: 6
  • configure slapd logging
[root]# cat /etc/rsyslog.conf
...
local4.*  /var/log/ldap.log

[root]# ps -ef | grep rsyslog
[root]# kill -HUP <pid of rsyslog>
  • configure ldap.conf
[root]# cat /etc/openldap/ldap.conf
BASE    dc=grouper,dc=edu
URI     ldap://127.0.0.1:389
BINDDN  cn=Manager,dc=grouper,dc=edu

sdf

  • No labels