...
2015/09/30: Make sure to run the grouperInstaller of the version of grouper you are upgrading from to revert all patches before upgrading. Note, If reverting patches doesn’t work, and you skipped that part in an upgrade, then before new patches are applied, look through the WEB-INF/classes dir for classfiles, and if you have never overridden or coded your own, then delete all classfiles in the subdirs. If you have, then keep yours, and delete the others.
Important Changes in Grouper 2.2 that impact the upgrade
...
- If you are upgrading from 1.6 or 2.0, take a look at item #1 in the 2.1 upgrade instructions.
- You should get v2.2 versions of the Grouper API, Grouper UI, Grouper WS, Grouper Daemon, etc. from the Grouper Downloads page. You will need to merge configuration files and JARs. See the v2.2 change log for more information. Also keep in mind that some of the configuration files are now handled differently via configuration overlay. The rest of this document focuses on upgrading the database.
- First you may want to analyze your tables to help speed up the upgrade. Analyze your tables.
Stop the Grouper Daemon. Once you prevent users from making updates to your Grouper instance, run the changeLogTempToChangeLog daemon to clear out the temp changelog using your existing v2.1 API. Here's an example using GSH.
Code Block gsh 0% loaderRunOneJob("CHANGE_LOG_changeLogTempToChangeLog")- Before performing any upgrade steps, export your Grouper registry. Options include performing a database backup (recommended) or using the XML Export utility in Grouper (not recommended since certain features may not get exported).
Using the 2.2 API, perform a registry check using GSH to create an SQL file that will contain the DDL to update your database. To do this, run: gsh -registry -check Note you may need to increase memory. For instance..
Code Block $ export MEM_MAX=2000m $ ./bin/gsh.sh -registry -check Using GROUPER_HOME: /opt/grouper Using GROUPER_CONF: /opt/grouper/conf Using JAVA: java using MEMORY: 64m-2000m Grouper starting up: version: 2.2.0, build date: 2014/05/23 13:16:59, env: <no label configured> grouper.properties read from: /opt/grouper/conf/grouper.properties Grouper current directory is: /opt/grouper log4j.properties read from: /opt/grouper/conf/log4j.properties Grouper is logging to file: /opt/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /opt/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: sa@jdbc:hsqldb:hsql://localhost:9001/grouper sources.xml read from: /opt/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml groupersource id: grouperEntities sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider This db user 'sa' and url 'jdbc:hsqldb:hsql://localhost:9001/grouper' are allowed to be changed in the grouper.properties Continuing... Grouper ddl object type 'Grouper' has dbVersion: 26 and java version: 28 Grouper database schema DDL requires updates (should run script manually and carefully, in sections, verify data before drop statements, backup/export important data before starting, follow change log on confluence, dont run exact same script in multiple envs - generate a new one for each env), script file is: /opt/grouper/ddlScripts/grouperDdl_20140523_13_19_30_162.sql Note: this script was not executed due to option passed in To run script via gsh, carefully review it, then run this: gsh -registry -runsqlfile /opt/grouper/ddlScripts/grouperDdl_20140523_13_19_30_162.sql
- In this example above, an SQL script called /opt/grouper/ddlScripts/grouperDdl_20140523_13_19_30_162.sql was created.
- Postgres only - If using postgres, you should see foreign keys being dropped at the top of the script. If not, try setting the ddlutils.schema grouper.properties setting and run again. If you still don't see foreign keys being dropped at the top of the script, manually drop all foreign keys before running the script.
- Postgres and hsql only - You should backup any non grouper views that depend on Grouper views, run the grouper script (which deletes those views due to drop view cascade), and then you should recreate those non grouper views.
Run the SQL script. To do this, run: gsh -registry -runsqlfile /path/to/sql/file.sql For instance..
Code Block $ ./bin/gsh.sh -registry -runsqlfile /opt/grouper/ddlScripts/grouperDdl_20140523_13_19_30_162.sql Using GROUPER_HOME: /opt/grouper Using GROUPER_CONF: /opt/grouper/conf Using JAVA: java using MEMORY: 64m-2000m This db user 'sa' and url 'jdbc:hsqldb:hsql://localhost:9001/grouper' are allowed to be changed in the grouper.properties Continuing... Script was executed successfully Grouper starting up: version: 2.2.0, build date: 2014/05/23 13:16:59, env: <no label configured> grouper.properties read from: /opt/grouper/conf/grouper.properties Grouper current directory is: /opt/grouper log4j.properties read from: /opt/grouper/conf/log4j.properties Grouper is logging to file: /opt/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /opt/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: sa@jdbc:hsqldb:hsql://localhost:9001/grouper sources.xml read from: /opt/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml groupersource id: grouperEntities sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider
Note that if one of the SQL statements in the script fails, the process will abort leaving the rest of the SQL statements from executing. If this happens, in most cases, you can't just re-run the full script since re-executing some of the DDL changes that previously succeeded would fail now (e.g. dropping a view or constraint that was previously dropped successfully.) You could edit the script to remove the statements that previously succeeded in order to re-execute the statement that failed and the ones after it. Or you can run the previous step again to generate a new SQL script.
- If you are upgrading from 1.6, take a look at item #14 in the 2.1 upgrade instructions.
Now that the DDL updates have been made, there are a few additional GSH commands that need to be run. To do this, run: gsh ../misc/postGrouper2_2Upgrade.gsh (The gsh script is in the "misc" directory.) Note you should check the output to make sure no errors are thrown. If you see an error, it is safe to re-run. For instance..
Code Block $ ./bin/gsh.sh misc/postGrouper2_2Upgrade.gsh Using GROUPER_HOME: /opt/grouper Using GROUPER_CONF: /opt/grouper/conf Using JAVA: java using MEMORY: 64m-2000m Grouper starting up: version: 2.2.0, build date: 2014/06/03 14:30:19, env: <no label configured> grouper.properties read from: /opt/grouper/conf/grouper.properties Grouper current directory is: /opt/grouper log4j.properties read from: /opt/grouper/conf/log4j.properties Grouper is logging to file: /opt/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /opt/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: sa@jdbc:hsqldb:hsql://localhost:9001/grouper sources.xml read from: /opt/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml groupersource id: grouperEntities sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider Type help() for instructions ########################################## # Grouper 2.2 Upgrade Step 1/7: Clear the temp change log before we start ########################################## loader ran successfully: Ran the changeLogTempToChangeLog daemon ########################################## # Grouper 2.2 Upgrade Step 2/7: Add group sets for new privileges (groupAttrRead, groupAttrUpdate, stemAttrRead, etc) ########################################## Searching for all composite groups to cache for later use Searching for missing self groupSets for groups Found 20 missing groupSets Done making 20 updates Searching for missing self groupSets for stems Found 18 missing groupSets Done making 18 updates Searching for missing self groupSets for attribute defs Found 24 missing groupSets Done making 24 updates ########################################## # Grouper 2.2 Upgrade Step 3/7: Add new privileges to point in time (groupAttrRead, groupAttrUpdate, stemAttrRead, etc) ########################################## Searching for missing active point in time fields Found 6 missing active point in time fields Done making 6 updates java.lang.Long: 6 ########################################## # Grouper 2.2 Upgrade Step 4/7: Remove old fields (legacy attributes) from point in time. ########################################## Searching for point in time fields that should be inactive Found 16 active point in time fields that should be inactive Done making 16 updates java.lang.Long: 16 ########################################## # Grouper 2.2 Upgrade Step 5/7: Migrate legacy attributes ########################################## Searching for legacy group types Found 7 legacy group types Done making 7 updates. Number previously migrated = 0. Searching for legacy attributes Found 16 legacy attributes Done making 16 updates. Number previously migrated = 0. Searching for lists Found 3 lists Done making 3 updates. Number previously migrated = 0. Searching for group type assignments Found 5 group type assignments Done making 5 updates. Number previously migrated = 0. Searching for attribute assignments Found 3 attribute assignments Done making 3 updates. Number previously migrated = 0. java.lang.Long: 34 ########################################## # Grouper 2.2 Upgrade Step 6/7: Add stem sets ########################################## Searching for missing self stemSets Found 9 missing self stemSets Done making 9 updates Searching for all stems to find existing stem sets with issues Found 12 stems Done making 3 updates java.lang.Long: 12 ########################################## # Grouper 2.2 Upgrade Step 7/7: Add group sets for new privileges to point in time (groupAttrRead, groupAttrUpdate, stemAttrRead, etc) ########################################## Searching for missing active point in time group sets Found 64 missing active point in time group sets Done making 64 updates java.lang.Long: 64
Note that for the 7th step above (Add group sets for new privileges to point in time (groupAttrRead, groupAttrUpdate, stemAttrRead, etc)), if it will be adding more than 100,000 group sets, then it will process them 100,000 at a time. If you would like to see the progress as it is processing each set of 100K, tail the grouper_error.log file.
Run: gsh ../misc/postGrouper2_2_1Upgrade.gsh (The gsh script is in the "misc" directory.)
Code Block ########################################## # Grouper 2.2.1 Upgrade Step 1/1: Remove MEMBER, ADMIN, UPDATE, and GROUP_ATTR_UPDATE privileges assigned to GrouperAll ########################################## Removing GrouperAll membership: groupId=6aac0469f7f24f47ba465ca4b14e8525, group name=etc:test1, field=admins. Removing GrouperAll membership: groupId=6aac0469f7f24f47ba465ca4b14e8525, group name=etc:test1, field=updaters. Removing GrouperAll membership: groupId=5d3562fb1f6d4c0daebc5b9bf9dce78c, group name=etc:test2, field=members. Finished running successfully. 3 changes made.
- Analyze your tables. (Again to avoid any performance issues.)
- If you are upgrading from 1.6 or 2.0, take a look at item #16 in the 2.1 upgrade instructions.
- Start the Grouper Daemon and all other Grouper components (UI/WS).
- The legacy attributes were backed up in separate tables. After verifying that everything is okay, you can drop those backed up tables by setting ddlutils.dropLegacyAttributes = true in grouper.properties, running gsh -registry -deep and then executing the SQL file.
...