Using the 1.5.0 API, create the new database schema. To do this, run: gsh -registry -runscript For instance..
[root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -runscript Using GROUPER_HOME: /srv/grouper Using GROUPER_CONF: /srv/grouper/conf Using JAVA: java using MEMORY: 64m-512m Grouper starting up: version: 1.5.0, build date: 2009/11/14 13:14:33, env: <no label configured> grouper.properties read from: /srv/grouper/grouper.properties Grouper current directory is: /srv/grouper log4j.properties read from: /srv/grouper/conf/log4j.properties Grouper is logging to file: /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV sources.xml read from: /srv/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider (note, might need to type in your response multiple times (Java stdin is flaky)) (note, you can allow or deny db urls and users in the grouper.properties) Are you sure you want to schemaexport all tables (dropThenCreate=F,writeAndRunScript=T) in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n): y Continuing... Grouper ddl object type 'Grouper' has dbVersion: 0 and java version: 22 Grouper ddl object type 'Subject' has dbVersion: 0 and java version: 1 Grouper database schema DDL requires updates (should run script manually and carefully, in sections, verify data before drop statements, backup/export important data before starting, follow change log on confluence, dont run exact same script in multiple envs - generate a new one for each env), script file is: /srv/grouper/grouperDdl_20091114_14_34_58_674.sql Script was executed successfully |
Use XML Import to import your Grouper registry using the backup you created. For instance..
./bin/gsh.sh -xmlimport GrouperSystem backup.xml |
Using the 1.5.0 API, perform a registry check using GSH to create an SQL file that will contain the DDL to update your database. To do this, run: gsh -registry -check For instance..
[root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -check Using GROUPER_HOME: /srv/grouper Using GROUPER_CONF: /srv/grouper/conf Using JAVA: java using MEMORY: 64m-512m Grouper starting up: version: 1.5.0, build date: 2009/11/14 13:14:33, env: <no label configured> grouper.properties read from: /srv/grouper/grouper.properties Grouper current directory is: /srv/grouper log4j.properties read from: /srv/grouper/conf/log4j.properties Grouper is logging to file: /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV sources.xml read from: /srv/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider (note, might need to type in your response multiple times (Java stdin is flaky)) (note, you can allow or deny db urls and users in the grouper.properties) Are you sure you want to schemaexport all tables (dropThenCreate=F,writeAndRunScript=F) in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n): y Continuing... Grouper ddl object type 'Grouper' has dbVersion: 13 and java version: 22 Grouper database schema DDL requires updates (should run script manually and carefully, in sections, verify data before drop statements, backup/export important data before starting, follow change log on confluence, dont run exact same script in multiple envs - g enerate a new one for each env), script file is: /srv/grouper/grouperDdl_20091114_13_16_01_466.sql Note: this script was not executed due to option passed in To run script via gsh, carefully review it, then run this: gsh -registry -runsqlfile /srv/grouper/grouperDdl_20091114_13_16_01_466.sql |
If you are okay with the SQL script, execute using GSH again. To do this, run: gsh -registry -runsqlfile /path/to/sql/file.sql For instance..
[root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -runsqlfile /srv/grouper/grouperDdl_20091114_13_16_01_466.sql Using GROUPER_HOME: /srv/grouper Using GROUPER_CONF: /srv/grouper/conf Using JAVA: java using MEMORY: 64m-512m (note, might need to type in your response multiple times (Java stdin is flaky)) (note, you can allow or deny db urls and users in the grouper.properties) Are you sure you want to run the sql file in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n): y Continuing... Script was executed successfully Grouper starting up: version: 1.5.0, build date: 2009/11/14 13:14:33, env: <no label configured> grouper.properties read from: /srv/grouper/grouper.properties Grouper current directory is: /srv/grouper log4j.properties read from: /srv/grouper/conf/log4j.properties Grouper is logging to file: /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV sources.xml read from: /srv/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider |
CREATE UNIQUE INDEX membership_uniq_idx ON GROUPER_MEMBERSHIPS (OWNER_ID, MEMBER_ID, FIELD_ID); then you need to find the rows which have dupes and delete ONE of them, e.g. select count(*), OWNER_ID, MEMBER_ID, FIELD_ID, max(id) from grouper_memberships group by OWNER_ID, MEMBER_ID, FIELD_ID having count(*) > 1; This returned several rows (of count 2), so I just deleted these. delete from grouper_memberships where (owner_id, member_id, field_id, id, 2) in (select OWNER_ID, MEMBER_ID, FIELD_ID, max(id), count(*) from grouper_memberships group by OWNER_ID, MEMBER_ID, FIELD_ID having count(*) > 1); commit; |
update GROUPER_ATTRIBUTES set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ATTRIBUTE_ASSIGN set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ATTRIBUTE_ASSIGN_VALUE set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ATTRIBUTE_DEF set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ATTRIBUTE_DEF_NAME set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ATTRIBUTE_DEF_NAME_SET set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ATTRIBUTE_DEF_SCOPE set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ATTR_ASSIGN_ACTION set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ATTR_ASSIGN_ACTION_SET set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_AUDIT_ENTRY set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_AUDIT_TYPE set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_CHANGE_LOG_CONSUMER set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_CHANGE_LOG_TYPE set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_COMPOSITES set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_FIELDS set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_GROUPS set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_GROUPS_TYPES set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_GROUP_SET set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_MEMBERS set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_MEMBERSHIPS set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_ROLE_SET set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_STEMS set hibernate_version_number = 0 where hibernate_version_number is null; update GROUPER_TYPES set hibernate_version_number = 0 where hibernate_version_number is null; commit; |
At this point, your DDL has been upgraded to v1.5.0. However, due to restructing in how we handle effective memberships, there is one additional step that needs to be taken for Grouper to be able to find memberships in your database. Membership objects are now creating by joining the grouper_memberships table with the grouper_group_set table. The latter table is a new table and needs to be populating now. To do so, start up GSH and run the following: new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().addAllMissingGroupSets() For instance..
[root@idms-devel-01 grouper]# ./bin/gsh.sh Using GROUPER_HOME: /srv/grouper Using GROUPER_CONF: /srv/grouper/conf Using JAVA: java using MEMORY: 64m-512m Grouper starting up: version: 1.5.0, build date: 2009/11/14 13:14:33, env: <no label configured> grouper.properties read from: /srv/grouper/grouper.properties Grouper current directory is: /srv/grouper log4j.properties read from: /srv/grouper/conf/log4j.properties Grouper is logging to file: /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV sources.xml read from: /srv/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider Type help() for instructions gsh 0% gsh 1% gsh 2% new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().addAllMissingGroupSets() |
Depending on the number of groups, folders and effective memberships you have, there may be a large number of GroupSets created. One GroupSet is created for each field for each group and stem and one GroupSet is created for each membership where the member is a group. By default, each GroupSet that is created will print a line. For instance, if you have one folder called "etc" and one group called "etc:wheel", you will see output similar to the following.
Adding self groupSet for etc for field naming / creators Adding self groupSet for etc for field naming / stemmers Adding self groupSet for etc:wheel for field access / viewers Adding self groupSet for etc:wheel for field access / updaters Adding self groupSet for etc:wheel for field access / readers Adding self groupSet for etc:wheel for field access / optins Adding self groupSet for etc:wheel for field access / optouts Adding self groupSet for etc:wheel for field access / admins Adding self groupSet for etc:wheel for field list / members |
And for each membership where the member is a group, you will see output similar to the following.
Adding groupSet for ownerGroupId = f182a1d4-8bc4-4c96-9249-44b2111ff2a1, memberGroupId = b4450fa0-d746-4d81-bb5d-7449dc5025b3 for field access / admins |
If you would like to prevent output from being printed to your screen, you can call the showResults(boolean) method on AddMissingGroupSets. For instance..
gsh 3% new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().showResults(false).addAllMissingGroupSets() |
Configure your grouper.properties file to have those backup columns and table dropped.
# before the group name etc was moved to the grouper_groups table, the attributes table # was backed up. If it should not be backed up, or if the upgrade is done and works, then it can # be removed, set to true, run: gsh -registry -deep ddlutils.dropAttributeBackupTableFromGroupUpgrade = true # Since grouper_memberships no longer has effective memberships, that table doesn't need via_id, # depth and parent_membership. If they were converted, this will drop the backup of those cols with: gsh -registry -deep ddlutils.dropMembershipBackupColsFromOwnerViaUpgrade = true |
Run gsh -registry -deep For instance..
[root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -deep Using GROUPER_HOME: /srv/grouper Using GROUPER_CONF: /srv/grouper/conf Using JAVA: java using MEMORY: 64m-512m Grouper starting up: version: 1.5.0, build date: 2009/11/14 14:54:09, env: <no label configured> grouper.properties read from: /srv/grouper/grouper.properties Grouper current directory is: /srv/grouper log4j.properties read from: /srv/grouper/conf/log4j.properties Grouper is logging to file: /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV sources.xml read from: /srv/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider (note, might need to type in your response multiple times (Java stdin is flaky)) (note, you can allow or deny db urls and users in the grouper.properties) Are you sure you want to schemaexport all tables (dropThenCreate=F,writeAndRunScript=F) in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n): y Continuing... Grouper ddl object type 'Grouper' has dbVersion: 0 and java version: 22 Grouper ddl object type 'Subject' has dbVersion: 0 and java version: 1 Grouper database schema DDL requires updates (should run script manually and carefully, in sections, verify data before drop statements, backup/export important data before starting, follow change log on confluence, dont run exact same script in multiple envs - generate a new one for each env), script file is: /srv/grouper/grouperDdl_20091114_15_02_17_992.sql Note: this script was not executed due to option passed in To run script via gsh, carefully review it, then run this: gsh -registry -runsqlfile /srv/grouper/grouperDdl_20091114_15_02_17_992.sql |
Check the SQL file to make sure it's okay and then run the SQL file. For instance..
[root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -runsqlfile /srv/grouper/grouperDdl_20091114_15_02_17_992.sql Using GROUPER_HOME: /srv/grouper Using GROUPER_CONF: /srv/grouper/conf Using JAVA: java using MEMORY: 64m-512m (note, might need to type in your response multiple times (Java stdin is flaky)) (note, you can allow or deny db urls and users in the grouper.properties) Are you sure you want to run the sql file in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n): y Continuing... Script was executed successfully Grouper starting up: version: 1.5.0, build date: 2009/11/14 14:54:09, env: <no label configured> grouper.properties read from: /srv/grouper/grouper.properties Grouper current directory is: /srv/grouper log4j.properties read from: /srv/grouper/conf/log4j.properties Grouper is logging to file: /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV sources.xml read from: /srv/grouper/conf/sources.xml sources.xml groupersource id: g:gsa sources.xml jdbc source id: jdbc: GrouperJdbcConnectionProvider |