Child pages
  • v1.5.0 Upgrade Instructions from v1.4.2

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Using the 1.5.0 API, perform a registry check using GSH to create an SQL file that will contain the DDL to update your database. To do this, run: gsh -registry -check For instance..

    Code Block
    [root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -check
    Using GROUPER_HOME: /srv/grouper
    Using GROUPER_CONF: /srv/grouper/conf
    Using JAVA: java
    using MEMORY: 64m-512m
    Grouper starting up: version: 1.5.0, build date: 2009/11/14 13:14:33, env: <no label configured>
    grouper.properties read from: /srv/grouper/grouper.properties
    Grouper current directory is: /srv/grouper
    log4j.properties read from:   /srv/grouper/conf/log4j.properties
    Grouper is logging to file:   /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties
    grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties
    grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV
    sources.xml read from:        /srv/grouper/conf/sources.xml
    sources.xml groupersource id: g:gsa
    sources.xml jdbc source id:   jdbc: GrouperJdbcConnectionProvider
    (note, might need to type in your response multiple times (Java stdin is flaky))
    (note, you can allow or deny db urls and users in the grouper.properties)
    Are you sure you want to schemaexport all tables (dropThenCreate=F,writeAndRunScript=F) in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n):
    y
    Continuing...
    Grouper ddl object type 'Grouper' has dbVersion: 13 and java version: 22
    Grouper database schema DDL requires updates
    (should run script manually and carefully, in sections, verify data before drop statements, backup/export important data before starting, follow change log on confluence, dont run exact same script in multiple envs - g
    enerate a new one for each env),
    script file is:
    /srv/grouper/grouperDdl_20091114_13_16_01_466.sql
    Note: this script was not executed due to option passed in
    To run script via gsh, carefully review it, then run this:
    gsh -registry -runsqlfile /srv/grouper/grouperDdl_20091114_13_16_01_466.sql
    


  • In this example above, an SQL script called /srv/grouper/grouperDdl_20091114_13_16_01_466.sql was created.
  • Review the script to make sure it looks okay.  The script shouldn't be dropping or truncating any tables.  However, it will drop and recreate views, some constraints, and some indexes.
    • If using postgres, you might see some tables being backed up and recreated
    • If using postgres, you should see foreign keys being dropped at the top of the script.  If not, try setting the ddlutils.schema grouper.properties setting and run again.  If you still dont see foreign keys being dropped at the top of the script, manually drop all foreign keys before running the script.
  • If you are okay with the SQL script, execute using GSH again.  To do this, run:  gsh -registry -runsqlfile /path/to/sql/file.sql  For instance..

    Code Block
    [root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -runsqlfile /srv/grouper/grouperDdl_20091114_13_16_01_466.sql
    Using GROUPER_HOME: /srv/grouper
    Using GROUPER_CONF: /srv/grouper/conf
    Using JAVA: java
    using MEMORY: 64m-512m
    (note, might need to type in your response multiple times (Java stdin is flaky))
    (note, you can whitelistallow or blacklistdeny db urls and users in the grouper.properties)
    Are you sure you want to run the sql file in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n):
    y
    Continuing...
    Script was executed successfully
    
    Grouper starting up: version: 1.5.0, build date: 2009/11/14 13:14:33, env: <no label configured>
    grouper.properties read from: /srv/grouper/grouper.properties
    Grouper current directory is: /srv/grouper
    log4j.properties read from:   /srv/grouper/conf/log4j.properties
    Grouper is logging to file:   /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties
    grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties
    grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV
    sources.xml read from:        /srv/grouper/conf/sources.xml
    sources.xml groupersource id: g:gsa
    sources.xml jdbc source id:   jdbc: GrouperJdbcConnectionProvider
    


  • If you script fails on:

...

  • At this point, your DDL has been upgraded to v1.5.0.  However, due to restructing in how we handle effective memberships, there is one additional step that needs to be taken for Grouper to be able to find memberships in your database.  Membership objects are now creating by joining the grouper_memberships table with the grouper_group_set table.  The latter table is a new table and needs to be populating now.  To do so, start up GSH and run the following:  new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().addAllMissingGroupSets()  For instance..

    Code Block
    [root@idms-devel-01 grouper]# ./bin/gsh.sh
    Using GROUPER_HOME: /srv/grouper
    Using GROUPER_CONF: /srv/grouper/conf
    Using JAVA: java
    using MEMORY: 64m-512m
    Grouper starting up: version: 1.5.0, build date: 2009/11/14 13:14:33, env: <no label configured>
    grouper.properties read from: /srv/grouper/grouper.properties
    Grouper current directory is: /srv/grouper
    log4j.properties read from:   /srv/grouper/conf/log4j.properties
    Grouper is logging to file:   /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties
    grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties
    grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV
    sources.xml read from:        /srv/grouper/conf/sources.xml
    sources.xml groupersource id: g:gsa
    sources.xml jdbc source id:   jdbc: GrouperJdbcConnectionProvider
    Type help() for instructions
    gsh 0%
    gsh 1%
    gsh 2% new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().addAllMissingGroupSets()
    


  • Note that when GroupSets are added, memberships are going to appear to the API as if they are new.  So if configured, membership hooks will fire, memberships will be added to the changelog and lastMembershipChange (used by LDAPPC) will be updated for groups and stems.  To avoid this, simply update the grouper.properties file before adding GroupSets.  To disable membership hooks, you can comment out the property hooks.membership.class.  To disable the changeLog, set changeLog.enabled to false.  And to prevent updates to lastMembershipChange, set groups.updateLastMembershipTime and stems.updateLastMembershipTime to false.  After the GroupSets are added, you can set those properties back to the way they were.
  • Depending on the number of groups, folders and effective memberships you have, there may be a large number of GroupSets created.  One GroupSet is created for each field for each group and stem and one GroupSet is created for each membership where the member is a group.  By default, each GroupSet that is created will print a line.  For instance, if you have one folder called "etc" and one group called "etc:wheel", you will see output similar to the following.

    Code Block
    Adding self groupSet for etc for field naming / creators
    Adding self groupSet for etc for field naming / stemmers
    Adding self groupSet for etc:wheel for field access / viewers
    Adding self groupSet for etc:wheel for field access / updaters
    Adding self groupSet for etc:wheel for field access / readers
    Adding self groupSet for etc:wheel for field access / optins
    Adding self groupSet for etc:wheel for field access / optouts
    Adding self groupSet for etc:wheel for field access / admins
    Adding self groupSet for etc:wheel for field list / members
    


  • And for each membership where the member is a group, you will see output similar to the following.

    Code Block
    Adding groupSet for ownerGroupId = f182a1d4-8bc4-4c96-9249-44b2111ff2a1, memberGroupId = b4450fa0-d746-4d81-bb5d-7449dc5025b3 for field access / admins
    


  • If you would like to prevent output from being printed to your screen, you can call the showResults(boolean) method on AddMissingGroupSets.  For instance..

    Code Block
    gsh 3% new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().showResults(false).addAllMissingGroupSets()
    


  • After you are satisified with the upgrade, there are some backup columns and a backup table that were creating during the registry upgrade that you can remove.  If you would like to remove those, do the following.
    • Configure your grouper.properties file to have those backup columns and table dropped.

      Code Block
      # before the group name etc was moved to the grouper_groups table, the attributes table
      # was backed up.  If it should not be backed up, or if the upgrade is done and works, then it can
      # be removed, set to true, run: gsh -registry -deep
      ddlutils.dropAttributeBackupTableFromGroupUpgrade = true
      
      # Since grouper_memberships no longer has effective memberships, that table doesn't need via_id,
      # depth and parent_membership.  If they were converted, this will drop the backup of those cols with: gsh -registry -deep
      ddlutils.dropMembershipBackupColsFromOwnerViaUpgrade = true
      


    • Run gsh -registry -deep  For instance..

      Code Block
      [root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -deep
      Using GROUPER_HOME: /srv/grouper
      Using GROUPER_CONF: /srv/grouper/conf
      Using JAVA: java
      using MEMORY: 64m-512m
      Grouper starting up: version: 1.5.0, build date: 2009/11/14 14:54:09, env: <no label configured>
      grouper.properties read from: /srv/grouper/grouper.properties
      Grouper current directory is: /srv/grouper
      log4j.properties read from:   /srv/grouper/conf/log4j.properties
      Grouper is logging to file:   /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties
      grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties
      grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV
      sources.xml read from:        /srv/grouper/conf/sources.xml
      sources.xml groupersource id: g:gsa
      sources.xml jdbc source id:   jdbc: GrouperJdbcConnectionProvider
      (note, might need to type in your response multiple times (Java stdin is flaky))
      (note, you can whitelistallow or blacklistdeny db urls and users in the grouper.properties)
      Are you sure you want to schemaexport all tables (dropThenCreate=F,writeAndRunScript=F) in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n):
      y
      Continuing...
      Grouper ddl object type 'Grouper' has dbVersion: 0 and java version: 22
      Grouper ddl object type 'Subject' has dbVersion: 0 and java version: 1
      Grouper database schema DDL requires updates
      (should run script manually and carefully, in sections, verify data before drop statements, backup/export important data before starting, follow change log on confluence, dont run exact same script in multiple envs - generate a new one for each env),
      script file is:
      /srv/grouper/grouperDdl_20091114_15_02_17_992.sql
      Note: this script was not executed due to option passed in
      To run script via gsh, carefully review it, then run this:
      gsh -registry -runsqlfile /srv/grouper/grouperDdl_20091114_15_02_17_992.sql
      


    • Check the SQL file to make sure it's okay and then run the SQL file.  For instance..

      Code Block
      [root@idms-devel-01 grouper]# ./bin/gsh.sh -registry -runsqlfile /srv/grouper/grouperDdl_20091114_15_02_17_992.sql
      Using GROUPER_HOME: /srv/grouper
      Using GROUPER_CONF: /srv/grouper/conf
      Using JAVA: java
      using MEMORY: 64m-512m
      (note, might need to type in your response multiple times (Java stdin is flaky))
      (note, you can whitelistallow or blacklistdeny db urls and users in the grouper.properties)
      Are you sure you want to run the sql file in db user 'groups', db url 'jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV'? (y|n):
      y
      Continuing...
      Script was executed successfully
      
      Grouper starting up: version: 1.5.0, build date: 2009/11/14 14:54:09, env: <no label configured>
      grouper.properties read from: /srv/grouper/grouper.properties
      Grouper current directory is: /srv/grouper
      log4j.properties read from:   /srv/grouper/conf/log4j.properties
      Grouper is logging to file:   /srv/grouper/logs/grouper_error.log, at min level WARN for package: edu.internet2.middleware.grouper, based on log4j.properties
      grouper.hibernate.properties: /srv/grouper/conf/grouper.hibernate.properties
      grouper.hibernate.properties: groups@jdbc:oracle:thin:@imsdev-db.oit.duke.edu:1612:IMSDEV
      sources.xml read from:        /srv/grouper/conf/sources.xml
      sources.xml groupersource id: g:gsa
      sources.xml jdbc source id:   jdbc: GrouperJdbcConnectionProvider