To upgrade from v1.4.0 grouper to v1.4.1, follow these instructions: (note the changelog)

 API / Web Service / User Interface / Extensions

• Ldappc is now included with grouper, if you are going to use ldappc then get the latest binary or source release of grouper, setup the config files similar to how you did for 1.3.*, and kickoff ldappc with gsh -ldappc
• If you are upgrading from 1.3.* to 1.4.1, and you have already compared the grouper.properties file with grouper.example.properties, and run the ddl command, then you are done.
• If you upgraded to 1.4.0, and now you are upgrading to 1.4.1, continue:
• Get the latest grouper.jar, e.g. from the binary release
• Merge grouper.properties with grouper.example.properties, adding properties: junit.test.loader, junit.test.ldappc, junit.test.ddl
• Database conversion.  The only change is the index on the name column of grouper_stems is now unique.  Here are 3 options for upgrade, pick one
  • Run: grouper_home/bin/gsh -registry -check
    • Then look at the result script, and execute that script, perhaps with:
      grouper_home/bin/gsh -registry -runsqlfile C:/mchyzer/isc/dev/grouper_v1_4/grouper/ddlScripts/grouperDdl_20090125_08_39_44_148.sql
  • -or- you can do this manually, change the stem name index to unique, and update grouper_ddl table Grouper entry DB_VERSION to 13:
    update grouper_ddl set db_version = 13 where object_name = 'Grouper'
  • -or- (not recommended) you could export the registry to backup, rebuild the registry with:
    • gsh -xmlexport GrouperSystem /whatever/20090125_1_4.xml
    • grouper_home/bin/gsh -registry -drop -runscript 
    • gsh -xmlimport GrouperSystem /whatever/20090125_1_4.xml

Web Services

• Get the latest grouper-ws.jar file

Grouper Client

•  Merge the grouper.client.properties with grouper.example.properties.  Note new entries for SSL, custom operations, and the new client version

To upgrade from v1.3.* grouper to v1.4, follow these instructions: (note the changelog)

 API / Web Service / User Interface / Extensions

• Turn off access to your registry
• Backup your database and export your registry to XML as a backup: (e.g. ant xml-export -Dcmd="GrouperSystem export.xml")
• The libraries (jar file) have largely been renamed and upgraded.  Delete all jars specific to grouper, and replace with the new list.  You can get the latest jars from the lib subdirs of the binary release
• Merge all of your config files with the new example config files (you can obtain these from the conf directory of the binary release).  While merging, read about the new settings, and pick your values.
  • Use a tool such as WinMerge (freeware), or Eclipse to make the task easy.  Or, use the new example files, rename, and put your customizations in them
  • Merge grouper.properties with grouper.example.properties
    • Note that grouper.properties was re-organized, so the order of the settings has changed
  • Merge grouper.hibernate.properties with grouper.hibernate.example.properties
  • Merge log4j.properties with log4j.example.properties
  • Merge sources.xml with sources.example.xml
  • Note you dont have to have JDBC credentials in this file if they are the same as grouper.hibernate.properties
  • Merge ehcache.xml with ehcache.example.xml
  • Merge grouper.ehcache.properties with grouper.ehcache.example.properties
  • Add grouper-loader.properties
  • Add morphString.properties
• If you have a custom subject source, recompile it, there are new methods to implement
• In general recompile any code you have that uses Grouper/Subject references: e.g. package structure has changed, and MemberFinder no longer throws MemberNotFound exception...
• You should probably update your database driver jar... there are examples of recent ones in lib/jdbcSamples... e.g. the error "composite_composite_idx in table grouper_composites references the undefined column "owner"" is fixed by updating the postgres driver

Database conversion (try in test env first) TRACK 1 (do this or track 2)

• This is probably the easier way to go, though possibly more time consuming for large registries.  Also, you can try track 2, and if it isnt working for you, then switch to track 1.
  • Note, if you are using HSQL, you should get the newest driver available (e.g. from GROUPER_HOME/lib/jdbcSamples).  Upgrade your hsql probably.  And do track 1 and not track 2.
• After exporting your data to xml (mentioned above), drop all the objects in your schema.  Note, all your UUID's will change
• Run: gsh -registry -runscript
• This will create all the tables
• Then import your XML (e.g.):

gsh -xmlimport GrouperSystem /opt/grouper/1.3.1rc1/grouper/20081117-1.3.1.xml

Database conversion (try in test env first) TRACK 2 (do this or track 1)

Note, if you are using HSQL, you should get the newest driver available (e.g. from GROUPER_HOME/lib/jdbcSamples).  Upgrade your hsql probably.  And do track 1 and not track 2.

• • This track will convert your data and structures to the new format
  • Look in your database (oracle or postgres only, not necessary for mysql) for unique constraints (not unique indices) on grouper tables (not necessary if Oracle).  If any exist, remove them
    • For oracle, execute the script generated by this query:

select 'alter table ' || table_name || ' drop constraint ' || constraint_name || ';' as the_command from user_constraints
where constraint_type in ('R', 'U')
and (table_name like 'GROUPER_%' or table_name like 'SUBJECT%')
 order by constraint_type, table_name;

• • • For postgres, execute the script generated by this query (note, substitute in your schema (public in case below), and catalog (grouper in the case below)

      SELECT 'alter table ' || table_name || ' drop constraint ' || constraint_name || ';' as the_command    FROM information_schema.table_constraints where constraint_type in ('FOREIGN KEY', 'UNIQUE')   and lower(table_name) like 'grouper%' and lower(table_schema) = 'public' and lower(table_catalog) = 'grouper'   order by constraint_type, table_name;

      
      

    • For mysql, it shouldnt be necessary, but here is the script to generate the drop script (change table_schema to match your schema, currently it is grouper):

SELECT concat(concat(concat(concat('alter table ', table_name), ' drop foreign key '), constraint_name), ';') as the_command
   FROM information_schema.table_constraints where constraint_type in ('FOREIGN KEY')
  and lower(table_name) like 'grouper%' and lower(table_schema) = 'grouper'
  order by constraint_type, table_name;

• • Run: gsh -registry -check (this command is in the bin dir of the binary release... make sure all your config files are configured)
    • Note: Seeing some errors here about tables missing, columns missing, ddl version mismatch is expected. 
  • It will give output like this:

C:\dev_inst\eclipse\workspace_v33\grouper\bin>gsh -registry -check
Using GROUPER_HOME:  C:\dev_inst\eclipse\workspace_v33\grouper\bin\..
Using GROUPER_CONF:  C:\dev_inst\eclipse\workspace_v33\grouper\bin\../conf
Using JAVA:          "c:\dev_inst\java/bin/java"
using MEMORY:        64m-512m
Grouper starting up: version: 1.4.0 build date: 2008/11/13 14:42:25
grouper.properties read from: C:\dev_inst\eclipse\workspace_v33\grouper\conf\grouper.properties
Grouper current directory is: C:\dev_inst\eclipse\workspace_v33\grouper\bin
log4j.properties read from:   C:\dev_inst\eclipse\workspace_v33\grouper\conf\log4j.properties
Grouper is logging to file:   C:\dev_inst\eclipse\workspace_v33\grouper\bin\..\logs\grouper_error.log, at min level WARN
 for package: edu.internet2.middleware.grouper, based on log4j.properties
grouper.hibernate.properties: C:\dev_inst\eclipse\workspace_v33\grouper\conf\grouper.hibernate.properties
grouper.hibernate.properties: grouper@jdbc:mysql://localhost:3306/grouper
sources.xml read from:        C:\dev_inst\eclipse\workspace_v33\grouper\conf\sources.xml
sources.xml groupersource id: g:gsa
sources.xml jdbc source id:   jdbc: GrouperJdbcConnectionProvider

Based on grouper.properties: ddlutils.schemaexport.installGrouperData=true
(note, might need to type in your response multiple times (Java stdin is flaky))
(note, you can allow or deny db urls and users in the grouper.properties)
Are you sure you want to schemaexport all tables (dropThenCreate=F,writeAndRunScript=F) in db user 'grouper', db url 'jd
bc:mysql://localhost:3306/grouper'? (y|n):
y
Continuing...
Grouper ddl object type 'Grouper' has dbVersion: 0 and java version: 12
Grouper ddl object type 'Subject' has dbVersion: 0 and java version: 1
Grouper database schema DDL requires updates
(should run script manually and carefully, in sections, verify data before drop statements, backup/export important data
 before starting, follow change log on confluence, dont run exact same script in multiple envs - generate a new one for
each env),
script file is:
C:\dev_inst\eclipse\workspace_v33\grouper\bin\..\ddlScripts\grouperDdl_20081114_01_11_45_007.sql
Note: this script was not executed per the grouper.properties: ddlutils.schemaexport.writeAndRunScript
To run script via gsh, carefully review it, then run this:
gsh -registry -runsqlfile C:\\dev_inst\\eclipse\\workspace_v33\\grouper\\ddlScripts\\grouperDdl_20081114_01_11_45_007.sql

C:\dev_inst\eclipse\workspace_v33\grouper\bin>

• Database conversion (continued)
  • Carefully examine the script (or have a SQL expert do this if you are not one) in that file (in this case: grouperDdl_20081114_01_11_45_007.sql
  • Make sure all operations are ok.  e.g. it should not drop any tables, or truncate tables, or anything else that looks bad.
  • Run this script in your DB IDE or grouper can do it with: gsh -registry -runsqlfile /wherever/filename.sql
  • Note: If it has trouble dropping constraints or indexes, you can manually remove those. All the proper indexes (unless you had custom ones) will be recreated. Then continue the script where it left off. 
• Turn on grouper, it should work.  Look in the logs, make sure no configuration errors or warnings are thrown. If they are, address them.
• If you are using a source release of grouper, you should get the new source release (including build.xml, build.properties etc), and copy your properties files into the /conf dir.  Merge the new build.example.properties with your version
• The database conversion script removed some cols, and relinked some foreign keys.  The old data is still in the table.  After you are satisfied grouper works correctly, you can remove these cols by changing the grouper.properties keys:

# after you have converted id's, and are happy with the conversion of removing the uuid col,
# this will remove the backup uuid cols when running the gsh command: registryInitializeSchema()
ddlutils.dropBackupUuidCols = true

# after you have converted field id foreign keys, and are happy with the conversion of removing the attribute name,
# membership list name, and type cols,
# this will remove the backup field name/type cols when running the gsh command: registryInitializeSchema()
ddlutils.dropBackupFieldNameTypeCols = true

• • Then generate the DDL again (the -deep option will examine the DB structure, not the DB version from the grouper_ddl table): gsh -registry -deep
    • Then examine the resultant script carefully.  Make sure the important operations (e.g. dropping/adding views are innocuous) are dropping the backup columns from the table
    • Turn off access to grouper
    • Backup your registry by database or xml export
    • Run that script
    • Verify the columns are removed
    • Grouper should work

Web services

• Axis jars have changed.  Remove the old ones, add new ones from the grouper-ws release in the lib/axis dir
• Nothing important has changed with grouper-ws.properties, but you may want to merge yours with grouper-ws.example.properties to be sure
• Replace or merge the axis.xml with the one from grouper-ws distribution
• Replace or merge the /services and /modules dirs from the grouper-ws distribution
• Two data changes were made to existing services:
  • Added two more fields to the wsGroupDetail object, adjust your clients appropriately (should only matter for SOAP clients or REST which are not coded correctly)
    • compositeType (should be UNION, COMPLEMENT, INTERSECTION, or blank for non-composite)
    • params
  • There was a weird situation where the result code was transmitted as an object...this is not transmitted anymore

<WsDeleteMemberLiteResult>
  <resultMetadata>
    <wsResultCode
class="edu.internet2.middleware.grouper.ws.soap.WsDeleteMemberLiteResult$WsDeleteMemberLiteResultCode">SUCCESS</wsResultCode>
    <resultCode>SUCCESS</resultCode>
    <resultMessage></resultMessage>
    <success>T</success>

• Merge grouper-ws build.properties with example, and note the group.lib.dir has changed 

User Interface

• Merge the grouper nav.properties with yours.  Note new properties about tooltips in types and attributes.