Page History

...

Steps needed to make a Grouper release

Remove unneeded config

Implement Chad's missing config script

Change default github branch

Final todo's

WS tests (DONE)
Look at jars not supposed to be there (from Chad) (DONE)
Chris and Emily to announce and update wiki
Emily check wiki make sure things are up to date
- This week

Verifying cherry-picks from branch are in master (main)

https://stackoverflow.com/questions/7566416/how-to-see-which-commits-in-one-branch-arent-in-the-other

...

For the commits needing attention, there is often no quick way to verify they have been applied to master (main). Probably the easiest thing to do is look at what changed using `git show <commit>`, see what changed, and then `git blame <filename>` in the master (main) branch to see if the changes somehow made it to master (main). Github also has a git blame function, so you can compare changes in two different windows. Another way to compare, if you can find a master (main) commit with the same commit message, is to do a `git show` in both commits and compare the differences (maybe using the diff command). If the only differences are in line numbers and spacing, the commits are a good match.

...

grouper.installer.example.properties
GiGrouperVersion.javaDocumentation for getting jars for client and installer from maven and maybe download a properties file from git

Grouper WS

grouper-ws.base.properties
If there is a new minor version with changes to the wsdl, add a new source folder for that version, copy the coresoap *.javas over to it, refactor to change the package, change the build.xml to build a WSDL for that version. Change GrouperService to be like a previous version but add new methods or change methods in it (DONE)
Make sure that there is no coresoap (package) in use in src/grouper_ws_v1_6, src/grouper_ws_v2_0, etc. Take out all src/grouper_ws_vx_x except one, make sure they dont depend on each other (DONE)
Generate the wssec aar's and commit them (DONE)
Generate WSDLs and commit them (DONE)
Add src/grouper-ws_v2_4 to the source list of the Maven build-help plugin in grouper-ws/grouper-ws/pom.xml (DONE)

Search for the old version (e.g. 2.2.2 or 2_2_2 or 2_2_002) in all the files (Chris)

See which config files / jar files / ddl changed since last release, make sure there are entries in the change log. (do we do this anymore? Grouper changes v2.2)

(TODO)

If its a minor release, change If its a minor release, change the release version policy page Make sure the crontab on webprod3 is installed for this version to make the "grouper.all*.tar.gz tarball"(DONE)

Git Release Commit

This is the single commit that will be the official release. All code work should be finished at this point.

...

Edit all the pom.xmls <version> tags

FROM <version>2TO <version>2.4.0-SNAPSHOT</version>
TO <version>2.4.0</version>

Code Block
mvn
Code Block
mvn versions:set -DnewVersion=2.4.0 # if the git diff looks ok: mvn versions:commit # otherwise # mvn versions:revert

...

Look for other snapshot references that needs to change: `find . -name pom.xml | xargs grep SNAPSHOT`

Compile grouperClient and morphString with the 2.4.0 tag, in case the jar size changes

Update checkConfig with the grouperClient file size

Update grouperClient and morphString jars in the grouper lib (anywhere else?)

Commit in the new branch. Note, if this is in the master (main) branch, or any other branch that Travis will act on, include "[ci skip]" in the commit message, so that it won't build and publish to the Sonatype snapshot repository.

...

In the branch, change pom 2.4.0->2.4.1-SNAPSHOT, update .travis.yml to include the 2.4 branch, commit, test build, commit, and push

In master (main), change pom 2.4.0-SNAPSHOT->2.5.0-SNAPSHOT, test build, commit, and push

...

Make sure all tables, views, and cols (of tables and views) have comments in the DB (oracle or postgres). (TO DO on a rainy day)
Make sure the grouper-ws and gruoper-ui web.xml file has the security basic auth in there (didn't get accidentally overridden in commit)

Security Review

Review UI / Review UI / WS code changes* from the previous version with respect to Open Web Application Security Project (OWASP) top 10 web application security vulnerabilities

...

Run API (SuiteDefault) JUnit tests (set true in all JUnit test includes in grouper.properties). Also run the Grouper Installer, it should end in success for the client connecting to the WS and you should be able to use the UI.

...

MySQL windows
MySQL unix with case sensitive table names
Postgres

Oracle

Code Block
create role GrouperRole; GRANT connect, resource, CREATE view TO GrouperRole; CREATE USER groupertest IDENTIFIED BY <pass>; GRANT connect, resource, GrouperRole TO groupertest;

Hsql

From demo server

Code Block

ps -ef | grep java   --- kill hsql
wget http://www.internet2.edu/grouper/release/x.y.z/grouper.apiBinary-x.y.z.tar.gz
cd grouper.apiBinary-2.0.0
cd bin
java -cp ../lib/jdbcSamples/hsqldb.jar org.hsqldb.Server -port 9001 -database.0 file:grouper -dbname.0 grouper &
./gsh.sh -registry -runscript
./gsh.sh -test -all

Try the UI with a few different languages in the browser request (en without US, french, something not common)

Try checkConfig tests.

Upgrade from x.y-1 to x.y (e.g. 2.3 to 2.4).

Install 2.3.0
- Download and run 2.4 installer
Open the UI, browse around
Upgrade to 2.4.0: v2.4 Upgrade Instructions from v2.3
- Download the installer

Open the UI, browse around

Install the grouper installer

Download grouper installer
Run: java -jar grouperInstaller.jar
Try the UI that was installed
Make sure gsh.sh and gsh works in api, ws, ui for 2.4.0

...

Database

...

Linux

...

OS X (10.6)

...

Solaris

...

Windows

...

HSQLDB

...

CH

...

MySQL

...

Oracle

...

PostgreSQL

...

MsSQL (NO LONGER SUPPORTED)

Web Service JUnit Tests (Grouper WS and Grouper Client) (Shilen work on for 2.4.0, has errors in client tests)

WS samples (DONE for 2.4.0)

WS javadoc (generate, commit, test). Make sure new operations / args / etc are documented in the WS doc page (TODO?)

UI internationalization tests (TODO?)

Test an API patch

Code Block

# try to see if someone else is testing, look when process started, wait if so or email list...
[appadmin@i2midev6 ~]$ ps -ef | grep gsh | grep test | grep all



# see if hsql is running, and stop if so
[appadmin@i2midev6 ~]$ ps -ef | grep hsql
[appadmin@i2midev6 ~]$ kill <the_pid>
[appadmin@i2midev6 ~]$ testGrouper2_3.sh > /tmp/grouperTestResults.txt & 2>&1
[appadmin@i2midev6 ~]$ tail -f /tmp/grouperTestResults.txt

Code Block

getGrouperSource.sh GROUPER_2_4_BRANCH
cd  /home/mchyzer/tmp/grouperDownload/build_mchyzer/grouper-GROUPER_2_4_BRANCH
cd grouper
./start-hsql.sh

cd /home/mchyzer/tmp/grouperDownload/build_mchyzer/grouper-GROUPER_2_4_BRANCH/grouper
ant dist
./bin/gsh.sh -registry -runscript
./bin/gsh.sh -test -all

Compile test on server

Code Block

/**
 * Test {@link Stem}.
 * <p />
 * @author  blair christensen.
 * @version $Id: TestStem.java,v 1.34 2009-12-07 07:31:09 mchyzer Exp $
 */
public class Test extends GrouperTest {

    // Private Class Constants
    private static final Log LOG = GrouperUtil.getLog(Test.class);

    /**
     * Method main.
     * @param args String[]
     */
    public static void main(String[] args) {
        TestRunner.run(new Test("testCache"));
        //TestRunner.run(TestStem.class);
    }

    public Test(String name) {
        super(name);
    }


javac -classpath .:../lib/grouper/*:../lib/test/*:../dist/lib/* -sourcepath . Test.java


 java -classpath ../conf:.:../lib/grouper/*:../lib/test/*:../dist/lib/grouper.jar:../lib/jdbcSamples/hsqldb.jar Test

Packaging and releasing

ssh to i2mibuild.internet2.edu
cd to ~mchyzer
Build all with: [mchyzer@i2mibuild mchyzer]$ bin/buildGrouperAll.sh GROUPER_x_y_z
This is the same as these individual commands:
- bin/buildGrouper.sh GROUPER_x_y_z
- bin/buildGrouperWs.sh GROUPER_x_y_z
- bin/buildGrouperClient.sh GROUPER_x_y_z
- bin/buildGrouperUi.sh GROUPER_x_y_z
- bin/buildGrouperQs.sh GROUPER_x_y_z
Note: sometimes the grouperWs doesnt build correctly due to a bug with javadoc, just try building the ws again...

Resulting .tar.gz's are in ~mchyzer/tmp/grouperAll/build_<username>. There are both source and binary tarballs for the API and Client packages.

scp packages built above, to:

Code Block

sftp login.internet2.edu    (then sftp to webprod3)
lcd /home/mchyzer/tmp/grouperAll/build_username
mkdir /home/htdocs/home/htdocs/software.internet2.edu/grouper/release/1.2.12
cd /home/htdocs/software.internet2.edu/grouper/release/1.2.12
mput *.tar.gz

Build to demo server, after the release is tagged, and built on the build server in Chris' directory... (TODO after release)

[appadmin@i2midev1 2.0.0]$ pwd
/opt/grouper/2.0.0
[appadmin@i2midev1 2.0.0]$ ./upgrade_2.0.0.sh

Packaging and releasing : provisioning

After tagging, publish to maven central by following the instructions on v2.1.0 Grouper Development Environment Using Maven.

Publish to maven central here

Copy grouper.psp-version.tar.gz to the release URL.

Archive the current release

Create PDF of current documentation for the release archive
- Browse -> Advanced -> Export Space
- select PDF, uncheck save comments, clear all
- select pages to export. try to omit non-product pages, defunct pages, and pages for future releases.
- export, save to desktop, rename and attach to GrouperWG/Archives
update Archives page with info from the current vN.N+Release+Notes page and links from the Software+Download page.

Update vN.N+Release+Notes page

Generally roll notes from the oldest release off and add notes for the new release. Keep previous release notes in there, so that we always have current + previous.

Update Software+Download page

https://spaces.at.internet2.edu/display/Grouper/Grouper+Downloads

Make sure you have copied this to the archives page per above
freshen links and rows in software download table
freshen cvs info at bottom of page
update release date

Review product pages

Ensure that each page identifies which version it's current for, as in "Building the Grouper API as of v1.4.1" as an h2 at the top.
Review incoming and outgoing links info for each page.
If there's new features with new pages, update the Grouper+Product page to appropriately reference the new pages.
Is it all there?
Are there pages in the Working Group or Community areas that have graduated to being core product doc? Negotiate with originator to move them over to the product pages.

Update GrouperWG/Home

Just update the "NEW!!" message on this page. Maybe review the Background section to see if it could use some freshening. Maybe add a news item

Review Training Videos

Determine which Grouper training videos need to be updated. Are new training videos required?

Review Impact, if any, on TIER Grouper Deployment Guide

Consider if the new release should lead to any changes/updates in the TIER Grouper Deployment Guide
if yes, discuss with Bill Thompson, Lafayette College

Update Grouper Wiki Documentation

Update the Grouper Downloads page

Update the spec sheet
Update the Grouper Deployment Guide

Update wiki pages for all changed features.
Add a new selection to the Grouper Admin Guides TOC if needed. See here https://spaces.at.internet2.edu/x/UAfw
- Be sure that Grouper wiki doc pages for new features are moved from the Development Items are and area and instead linked from the Admin Guides area.
Review the Glossary, remembering this could be a first place new people go to learn important terms. It seems rusty to Emily but not exactly sure how to improve it. HELP appreciated.
Be sure that wiki pages on new Features are linked correctly from the Admin Guides TOC page

Release Numbering and Testing Period

As of Feb. 2012, these procedures were adopted regarding release numbering and testing:

- The X.X.0 release will be considered a pre-release, like a Release Candidate
- That will be the testing release
- The X.X.1 release will be the actual release, presumably after any initial bugs identified have been fixed.

Packaging team

Talk to the InCommon Trusted Access Platform packaging team to see what their plan is

Notify about the release

Compose email to grouper-dev@internet2.edu and grouper-users@internet2.edui and grouper-announce@internet2.edu with highlights of the new release and link to the Grouper Downloads page
email to mw-announce [at] internet2.edu email list
Highlights should resemble those on the vN.N Release Notes page and maybe even be identical.
If it's especially enlightening, also include a direct link to the changelog (vN.N+Release+Notes#vN.NReleaseNotes-changelog) or to an appropriate Jira report.
Change jira admin to make that version released, and if the next build or version isnt there, add it

Other Outreach

update Grouper website features page to highlight newest Grouper features
Social Media: Announce the new Grouper release on the InCommon and Internet2 Facebook and Twitter accounts as appropriate (Jessica Coltrin coordinates InCommon social media as of April 2020)
work with Jessica Coltrin (and possibly Sara Aly, Internet2 Communications Manager,) around other possible publicity/press releases etc.
A Grouper Team member (DaveL for the Grouper 2.2 release) will create a Youtube video (using same tools as the Grouper Training videos) highlighting new features
- promote this video in coordination with the Internet2 Marketing and Communications Team
schedule a webinar to promote the new release

Publishing Maven jars to Sonatype (Maven Central)

Make sure you are at the official release commit, as it's the only commit without a -SNAPSHOT version number

git checkout GROUPER_2_4_0 (warnings about detached HEAD are ok, just remember to checkout an actual branch when finished)

...

language	bash

[mchyzer@i2midev6 bin]$ getGrouperSource.sh GROUPER_2_5_BRANCH
cd /home/mchyzer/tmp/grouperDownload/build_mchyzer/grouper-GROUPER_2_5_BRANCH/grouper-parent
mvn
cd ../grouper
mkdir -p dist/lib
cp /home/mchyzer/.m2/repository/edu/internet2/middleware/grouper/grouper/2.5.0-SNAPSHOT/grouper-2.5.0-SNAPSHOT.jar dist/lib/grouper.jar
cp /home/mchyzer/.m2/repository/edu/internet2/middleware/grouper/grouper/2.5.0-SNAPSHOT/grouper-2.5.0-SNAPSHOT-tests.jar lib/grouper/
touch conf/grouper.client.properties
cp misc/grouper.hibernate.example.properties conf/grouper.hibernate.propertiescp misc/subject.example.properties conf/subject.properties
emacs conf/grouper.hibernate.properties
echo "abc123" > conf/morphString.properties
mvn dependency:copy-dependencies
mkdir -p lib/grouper
cp target/dependency/* lib/grouper/
./start-hsql.sh &
./bin/gsh.sh -registry -runscript
./bin/gsh.sh -test -all

Code Block

ps -ef | grep java   --- kill hsql
wget http://www.internet2.edu/grouper/release/x.y.z/grouper.apiBinary-x.y.z.tar.gz
cd grouper.apiBinary-2.0.0
cd bin
java -cp ../lib/jdbcSamples/hsqldb.jar org.hsqldb.Server -port 9001 -database.0 file:grouper -dbname.0 grouper &
./gsh.sh -registry -runscript
./gsh.sh -test -all

Try the UI with a few different languages in the browser request (en without US, french, something not common)

Try checkConfig tests.

Upgrade from x.y-1 to x.y (e.g. 2.3 to 2.4).

Install 2.3.0
- Download and run 2.4 installer
Open the UI, browse around
Upgrade to 2.4.0: v2.4 Upgrade Instructions from v2.3
- Download the installer

Open the UI, browse around

Install the grouper installer

Download grouper installer
Run: java -jar grouperInstaller.jar
Try the UI that was installed
Make sure gsh.sh and gsh works in api, ws, ui for 2.4.0

Database	Linux	OS X (10.6)	Solaris	Windows
HSQLDB	CH
MySQL
Oracle
PostgreSQL
MsSQL (NO LONGER SUPPORTED)

Web Service JUnit Tests (Grouper WS and Grouper Client) (DONE)

WS samples (DONE)

WS javadoc (generate, commit, test). Make sure new operations / args / etc are documented in the WS doc page (TODO?)

UI internationalization tests (TODO?)

Test API

Code Block

# try to see if someone else is testing, look when process started, wait if so or email list...
[appadmin@i2midev6 ~]$ ps -ef | grep gsh | grep test | grep all

getGrouperSource.sh GROUPER_2_4_BRANCH
cd  /home/mchyzer/tmp/grouperDownload/build_mchyzer/grouper-GROUPER_2_4_BRANCH
cd grouper
./start-hsql.sh

cd /home/mchyzer/tmp/grouperDownload/build_mchyzer/grouper-GROUPER_2_4_BRANCH/grouper
ant dist
./bin/gsh.sh -registry -runscript
./bin/gsh.sh -test -all

Packaging and releasing

ssh to i2mibuild.internet2.edu
cd to ~mchyzer

Update Software+Download page

https://spaces.at.internet2.edu/display/Grouper/Grouper+Downloads (DONE)

Make sure you have copied this to the archives page per above
freshen links and rows in software download table
freshen cvs info at bottom of page
update release date

Review product pages

Ensure that each page identifies which version it's current for, as in "Building the Grouper API as of v1.4.1" as an h2 at the top.
Review incoming and outgoing links info for each page.
If there's new features with new pages, update the Grouper+Product page to appropriately reference the new pages.
Is it all there?
Are there pages in the Working Group or Community areas that have graduated to being core product doc? Negotiate with originator to move them over to the product pages.

Update GrouperWG/Home

Just update the "NEW!!" message on this page. Maybe review the Background section to see if it could use some freshening. Maybe add a news item

Review Training Videos

Determine which Grouper training videos need to be updated. Are new training videos required?

Review Impact, if any, on TIER Grouper Deployment Guide (DONE?)

Consider if the new release should lead to any changes/updates in the TIER Grouper Deployment Guide
if yes, discuss with Bill Thompson, Lafayette College

Update Grouper Wiki Documentation

Update the Grouper Downloads page (DONE)

Update the spec sheet (DONE)
Update the Grouper Deployment Guide (DONE)
Update wiki pages for all changed features.
Add a new selection to the Grouper Admin Guides TOC if needed. See here https://spaces.at.internet2.edu/x/UAfw
- Be sure that Grouper wiki doc pages for new features are moved from the Development Items are and area and instead linked from the Admin Guides area.
Review the Glossary, remembering this could be a first place new people go to learn important terms. It seems rusty to Emily but not exactly sure how to improve it. HELP appreciated.
Be sure that wiki pages on new Features are linked correctly from the Admin Guides TOC page

Notify about the release

Compose email to grouper-dev@internet2.edu and grouper-users@internet2.edui and grouper-announce@internet2.edu with highlights of the new release and link to the Grouper Downloads page
email to mw-announce [at] internet2.edu email list
Highlights should resemble those on the vN.N Release Notes page and maybe even be identical.
Change jira admin to make that version released, and if the next build or version isnt there, add it

Other Outreach

update Grouper website features page to highlight newest Grouper features
Social Media: Announce the new Grouper release on the InCommon and Internet2 Facebook and Twitter accounts as appropriate (Jessica Coltrin coordinates InCommon social media as of April 2020)
work with Jessica Coltrin (and possibly Sara Aly, Internet2 Communications Manager,) around other possible publicity/press releases etc.
A Grouper Team member (DaveL for the Grouper 2.2 release) will create a Youtube video (using same tools as the Grouper Training videos) highlighting new features
- promote this video in coordination with the Internet2 Marketing and Communications Team
schedule a webinar to promote the new release

Release in sonatype

...

The Maven step will package all the artifacts, gpg sign them (creating *.asc for all the objects) then upload them to the Sonatype staging repository
...
If something isn't right with the staging repository, just drop it and the Maven build will create a new one with a different increment number.

Grouper Failing Tests

...

Workflow for container including CI

Build

add tag (GROUPER_RELEASE_a.b.c) and push

CI (travis)

tag triggers a build
download that zip archive from github (github makes archives available for any tag)
unpack archive
Parse the tag to convert GROUPER_RELEASE_a.b.c to a.b.c
abort if the version from the tag doesn't match version in grouper.version
update maven versions to the new version (`mvn versions:set -DnewVersion=a.b.c`)
build and deploy to staging
Mail committer that either the staged jars in Maven Central are ready to release, or any errors

INSTALL

input parameter to grouper installer is grouper version
download that zip archive from github (github makes archives available for any tag)
download the jar plus dependencies from maven
assemble the container

Grouper Failing Tests

Test	Owner	Status and date	Reason it was failing

See Also

Release steps for new build

...

Page tree

Page History

Versions Compared

Old Version 209

New Version Current

Key

Steps needed to make a Grouper release

Remove unneeded config

Change default github branch

Final todo's

Verifying cherry-picks from branch are in master (main)

Git Release Commit

Security Review

Security Review

Test an API patch

Packaging and releasing

Packaging and releasing : provisioning

Archive the current release

Update vN.N+Release+Notes page

Update Software+Download page

Review product pages

Update GrouperWG/Home

Review Training Videos

Review Impact, if any, on TIER Grouper Deployment Guide

Update Grouper Wiki Documentation

Release Numbering and Testing Period

Packaging team

Notify about the release

Other Outreach

Publishing Maven jars to Sonatype (Maven Central)

Test API

Packaging and releasing

Update Software+Download page

Review product pages

Update GrouperWG/Home

Review Training Videos

Review Impact, if any, on TIER Grouper Deployment Guide (DONE?)

Update Grouper Wiki Documentation

Notify about the release

Other Outreach

Release in sonatype

The Maven step will package all the artifacts, gpg sign them (creating *.asc for all the objects) then upload them to the Sonatype staging repository...If something isn't right with the staging repository, just drop it and the Maven build will create a new one with a different increment number.

Grouper Failing Tests

Workflow for container including CI

Build

CI (travis)

INSTALL

Grouper Failing Tests

The Maven step will package all the artifacts, gpg sign them (creating *.asc for all the objects) then upload them to the Sonatype staging repository
...
If something isn't right with the staging repository, just drop it and the Maven build will create a new one with a different increment number.