Grouper Call Aug. 18, 2021

Attending

Chris Hyzer, Penn, Chair
Chad Redman, University of North Carolina Chapel Hill
Vivek Sachdiva, independent
Shilen Patel, Duke
Carey Black, tOSU
Steve Zoppi, Internet2
Emily Eisbruch, Internet2

DISCUSSION

Removing Grouper ALL from Config Assignement (Slack discussion from U. Arizona )

Removing Grouper All from Config assignments makes loader go faster
Good workaround
Chris: when loader creates group, there is setting in grouper properties give privileges
That takes time
If security group and loader
Can have inherited priv from a rule
And might have the same check…..
Grouper 3.0…. Future state
Optimize the database to handle that if possible

Grouper 2.6 Plans

Proposal, as Chris presented it on Grouper Slack

Strawman Grouper Roadmap

1. We stop work on 2.5
a. 2.5 will be our stable release
b. Security fixes
c. Monthly new image whether we need it or not with OS patches
d. High priority low risk bug fixes (few and far between)
2. Focus on 2.6
a. 2.6 is the new 2.5, move to this if you want new stuff
b. Security fixes
c. Monthly new image with bug fixes, enhancements, OS patches, etc
3. Pre 2.5 will be "unsupported". Everyone needs to get up to 2.5 or 2.6 before too long
4. 2.6 we will finish the new Subject Source adapter for LDAP and SQL
a. You can migrate to this in 2.6 or wait until 3.0
b. Note, it works in 2.5, but might be better to use from scratch and not migrate too... we will address this
5. 2.6 we will finish all things provisioning
a. You can migrate to new provisioning framework for LDAP, SQL, azure, box, etc, or wait until 3.0
6. Once we are done with these tasks (3 months?) we will start work on 3.0
a. We will continue to support provisioning in 2.6 (bugs and enhancements)
b. We will drastically change the database model of Grouper focusing on performance and reducing the storage needed
c. We will rewrite some core pieces of the code to accomodate these changes and update other things (e.g. caching and how privileges are computed in java)
7. 3.0 will take a while for a release (6-12 months)
a. 3.0 we will remove the legacy subject sources (need to migrate to new way)
b. 3.0 we will remove old provisioners (need to use the new provisioning framework)
8. When 3.0 is released, it will be the release with enhancements
a. 2.6 will be the stable release with no enhancements
b. 2.5 will be unsupported
c. Rinse, repeat

For 2.5.55 There are jiras including

Loader failsafe
LDAP to SQL sync
Proxy

If Vivek finishes test suite
Shilen does load testing
Also
Go thru JUNIT, test suite
Misc , Rabbit MQ if unused may drop connection, does not recovery, it stays failed, Chad has a fix for that
Do this in next week or two?
Before last week in August would be great
Otherwise month after
Doing monthly releases
2.6 same as 2.5.55
In 2.5 branch security work
Continuing doing provisioning into state where they replace existing provisioners
Try a provisioner
Go back to a different version of Grouper
Hopefully people in 2.6 and switch to provisioning
THen going to 3.0 not as big of a change
Support a subset of the code looming in Grouper
Once 2.6 is available, people show have Grouper pre 2.5 won't have reason not to upgrade
Provisioner fixes in next 6 months
Then work on Grouper 3.0
Grouper 3.0 will have a major change in how we use the database
To get things fast and efficient
Longterm, Grouper 3.0 will still support change log consumers and ESB consumers
Keep the way existing messaging works
For other stuff, we don’t need old, Box, etc.
New provisioning framework is better

Matt:

new Duo provisioner does not do all that the old Duo provisioner does
Yes, we need to compare new and old
There was some review of the Provisioner Framework
And there were some questions about why some things were done
Supporting only certain objects on both ends
Prefer target end to be open ended
Prefer outbound flow to target system not to be bounded
Chris: Grouper is not Midpoint, a complete provisioner system, we are concerned w Groups, people, and memberships
For what’s not there, we can add it, or keep the legacy provisioner
There is a way to support additional details for provisioning , using metadata
Matt: use case: wanted to back user set with an AD group and have a local user registered in application, get GUID out of AD
Chris : use subject link. Subject attribute, SQL link, or LDAP link
Subclass any class

Back to high level, next couple of weeks

Grouper Versions

Finish 2.5.55
Then 2.6
Makes sense
Good idea to stabilize Grouper 2.5
Matt: prefers semantic numbering,
Signals breaking changes
Before 2.5.55 is a breaking change
After 2.5.55 it is not
Some patch releases have been breaking changes
Carey: If there is not a difference between 2.5.5 and 2.6 , then why upgrade?
People need operating system updates
Keep operating system you deploy with up to date
Having 2 branches active at a time is confusing?

Chris:

In a lot of past minor version updates, since we have the patches, the fully patched previous version is very similar
Except a 3.0 type version
But for 2.5, there are many build numbers, the third number, it's hard to jump back and forth, there are upgrade steps
There are orgs that don’t update very often
Use more stable
Not everyone wants something as dynamic
No Grouper code updates except security fix
Two supported versions of Grouper
One stable version and one with enhancements, for those who want to kick the tires
Never want to make another patch to 2.4
Duke would go with the enhancement version likely
We will give guidance on how often to do Grouper updates
Penn does Grouper updates monthly, which has worked well
Whether you pick 2.5 or 2.6 you should update monthly
Duke generally updates every 6 months
Vivek: proposed versioning strategy is good ( Major / Minor patches)
Important is that the users understand the meaning of Grouper versioning
Chris: we have hybrid semantic versioning,
Try to create a standard vocabulary
Provisioning : better to use Grouper 2.6
GSH templates and other advanced features, use Grouper 2.6
If you want as is 2.5, if you want the more dynamic Grouper , go to 2.

Current Work

Vivek

Incrementals, recalcs
Give me everything from target side
Compare
Optimize as much as possible
Minimize recalcs
No need for group to be recalc , just look at membership
There were some bugs in the Framework, fixing those
Recalc is complex, a group can be recalc
If group not recalc , the memberships in side , some recalc and some not
Don't want to recalc unless need to
Going thru the memberships, not implemented before
Now with the code and the LDAP DOA that Shilen put in
We take a membership, as the DOA what its state is
If things removed from target…
Add null member
May throw error
Do recalc
Should sync up
A lot of confusing minute issues
Making progress
Messaging provisioner
Putting framework thru different situations to make sure things work
Vivek: provisioning framework, fixing issues
Hoping to stabilize it soon
Full provisioning works more easily
Incremental provisioning is more challenging
Recalc wiki page is helpful Grouper provisioning framework recalc logic
Codify with unit tests

Shilen

Grouper provisioning daemon running again
Updated query around optimization
Performance testing
Noticed a few issues, will follow up
Plan to upgrade to Grouper 2.6 when released
Chris: it will be helpful to get users using Grouper Provisioning Framework in production

Chris

Working with Vivek , team coding on provisioning
Working on performance issues
Will do unit tests for the upcoming Grouper release
Big issue at last Oracle security patches
Impacted Duke less
Grouper Training in 6 weeks

Chad

New Grouper Training container
Improving the mock data
Students in Grouper training can see more realistic and relatable data
Subject source example improved now
Reduced number of containers for training
Matt: suggestion present GSH template example around ???? during training
Get to base tables from Grouper UI
Chad : we give examples using LDAP
Chris: yes, good to give real world examples
Chad: we don’t do a lot in Training on how to create a loader job,
Would be a good thing to go into
Now we have more data for this type of thing
GRP 2852 , deleting an attribute assignment, if you click one time too many, you can delete attribute. Chad added a property in UI, where it's not allowed. Committed on Aug 9
In Grouper Container, made change , handling escapes and dollar signs