Attending
- Chris Hyzer, Penn, Chair
- Chad Redman, University of North Carolina Chapel Hill
- Shilen Patel, Duke
- Carey Black, the Ohio State University
- Vivek Sachdiva, independent
Jeff Williams - University of North Carolina Greensboro
- Steve Zoppi, Internet2
- Emily Eisbruch, Internet2
Intellectual Property reminder: http://www.internet2.edu/membership/ip.html
New Action Items from this call
- AI Chris will go over Shilen’s LDAP provisioning work with Vivek, short working session with Chris, Shilen and Vivek
- AI Vivek will merge 2.5 branch into provisioning branch
DISCUSSION
- Review AIs Grouper Project Action Items (Google Doc)
- Approve minutes
- Review AIs
- Agenda bash
Upcoming Training: Grouper School, Oct 13-16, 2020
https://www.incommon.org/academy/grouper/
- 4 half days
- Chris and Chad will be trainers
- Canvas will be used for content management
Current work tasks, and next tasks
Vivek
Grouper generic provisioner UI tasks
- DB configuration, provisioning
- Added config features
- Can see all properties outside of the base
- Can see history
- Large value entries , stored in different column
- Different options for different databases
- Not all have the clob
- In history table and normal table
- Externalized text is now in database
- Makes things easier
- All Database config ui changes are made
- Is there a case where you need to null something out in an override?
- Explain in documentation
- helpful if blanks mean same as not being there
- Can use JEXL expression as a workaround
- Was an issue in HSQL database
- Can see in the history
- Vivek will work on SQL provisioning next in same branch where Shilen works on LDAP provisioning
- Collaborate and don’t step on toes
- AI VIVEK will merge 2.5 branch into provisioning branch
Shilen
– provisioning, bad membership finder
- At Duke had some upgrade issues with 2.5.33 container
- Latest TOMEE environment variables were not getting passed
- Rebooting containers used to work, needs a fix now
- When upgrading attribute data from recent memberships, new queries Chris had created
- Recent memberships, query it performs is more “expensive” in using space for the query
- DBAs need to look at it
- For now Shilen runs it when nothing else running
- Used to have 1 query to join the attribute value data with the point in time data
- Different databases deal w time differently
- Two changes , supposed to make it easier
- 1 ) adding Grouper timetable, stores in database and
- 2 ) attributes that configure the recent memberships are SQL synced into a table
- Disappointing that those changes did not make things better
- Model where you take half the query and put in a table has generally worked
- Assuming it’s related to a missing column or index
- Shilen: there have been such issues in the past
- Chris: Point in time keeps history of members and groups
- Could have a group deleted and recreated, what should be in the memberships?
- Based on the names or based on the UUID of the group?
- It’s an edge case
- Makes queries more complicated
- PIT tables
- Shilen: PIT tables contain the UUID of the group and the name, but not old name
- Using the UUID makes sense?
- Chris: if you rename a group or delete and recreate, then there can be performance issues
- Suggestion to use a flag for do you want to include deletes and recreates
- MYSQL cannot be used on top of other views
- New feature on SQL sync
- Query instead of table
- Another view
- SQL sync take query against AVAL? view
- Overall loader query uses a view, maybe it should be a query in the loader
USDU issue in container v 33, attributes that used to be normal , but now are columns on grouper member table…
Grouper USDU v2.5+
- Deletes attributes
- USDU checks to see if attributes exist
- If you have old versions of the API and custom apps
- It recreates the attributes
- Shilen updated the USDU code
- It logs a warning (looks at version) or errors out
- Handles that issue
- If you upgrade UI but not Daemon..
- Difficult to run different versions of same thing against database
- Duke has custom apps, needs to upgrade those
- LDAP provisioning work
- Created a simple basic translator
- Can provision group to ldap in simple way
- With no linking
- Takes all the target group objects from Grouper and translates them for the LDAP
- AI Chris will go over Shilen’s LDAP provisioning work with Vivek, short working session with Chris, Shilen and Vivek
- Will move things back and forth, target system to Grouper
- Translates from Grouper to LDAP
- Does Comparision
- For Full Sync
- Michael found issues w Bad Membership FInder
- https://spaces.at.internet2.edu/display/Grouper/Bad+Membership+Finder+Utility
- Shilen added a change log consumer
- By default runs every minute
- Looks at membership adds and deletes
- Now it will get fixed in a minute or so
- This is for composite issues
- Assuming this is most of the issues
- There may be group set issues also
- Linking of group sets, things get missed
- Not critical for Duke right now
- It’s ok to get fixed nightly
- Can see in Loader logs
- Daemon logs show what it does
- Loader log shows adds and deletes
- Going forward, Full sync should not find composite issues , should be handled as they occur
- Something to handle in the future… missed this
- Carey: Flattened membership , only see effective nested result of that change
- Only see the 1st group getting added
- Someone who does not turn on flattened events is a problem
- Shilen: we allow non flattened events for membership, not remembering why
- Carey: if you don’t have configs set right, there are issues
- Be sure the dependencies are considered as we make incremental changes
- Issues around flattened and non flattened, gets complex
- Finding the missing links down the chain
- Multi thread when added
- Shilen: Group sets are not in the change log
- Works now for people with the default config
- Chris: suggested model where daemon that runs every minute
What should Grouper support?
Specsheet
Quit supporting MYSQL?- Must move to Shib 4 by end of 2020
- Related to supportability of library components, (JAVA8 for example)
- Must move to Shib 4 by end of 2020
- Shib project is aggressive on dropping support after one year or less
- Chad: Dig into it and find out why MYSQL does not support views
- Chris: it does not handle a view on a view on a view
- Does not operate efficiently
- How many versions going back should Grouper support?
- Guidance is important
- When 2.6 is released, support only 2.5 and 2.6?
Chris – Zoom connector and next release
Grouper Zoom provisioning
- Tasks for 2.5.32
- Tested for databases
- 2 new tables and some columns
- Hope to release by end of weekend
- Documenting the Zoom connector
- U Penn has Automated populations and Ad Hoc zooms
- Hope to consolidate and get people to go to main account where possible
- Loaders to make automated populations so you are in one group
- Automated populations for includes and excludes , different folder
- Ad hocs for includes and excludes
- A custom UI explains
- Hook up to Shib error page
- If you are not in a group that is allowed to go to Zoom, there is an explanation
- Grouper connector, some things loaded from Zoom
- Groups in Zoom are in Grouper for reporting reasons
- Roles from Zoom , user status from Zoom
- Can make composites from that data
- Troubleshooting, including making an API call to Zoom
- Chris will document the Penn approach
- Carey: does it deal w multiple accounts in one interest
- Chris: Loads sub account users
- Zoom API can talk sub accounts, but trying to get everyone in main account
- Duo sub accounts, you can only be an admin in one account
Chad – Azure, Misc
- New container handling approach is more efficient
- Daily report
- Default mail to address was Chris Hyzer
- This is Fixed in 2.5.34
- Daemon job visualization did not work in production
- Doing loader queries is sometimes slow
- Need to look at that
- Have a lot of loader jobs
- Hope to reduce that
- GSH upgrade , first thing you do is slow
- Needs user privileges for sysadmin group
- Thousands of calls
- Cache related
- AI Chris look at JIRA 2937 https://todos.internet2.edu/browse/GRP-2937
- Not using object types
- Create a new stem and it copies all existing
- Not efficient?
- Looks up object type marker and not caching that
- Chris: group creates are slow, perhaps use a change log consumer for that
- Increasing priority
- Chris: maybe cache all attributes
- Chad: no other performance issues
- Will be working w grace periods feature
- For next two weeks, looking at screen when delete an attribute def
-
- Chad : Attribute for auto create objects is helpful
ISSUE ROUNDUP
Grouper Slack
Zachary Running 2.5.29 when deleting a folder with dozens of groups with many users (up to 800k) through the UI, the UI pod stops working.
Chris Hyzer We have a fix for pspng going slow in change log for v2.5.34 (upcoming release).
Zachary how do I stop privilege inheritance to one group in a folder?
Erik I've scheduled my PSPNG full sync job to be once weekly, it almost always in the "ERROR" state on the all-daemons status page, because it hasn't run in the time window that it checks. Is it not considering my Quartz schedule in calculating the ERROR state?
Lacey having issues with privileges in an Admin security group for an app policy, using GDG template structure.
Erik I ran into this, because the inheritance is provided via rule, you have to run the rule daemon to get them to propagate.
Peter Is the provisioning functionality described in https://spaces.at.internet2.edu/display/Grouper/Grouper+LDAP+provisioner+in+v2.5 currently in 2.5 (2.5.29)?
Drew I'm looking through the rule use cases (https://spaces.at.internet2.edu/display/Grouper/Grouper+rules+use+cases) for the syntax for how to add a member to a group when the rule is applied.
Carey Rules question.... Email notification on flattened membership add to group
Can the "to address" ( AKA: ruleThenEnumArg0 ) value be a group name? (or a Jexl expression to get all email addresses for members of a group? )
Jeffrey Is there a time when older versions of web service calls will be unsupported.
Beth Docker container patch is working! We are creating, modifying membership, and deleting HiddenMembership Unified groups.
Looking at the Grouper o365 model, the group owner is not included in the API call (supported in the documentation, but we don't include it). In the Office 365 audit logs, there appears to be an operation to add a group owner to the newly-created group. However, looking at the group, there still doesn't appear to be an owner.
Is this something that is configured in association with the provisioner configuration? Is there a "standard owner" that is always associated with the Grouper-created group? (edited)
Ryan upgrading from 2.5.29 to 2.5.33 and we are seeing the follow error:
error: Problem converting JSP to string: /WEB-INF/grouperUi2/index/indexMain.jsp, Problem calling method indexMain on edu.internet2.middleware.grouper.grouperUi.serviceLogic.UiV2Main
Jeffrey Is there an official site that lists which versions of grouper is supported? I'm not finding it,
Chris Hyzer Maybe with the next release I would like to adjust that so only one previous minor version is supported.
Zachary about the ws built-in authn wiki page and this one. It is unclear to me that the action you actually need to take for a fresh-out-of-the-box Grouper to use the built-in DB for WS authn is to set grouper.is.ws.basicAuthn=true in grouper.hibernate.properties and set a password in the DB for the principal. While this information does exist between these two pages, it's non-trivial to follow.
Chris Hyzer in addition to the pre-announcement about supported versions I made at 2:18pm yesterday, I would also like to change the guidance of MySQL with Grouper. We have seen several MySQL performance issues with large deployments, so I would like to identify that in the specsheet as new deployers pick their database type.
Erin Murtha we have shifted the Grouper training dates by a week. The class will now take place October 13 - 16. Didn't get to sign up yet? No worries - there are still seats available! Register soon to take advantage of discounted rates. Thanks! https://www.incommon.org/academy/grouper/
Chris Hyzer Better custom templates in Grouper to automate tasks. Here is a synthesis of several past request for Grouper and an example of how it could be used based on how we use Zoom at Penn... in order to add a school to the main zoom account, we have to create a bunch of groups and add some to other groups... instead of editing and running a manual GSH script, it could be more UI based. And declaratively configured (no java no container rebuild).
Shrey I was trying to work on a minor issue with Grouper Attestation Feature related to the content of the email notification. The server url is configured in the setenv file based on different environments. Trying to get the server url value from the property file but it doesn't seem to recognize it and displays this below value as it is in the email body.
Carey Grouper daemon "other job" to run a script. Is it possible to get an example of a GSH script that would log "subjobs"? Or is that possible with the current implementation?
Sudheer v can we enforce restrictions on grouper folder naming conventions just like we can on group names?
WIKI Updates
- Grouper SQL provisioner in v2.5 tasks
- Grouper membership SQL provisioner in v2.5
- Grouper custom template via GSH
- Specsheet
- Grouper web services - authentication - built-in Grouper
- Grouper v2.5 customize container config files
- v2.5 Release Notes
- Grouper email notifications (new notes from Michael G and Carey)
- Bad memberships finder v2.5 update
- Grouper rules use case - Email notification on flattened membership add to group
- Grouper Zoom provisioning
- Grouper Azure provisioner
- Grouper Documentation Requests and Suggestions
JIRAs
- GRP-2936 delete large group can cause errors
- GRP-2935
USDU may not run after upgrade to 2.5.33 if old api recreates subjectResolutionResolvable/subjectResolutionDeleted
- GRP-2934
Create grouper_file table and store workflow and reports related files into it - GRP-2933
Find bad membership change log consumer to fix composites - GRP-2932
document container overlays the right way and fail if patches fail - GRP-2931
add allowedRequestAttributesPattern to ajp connector - GRP-2930
add zoom loader job for subaccounts - GRP-2929
add zoom to custom ui - GRP-2928
WS message send/receive move exchangeType and queueType to dedicated arguments - GRP-2927
Nesting privilege sets of groups ( instead of members of a group ) - GRP-2926
Privilage errors in application template - GRP-2925
recent memberships should not use a view which is built on top of another view - GRP-2924
add tomee option in container for address to listen on - GRP-2923
deleting folder in ui can crash container
GRP-2922
zoom connector load and provision status - GRP-2921
pspng should cache if groups are provisionable
- Re: [grouper-users] Grouper 2.2.x with MySQL 8.0.x errors, Baron Fujimoto, 08/06/2020
- [grouper-users] Replacing container self-signed certs, Hafer, Christopher G, 08/13/2020
- [grouper-users] lite-ui use case, Francesco Malvezzi, 08/17/2020
- Re: [grouper-users] lite-ui use case, Hyzer, Chris, 08/17/2020
- [grouper-users] REL-8 & PODMAN, T-Heetderks, 08/19/2020
- [grouper-users] Issues installing GROUPER on Red Hat 7, T-Heetderks, 08/19/2020
Next Grouper call: Wed Sept. 2, 2020