Child pages
  • 19-August-2020
Skip to end of metadata
Go to start of metadata

 

Attending 

  • Chris Hyzer, Penn, Chair
  • Chad Redman, University of North Carolina Chapel Hill
  • Shilen Patel, Duke
  • Carey Black, the Ohio State University
  • Vivek Sachdiva, independent
  • Jeff Williams - University of North Carolina Greensboro

  • Steve Zoppi, Internet2
  • Emily Eisbruch, Internet2


Intellectual Property reminder: http://www.internet2.edu/membership/ip.html

 Grouper Action Items are here  


 New Action Items from this call

  • AI Chris will go over Shilen’s LDAP provisioning work with Vivek, short working session with Chris, Shilen and Vivek
  • AI Vivek will merge 2.5 branch into provisioning branch 

DISCUSSION


Upcoming Training: Grouper School, Oct 13-16, 2020

https://www.incommon.org/academy/grouper/

  • 4 half days
  • Chris and Chad will be trainers
  • Canvas will be used for content management

Current work tasks, and next tasks

Vivek 

Grouper generic provisioner UI tasks

  • DB configuration, provisioning
  • Added config features
  • Can see all properties outside of the base
  • Can see history
  • Large value entries , stored in different column
  • Different options for different databases
  • Not all have the clob
  •  In history table and normal table
  • Externalized text is now in database
  • Makes things easier
  • All Database config ui changes are made

  • Is there a case where you need to null something out in an override?
  • Explain in documentation
  • helpful if blanks mean same as not being there
  • Can use JEXL expression as a workaround
  • Was an issue in HSQL database
  • Can see in the history

  • Vivek will work on SQL provisioning next in same branch where Shilen works on LDAP provisioning
  • Collaborate and don’t step on toes
  • AI VIVEK will merge 2.5 branch into provisioning branch  

Shilen 

– provisioning, bad membership finder

  • At Duke had some upgrade issues with 2.5.33 container
  • Latest TOMEE environment variables were not getting passed
  • Rebooting containers used to work, needs a fix now
  • When upgrading attribute data from recent memberships, new queries Chris had created
  • Recent memberships, query it performs is more “expensive” in using space for the query
  • DBAs need to look at it
  • For now Shilen runs it when nothing else running
  • Used to have 1 query to join the attribute value data with the point in time data
  • Different databases deal w time differently
  • Two changes , supposed to make it easier
  •   1 ) adding Grouper timetable, stores in database and 
  •   2 )  attributes that configure the recent memberships are SQL synced into a table
  • Disappointing that those changes did not make things better
  • Model where you take half the query and put in a table has generally worked
  • Assuming it’s related to a missing column or index
  • Shilen: there have been such issues in the past
  • Chris: Point in time keeps history of members and groups 
  • Could have a group deleted and recreated, what should be in the memberships?
  • Based on the names or based on the UUID of the group?
  • It’s an edge case
  • Makes queries more complicated
  • PIT tables
  • Shilen: PIT tables contain the UUID of the group and the name, but not old name
  • Using the UUID makes sense?
  • Chris: if you rename a group or delete and recreate, then there can be performance issues
  • Suggestion to use a flag for do you want to include deletes and recreates 
  • MYSQL cannot be used on top of other views
  • New feature on SQL sync
  • Query instead of table
  • Another view
  • SQL sync take query against AVAL? view
  • Overall loader query uses a view, maybe it should be a query in the loader


USDU issue in container v 33, attributes that used to be normal , but now are columns on grouper member table…

  • Grouper USDU v2.5+

  • Deletes attributes
  • USDU checks to see if attributes exist
  • If you have old versions of the API and custom apps
  • It recreates the attributes
  • Shilen updated the USDU code
  • It logs a warning (looks at version) or errors out
  • Handles that issue 
  •  If you upgrade UI but not Daemon..
  • Difficult to run different versions of same thing against database
  • Duke has custom apps, needs to upgrade those


  • LDAP provisioning work
  • Created a simple basic translator
  • Can provision group to ldap in simple way 
  • With no linking
  • Takes all the target group objects from Grouper and translates them for the LDAP
  • AI Chris will go over Shilen’s LDAP provisioning work with Vivek, short working session with Chris, Shilen and Vivek
  • Will move things back and forth, target system to Grouper 
  • Translates from Grouper to LDAP
  • Does Comparision
  • For Full Sync


  • Michael found issues w Bad Membership FInder
  • https://spaces.at.internet2.edu/display/Grouper/Bad+Membership+Finder+Utility
  • Shilen added a change log consumer
  • By default runs every minute 
  • Looks at membership adds and deletes
  • Now it will get fixed in a minute or so
  • This is for composite issues
  • Assuming this is most of the issues 
  • There may be group set issues also
  • Linking of group sets, things get missed
  • Not critical for Duke right now
  • It’s ok to get fixed nightly
  • Can see in Loader logs
  • Daemon logs show what it does
  • Loader log shows adds and deletes
  • Going forward, Full sync should not find composite issues , should be handled as they occur
  • Something to handle in the future… missed this
  • Carey: Flattened membership , only see effective nested result of that change
  • Only see the 1st group getting added 
  • Someone who does not turn on flattened events is a problem
  • Shilen: we allow non flattened events for membership, not remembering why
  • Carey:  if you don’t have configs set right, there are issues
  • Be sure the dependencies  are considered as we make incremental changes
  • Issues around flattened and non flattened, gets complex
  • Finding the missing links down the chain
  • Multi thread when added
  • Shilen: Group sets are not in the change log
  • Works now for people with the default config
  • Chris: suggested model where daemon that runs every minute 

 

What should Grouper support?

  • Specsheet

    Quit supporting MYSQL?
    • Must move to Shib 4 by end of 2020
    • Related to supportability of library components, (JAVA8 for example)
    • Shib project is aggressive on dropping support after one year or less 
  • Chad: Dig into it and find out why MYSQL does not support views
  • Chris: it does not handle a view on a view on a view
  • Does not operate efficiently

  • How many versions going back should Grouper support?
  • Guidance is important
  • When 2.6 is released, support only 2.5 and 2.6?

 

Chris – Zoom connector and next release 

Grouper Zoom provisioning

  • Tasks for 2.5.32
  • Tested for databases
  • 2 new tables and some columns
  • Hope to release by end of weekend


  • Documenting the Zoom connector
  • U Penn has Automated populations and Ad Hoc zooms
  • Hope to consolidate and get people to go to main account where possible
  • Loaders to make automated populations so you are in one group
  • Automated populations for includes and excludes  , different folder
  • Ad hocs for includes and excludes

 

  • A custom UI explains 
  • Hook up to Shib error page
  • If you are not in a group that is allowed to go to Zoom, there is an explanation
  • Grouper connector, some things loaded from Zoom
  • Groups in Zoom are in Grouper for reporting reasons
  • Roles from Zoom , user status from Zoom
  • Can make composites from that data
  •  Troubleshooting, including making an API call to Zoom
  • Chris will document the Penn approach
  • Carey: does it deal w multiple accounts in one interest
  • Chris: Loads sub account users
  • Zoom API can talk sub accounts, but trying to get everyone in main account
  • Duo sub accounts, you can only be an admin in one account
     

Chad – Azure, Misc

 Grouper Azure provisioner

  • New container handling approach is more efficient
  • Daily report
  • Default mail to address was Chris Hyzer
  • This is Fixed in 2.5.34
  • Daemon job visualization did not work in production
  • Doing loader queries is sometimes slow
  • Need to look at that
  • Have a lot of loader jobs
  • Hope to reduce that 
  • GSH upgrade , first thing you do is slow
  • Needs user privileges for sysadmin group
  • Thousands of calls 
  • Cache related
  • AI Chris look at JIRA 2937   https://todos.internet2.edu/browse/GRP-2937
  • Not using object types
  • Create a new stem and it copies all existing
  • Not efficient?
  • Looks up object type marker and not caching that
  • Chris: group creates are slow, perhaps use a change log consumer for that
  • Increasing priority
  • Chris: maybe cache all attributes
  • Chad: no other performance issues
  • Will be working w grace periods feature
  • For next two weeks, looking at screen when delete an attribute def
  •  
  • Chad : Attribute for auto create objects is helpful


ISSUE ROUNDUP

Grouper Slack

 

Zachary Running 2.5.29 when deleting a folder with dozens of groups with many users (up to 800k) through the UI, the UI pod stops working.   

Chris Hyzer We have a fix for pspng going slow in change log for v2.5.34 (upcoming release).

Zachary  how do I stop privilege inheritance to one group in a folder?  

Erik   I've scheduled my PSPNG full sync job to be once weekly, it almost always in the "ERROR" state on the all-daemons status page, because it hasn't run in the time window that it checks. Is it not considering my Quartz schedule in calculating the ERROR state? 

Lacey having issues with privileges in an Admin security group for an app policy, using GDG  template structure. 

Erik  I ran into this, because the inheritance is provided via rule, you have to run the rule daemon to get them to propagate.

Peter  Is the provisioning functionality described in https://spaces.at.internet2.edu/display/Grouper/Grouper+LDAP+provisioner+in+v2.5 currently in 2.5 (2.5.29)?  

Drew  I'm looking through the rule use cases (https://spaces.at.internet2.edu/display/Grouper/Grouper+rules+use+cases) for the syntax for how to add a member to a group when the rule is applied.  

Carey  Rules question.... Email notification on flattened membership add to group

Can the "to address" ( AKA: ruleThenEnumArg0 ) value be a group name? (or a Jexl expression to get all email addresses for members of a group? )

Jeffrey  Is there a time when older versions of web service calls will be unsupported.  

 

Beth   Docker container patch is working!  We are creating, modifying membership, and deleting HiddenMembership Unified groups.

Looking at the Grouper o365 model, the group owner is not included in the API call (supported in the documentation, but we don't include it). In the Office 365 audit logs, there appears to be an operation to add a group owner to the newly-created group.  However, looking at the group, there still doesn't appear to be an owner.

Is this something that is configured in association with the provisioner configuration?   Is there a "standard owner" that is always associated with the Grouper-created group? (edited) 


Ryan  upgrading from 2.5.29 to 2.5.33 and we are seeing the follow error:

error: Problem converting JSP to string: /WEB-INF/grouperUi2/index/indexMain.jsp, Problem calling method indexMain on edu.internet2.middleware.grouper.grouperUi.serviceLogic.UiV2Main

 

 Jeffrey  Is there an official site that lists which versions of grouper is supported? I'm not finding it,

Chris Hyzer Maybe with the next release I would like to adjust that so only one previous minor version is supported.   

Zachary     about the ws built-in authn wiki page and this one.  It is unclear to me that the action you actually need to take for a fresh-out-of-the-box Grouper to use the built-in DB for WS authn is to set grouper.is.ws.basicAuthn=true in grouper.hibernate.properties and set a password in the DB for the principal.  While this information does exist between these two pages, it's non-trivial to follow.


Chris Hyzer     in addition to the pre-announcement about supported versions I made at 2:18pm yesterday, I would also like to change the guidance of MySQL with Grouper.  We have seen several MySQL performance issues with large deployments, so I would like to identify that in the specsheet as new deployers pick their database type.   

Erin Murtha   we have shifted the Grouper training dates by a week. The class will now take place October 13 - 16. Didn't get to sign up yet? No worries - there are still seats available! Register soon to take advantage of discounted rates. Thanks! https://www.incommon.org/academy/grouper/

 

Chris Hyzer   Better custom templates in Grouper to automate tasks.  Here is a synthesis of several past request for Grouper and an example of how it could be used based on how we use Zoom at Penn... in order to add a school to the main zoom account, we have to create a bunch of groups and add some to other groups... instead of editing and running a manual GSH script, it could be more UI based.  And declaratively configured (no java no container rebuild).  

 

Shrey  I was trying to work on a minor issue with Grouper Attestation Feature related to the content of the email notification. The server url is configured in the setenv file based on different environments. Trying to get the server url value from the property file but it doesn't seem to recognize it and displays this below value as it is in the email body.  


Carey  Grouper daemon "other job" to run a script.  Is it possible to get an example  of a GSH script that would log "subjobs"?  Or is that possible with the current implementation?

Sudheer v can we enforce restrictions on grouper folder naming conventions just like we can on group names?


WIKI Updates

JIRAs

 

EMAIL

 

 

Next Grouper call: Wed Sept. 2, 2020


  • No labels