Child pages
  • 24-June-2020
Skip to end of metadata
Go to start of metadata

 

Attending 

  • Chris Hyzer, Penn, Chair
  • Chad Redman, University of North Carolina Chapel Hill
  • Shilen Patel, Duke
  • Carey Black, the Ohio State University
  • Vivek Sachdiva, independent
  • Jeff Williams - University of North Carolina Greensboro

  •  Emily Eisbruch, Internet2


Intellectual Property reminder: http://www.internet2.edu/membership/ip.html


 Grouper Action Items are here  

New Action Items

  • AI Shilen update this USDU wiki 
  • AI Chris and Shilen discuss performance and memory issues 

Discussion


   Plan for next Grouper release, Grouper Release 2.5.30

  • Wait for the the DDL?
  • Changes to database configuration to keep large configs and history
  • At Duke , need to update upstream packages every 2 weeks since Linux gets updated.
  • Best practice to release each month?
  • TomEE has a security release
  • Cut a Grouper release and include TomEE?
  • DECISION Grouper should probably release every 2 weeks
  • Plan a release in 2 weeks that includes the DDL 

Current work tasks, and next tasks

Vivek –Provisioning configuration

  • https://spaces.at.internet2.edu/x/CwBbC
  • Provision configuration from the UI
  • Add provisioning configuration
  • LDAP and SQL so far, there will be more added
  • Think about a view for later
  • To configure a provisioner you need an external system
  • If the external system is LDAP or SQL, you can reuse
  • If its a new web services or API we can make a quick external system with endpoints etc
  • Then make provisioner 
  • Some things are translated
  • Some things will be new, LDAP or SQL specific
  • For a new provisioner we can see what is common and what is new
  • Provisioner tells Grouper how things are mapped
  • What the config is to find things in target
  • Does not do provisioning
  • There will be new daemons
  • Go to Daemon config screen
  • To set up the provisioning
  • Grouper Loader Properties has some changes
  • Making things more generic
  • 3 things we can configure
  • External systems, daemons and provisioners
  • Want this to be easier to configure than a properties file
  • Should be intuitive
  • You can see all provisioners and focus in on one
  • Some Customizing will be possible
  • Don’t have to use the framework
  • UNCC use case - logic around whether a group is provisionable
  • Additional logic beyond using attributes
  •  Can  use a hook for do no provision until there is a member
  • Use a  subclass to speed up work
  • Logic to inherit
  • Check group size
  • Flag for is it provisioned or not

  • UI Task page https://spaces.at.internet2.edu/display/Grouper/Grouper+LDAP+provisioner+in+v2.5+tasks
  • Make a subject source
  • Connect from an external system
  • Chris and Shilen will start working on this
  • Make a sub image
  • Publish the image, have it in Dockerhub , could help for testing, and keep things more consistent
  • Have git project in grouper misc
  • Run the container and point tests to it
  • Maybe something already in PSPNG folder  
  • Chad can work on that
  • Simple group membership provisioner using subject ID
  •  There are switches to see how things are resolved
  • Vivek will do some refactoring 
  • Vivek and Chris will talk about next tasks
  • Use list of next UI tasks


Chris – Recent memberships, templates, composites ng

  • Finish blob tables

  • Chris working on recent memberships
  • recent memberships were configured before in a way that was not ideal
  • 3 settings
  • Multi valued
  • For loader config, better to mark group things going into instead of group coming out of
  • Loader job that is not a real loader job, could be confusing
  • Move this work out of the loader into  a separate area
  • Chris will post to the list to get input on the architecture
  • 3 new attributes
  • Hopefully switching from custom folders to etc folder as recommended in Grouper Deployment Guide
     
  • At U Penn  (Penn Groups) using a custom UI  / O365 helpdesk
  • Chris will share on the Grouper wiki

  • Way to put attributes on things to help with loader
  • https://spaces.at.internet2.edu/display/Grouper/Grouper+selective+loader+from+attributes+example
  • Goes in different direction from GDG, with policy groups and visualization
  • This is not built into Grouper now
  • Just an example of how to use Grouper
  • You would create your own attributes
  • Views on top so you can see in views
  • Could be the need for all these lists can be re-architected
  • Hard to troubleshoot
  • Should we enable more complex equations/ queries for loader jobs?


Shilen – usdu, daemon scheduling

  • USDU, moving attributes to members table is done
  • Found bugs and fixed them 
  • AI Shilen update this USDU wiki https://spaces.at.internet2.edu/display/Grouper/Unresolvable+Subject+Deletion+Utility+(USDU)
  • Member queries take time
  • Code pulls all member objects being used, for each one it resolves the subject, then resolves the member again to update subject attributes in the member table.  Not being added to flash cache. Needs to be reworked to be more efficient. 
  • USDU resolves groups that are members
  • But this does not need to be done
  • Does checks that are not relevant
  • Bad membership finder utility https://spaces.at.internet2.edu/x/XYbd  can clean this up
  • Member table not used for point in time
  • AI Chris and Shilen discuss the performance and memory issues 
  • LDAP issues can vSary depending on the service 
  • If not querying on well indexed field , it impacts performance
  • Issue Lacey is having


Chad – pspng troubleshooting

  • Chatted w Michael G
  • Full sync and incremental running at same time and interfere with each other
  • Suggestion for a flag to help handle this

  • Issue around deleting a whole folder accidentally
  • Perhaps look at point in time
  • Delete a provisioned group issue
  • One solution is wait for new provisioning framework to be released.

 Bill – ad hoc type

Issue Round up


InCommon Grouper Slack 

Ross W   do not see the shibboleth logs being saved to the host file system when there is a bind mount to /opt/grouper/logs, but -e GROUPER_LOG_TO_HOST='true' makes them disappear from the container. (Again 2.5.29)

Alex P problem with PSPNG. One of our provisioners to OpenLDAP does not support empty groups (the schema groupOfNames mandates at least one member). Creation works fine ... However, the deleting everyone out of the group in question causes problem 

Carey RE: Subject+API     Does anyone setup a Subject API ( to an LDAP source ) to be able to reference Groups? or just "person" objects?

Jonathan is there a trick to using /grouper-ws/servicesRest web api with this: https://spaces.at.internet2.edu/display/Grouper/Install+the+Grouper+v2.5+container+maturity+level+-1+quick+start ?  

Jeffrey C been noticing the following line in the logs for our Grouper 2.5.29 sandbox:

'Full GC (System.gc())'

Does the code call garbage collection in some cases?

Jonathan S  Is there a way to get all subjects from grouper via the web api?  

Carey     new feature 

  Navigate to your Member ( click on the link next to "Logged in as <your name here>" in the upper right corner) then select "Functions" --> Visualization.

it works for some Members on my test server, but it consistently fails for me.  

Chris Hy - when we talk about grace periods (and recent memberships), i assume subjects will not be included in the group if they are USDU deleted.  if they are simply unresolvable (and not deleted), then they will be included.  is that what we want or do we need more knobs and switches there?

Jeffrey C   may be a bug with setting the environment variable GROUPER_MAX_MEMORY with executing gsh. It doesn't seem to set the max memory.

Lacey Is the only way to change “Institute of Higher Education” to import or mount the grouper.text.en.us.properties? Is there a way to change it in the db? (edited) 

Andy Is there any documentation on tuning the ehcache sizes in Grouper?  

Andy  Our term rolled over today, and I was paged for high load average this morning when a lot of groups changed.

Chris Hyzer  hoping we can limp along with pspng largely as is (we are working on a couple issues), and migrate to the new provisioning in a couple months

Chris Hyzer when we communicate stuff about the new provisioning framework, please read and give any comments , when we have code to use, please spend some time testing it and giving feedback 

Andy   would love to help on the replacement LDAP provisioner.  That is incredibly important for us (probably most people).  We want the best possible performance

Erik What is a configuration in the UI that would allow me to display groups as entities with their full path?  

Lacey  Has anyone implemented JWT auth for web services in 2.5.29?

Christopher B access control question (bill?): i want to delegate creation of groups in a folder to a department...... 

Chris Hyzer    created the git branch GROUPER_2_5_BRANCH. 

Chris Hyzer   A school asked for an example of "dynamic groups", or "attribute based groups", or "more powerful composites".   https://spaces.at.internet2.edu/display/Grouper/Grouper+loader+from+attributes+example

Carey RE: Grouper+deprovisioning

   Are there any scale/practical limits to how many "deprovisioning" groups the system can handle?

   Are there any scale/practical limits to how many "groups/folders"  ( to be deprovisioned) the system can handle?

Erik C  sudden issue just popped up (running 2.4.0-a85-u53-w10-p12-20191223) in the UI, trying to delete a group from another group, and getting the red bar:

Jeffrey  docker question,   how do all of you manage setting up an interactive gsh shell.  

RichardF What controls subject searching in the generic upper right search box? I have a JDBC subject source configured, and I can't find users all of the ways I would like to through that search. 

Carey FWIW: I was able to finally "guess" at an issue I was having with a SQL_GROUP_LIST Loader job, and I want to share for those who might fall in the same hole.

Andy  question about changing the ehcache size for PITMembership back on June 13th.  Scroll back a bit for context.  I'm still seeing ehcache soft-lock warnings during "large" loader jobs, and I'm wondering if I should adjust the ehcache settings.

Chris Hyzer   another loader based on attributes example.   https://spaces.at.internet2.edu/display/Grouper/Grouper+selective+loader+from+attributes+example

Carey Is it "safe" to manually implement https://todos.internet2.edu/browse/GRP-1807 in a 2.5.29 container?  

Jeffrey    I just noticed that my grouper_ui docker service is constantly restarting. 

Scott K  using image 2.5.29. If I use the UI config editor to set something like ldap.personLdap.validateTimerPeriod in grouper-loader.properties and save it to the database, do I have to restart the WS image to “see” the change,  

Krishnan We are doing a POC for Grouper implementation at NCState University. Looking for a contact who has implemented Grouper integration with Google for google groups. 

Carey I would like to have a GSH script that can do an SQL query and return an Array (rows) of Arrays(strings) from a query.   

Andy    get the following warning in my logs when the pspng initializes:

Erik- Loader job question: I've got a bunch of loader jobs that nicely build basis groups using LDAP_GROUPS_FROM_ATTRIBUTE, so now I want to simply create ref groups on a one-to-one basis...

Carey - I have moved all my passwords to a "get it at run time method" and I am wondering if the morphString.key matters at all under those conditions.     

Alex  is there a good way to disable a changelog consumer from GSH?  

Andy  Has anyone else noticed that if you delete a pspng-provisioned group (in Grouper UI) without first removing the provision_to attribute, pspng does not delete the group from the target system?

NickR -  Been bashing my head against a wall for a couple days trying to get the string normalizer function from this wiki page working in our MariaDB instance: https://spaces.at.internet2.edu/x/7oXd

now have groups that represent organizational roles from the Federation Manager in our dev Grouper instance. This is step one of our plan to move the FM behind the Internet2 Collaboration Platform.


Grouper Users Email



JIRAs

Grouper WIKI updates


Next Grouper Call: July 8, 2020

  • No labels