Child pages
  • 19-Feb-2020
Skip to end of metadata
Go to start of metadata

Attending 

  • Chris Hyzer, Penn, Chair
  • Chad Redman, University of North Carolina Chapel Hill
  • Shilen Patel, Duke
  • Carey Black, the Ohio State University
  • Vivek Sachdiva, independent
  •  Bill Thompson, Lafayette College
  • Jeff Williams, University of North Carolina Greensboro
  • Emily Eisbruch, Internet2


  Action Items

 Grouper Action Items are here  

Discussion

 Administrivia

Upcoming


 Grouper 2.5 release

Tasks needing to be done

  • Gantt chart (chad)
  • Misc ui page re-arrange
  • Maven working well
    • Library conflicts?  Yes, fixed!
  • Oracle jar not in public repositories, how to add?
    • Download jar and put in Docker file , put in op container. Lib
    • For Oracle users, will need special instructions
    • Same issue for any JDBC connection ( via loader jobs too )
  • Shilen, disable groups, there is a fix needed
  • If you disable a group it disables every membership including privileges
  • If not a Grouper admin and you disable group, you won’t be able to re-enable
  • Probably Admin Privileges should not be removed

  • PSPNG 
    • If  I don’t have access is it because I am not in a certain group? Has there been a backup? How to re-sync?  Takes a lot of digging
    • Pre sorting before provisioner has to process the event is wise
    • Have wanted a log for each provisioner for history
    • That’s hard to do with  the current raw logs
    • PLAN
      Vivek wrap up installer work
    • Shilen wrap up group disable work
    • Then Chris will work with Vivek and Shilen to get PSPNG improvements into Grouper 2.5 release

    • agrees  about need for more readable logs for end users 
    •  Users want option to sync Grouper stem to one of the provisioners
    • Group out of sync and want full sync in background, also at stem level
    • performance issues are important and urgent
    • We must do something in that area, even if it pushes out Grouper 2.5 release
    • Migration to new framework that Vivek wrote for PSPNG is more than a build # increment on a container so it should be included with Grouper 2.5
    • Chris looking at Changelog consumer / ESB listener consumer
    • Special changelog consumer to change messages so it’s a real java bean
    • If messages go to JSON you can encrypt them
    • Chris has been looking at workflow for ESB changelog consumer
    • Push tasks that PSPNG does up the stack
    • Look at messages , see if they are provisionable based on standard framework , see if full syncs or group syncs have happened we can pop some messages off queue
    • Are there edge cases Chris is missing?
    • Hard to troubleshoot
    • Would like to see retooling of ESB consumer workflow
    • Then look at PSP NG to see what we can tie into
    • Instead of PSP NG doing its own checking to see if something is provisionable
    • Instead of checking each event,
    • Rely on upstream framework and do batching there
    • For real time ESB events or for Full Sync
    • Discussion last week w Shilen , Bert, Vivek, 
    • Perhaps use Shilen’s approach of full syncs
    • For Grouper 2.5, use new ESB changelog approach
    • Matt: agrees with this general approach
    • Can’t get rid of change log temp 
    • But might be able to enhance it
    • Only give provisioners events that they care about
    • Look at things we can do with batching 
    • Good to know when there is a large queue
    • Eventually  would like see  if a concept of user sync makes sense, for greater efficiency
    • Right now there is a group sync
    • Could  have a general provisioner page that shows what is in the queue
    • Automated threshold to get a notification
    • Vivek: keeping systems in sync and Grouper maintaining info on what has been synced and what is configured to be synced
    • What about problems when one system does not acknowledge receiving ?
    • False positives
    • Things can get out of sync
    • If you are going to make a call to a target , you should get feedback
    • Logic is used for failure situations
    • JDBC syncing also does logic for failures
    • At Duke, some cases where  data must be in LDAP immediately
    • Vivek : should there be a system between Grouper and LDAP?
    • A queue ,   answer: that is what the change log is
    • Jeff thoughts on PSP NG: 
    • Unit tests for PSP NG   
    • Jeff will let us know 
    • Run unit tests for PSP NG or create new ones?
    • Bert has created script based GSH based tests for PSP NG
    • Shilen  needs to take a look at what is there now
    • Perhaps convert to JUNIT 
    • Get info from Jeff? 
    • Shilen will get into the code first
    •  
    •  
  • Chris, Plan is that from UI, click on user or provisioner and see in database what has happened recently
  • Keep log of when last group syncs were 
  • There are several types of syncs
  • Should have improved logs and reporting mechanisms, button you push
  • Sync button could compare in the UI space 
  • See what is out of sync and send message to change log consumer
  • Good to have button on a member or group or object to sync
  • Not sure about the folder level, table this for now, not for 1st release
  • Info stored in database so it can be seen in UI
  • Jeff: Sometimes CIOs want a report or a log, don’t want to go to UI
  • Logging in JSON may be easy lift
  •  
  •  Bill T  - if Grouper product is going to have support for this provisioning, it should work fast
  • In simple cases it needs to work
    • Simple = deploy to LDAP and AD
    • Though there can be complexity


  • At Lafayette, looking at cloud to handle some of the issues
  • Big lift for Grouper  to reproduce what Rabbit MQ offers 
  • Chris: useful to have some logic in UI , to potentially keep user informed, so it’s not all done by messaging
  • Chris : would like Grouper to be easy to use for provisioning
  • Integrate sync with Grouper provisioner
  • Lafayette will upgrade to Grouper 2.5 and then will provide feedback
  • Chris: use case from Grouper Training : how to easily send this group to this LDAP DN and that group to that LDAP DN
  • When you mark a group as provisionable, have an attribute for the target path
  • Provisioner for main OU and another provisioner for ad hoc
  • Can use threads 
  • Prioritize to avoid backlog
  •  Hope that more institutions will migrate to PSPNG after Grouper 2.5 release

Cron Job testing (Chad)

  • Chad   working on the cron job unit testing ,  using HSQL , loader and sync table, set database driver to postgress, then every test after does not work.
  • There are Grouper properties to skip those.
  • Chad will try that.  Mostly runs. But not accurate representation ,
  • Error in the tests , can’t find the root folder, can’t access the database, because of the driver issue.
  • We  do want to do HSQL. it is an easier life , running postgress involves Docker
  • Or run postgress database and create  new databases and delete them at end
  • Grouper loader test , issue at line 147
  • Overrides in config come out after every run?
  • Should start from blank slate
  • Should test every database
  • Have a dedicated database scheme for testing each branch
  • Schedule tests of branches to not happen at same time
  • Chad created alternate port
  • Create a quick docker empty postgres database and connect  to it and then kill it when tests are done, to eliminate need to delete files
  • Just start postgres container
  • Still  ports must be unique
  •  Sending summary every night or only if errors? 
  • Use email with attachment
  • Or email with summary with  a link to a file 
  • Will expose the directory
  • To see the last week of runs

Current work tasks, and next tasks

Vivek – Building and packaging

  • Build  container is done. 
  • Allows creating a web app directory containing Grouper UI, web services, etc. 
  •  Runs under Tomee .
  •  The intent is for Chris Hubing to build containers. 
  • Now working w  Chris Hubing on building the container, 
  • Plan to be done by end of this week
  • Chris Hubing is making container, then next step is Vivek working on how to install that container, needs to be easy as possible for the campuses
  • Getting version # out of manifest of jars - not done yet
  • Chad look at this issue?
  • Grouper, Grouper client and web service needs to know its version
  • Chad adjusted that in manifest and jar a year ago, possibly not the misc projects
  • Might only need for client
  • If we assume someone running the container
  • May want to check other places...
  • Diagnostic

Chris –  SQL sync, bugs

  • See question re expression language (below)
  • Filter out what you don’t need like SOR groups
  • Filter by policy group
  • Worried someone is doing expression that won’t be covered by new approach

=========

    • Chris Hyzer  (from Slack)
    •   We are going to change how PSPNG groups are identified for provisioning.  In order to accomodate everyone's requirements, I'm interested in the various ways people configure this.  For instance, at Penn, we just use the default, which means groupSelectionExpression is not configured, and it uses the provision_to and do_not_provision_to attributes.  If you do not use the default, can you please reply to this thread with your config (groupSelectionExpression, attributesUsedInGroupSelectionExpression, attributesUsedInGroupSelectionExpressionAreComparedToProvisionerName) or your requirements (if no one else mentioned it).  In addition to configuring provisioning attributes on folders and groups, I would assume we would allow an Expression Language to filter by name, and would also allow requiring types of groups (e.g. only policy). Does that capture it or do we need more?
    • https://spaces.at.internet2.edu/display/Grouper/Grouper+provisioning+in+UI
    • https://spaces.at.internet2.edu/display/Grouper/Grouper+provisioning%3A+identifying+groups+for+provisioning

==============


Shilen – permissions issues, other 2.5 tasks

Chad – maven builds

 Bill – ad hoc types

Jeff – pspng


Moving forward and handling JIRAS

 

  •  New Grouper 2.4 release in next couple of days
  •  Re JIRAs , will do bug roundup after Grouper 2.5 is released


Issue Roundup


Grouper SLACK


Feb 5: Andrew M - am I supposed to be able to set searchResultPagingEnabled on the pspng configuration properties?  

Feb 5 Andrew M - found a bug


Feb 6 Jeffrey C -Has there been any changes in the privileges evaluation?

Feb 6 : MG: - want to do LDAP to a table in the grouper DB - 

Feb 6 Andrew M  - group renames (name change), including moves, will always result in the deletion and re-creation of the group provisioned in the target.

Feb 6 Andrew M -The logic in processAnyChangesInGroupSelection() calculates deselectedGroups and newlySelectedGroups sets based on the hash of the group name.

Feb 7 Jeffrey C-  split up our WS from our UI instance of grouper onto separate servers. I know this is best practice but   you have the services combined in the non-production, why do you need extra servers to do the work that is combined today

Feb 7 Mathew B - Loader Job , groovy shell 

Feb 8  Jeffrey C - a search from the Grouper UI   takes a long time

Feb 11 you cant rename root

Feb 12 Justin R- container issues 

Feb 14 Alex P -  purging no-longer needed PIT and Audit data

Feb 15  Chris Hyzer -We are going to change how PSPNG groups are identified for provisioning

Feb 15 Jordan  D - setting inherited privileges with the web services for some automation. It looks like I'd need to set 9 attributes for each

Feb 17 Jeffrey W: Ldap Loader : I have an LDAP_SIMPLE loader type that seems to work, but I'd like to specify the target group instead of it populating the loader group.

Feb 18 Michael G - Grouper integer on IDs issue

Feb 18 J Crawford - changelog temp to changelog performance

Feb 19 - Changelog consumers have next start time in the past.




Grouper Wiki updates


 


JIRA since Feb. 5, 2020

Next Grouper Call: Wed. March 4, 2020

 

 

  • No labels