Grouper Working Group Notes of Jan. 5, 2022

  Attending 

  • Chris Hyzer, Penn, Chair
  •  Shilen Patel, Duke
  • Chad Redmon, UNC
  • Carey Black, Purdue

 

New Action Items

  • AI - Chris look at making a script for Stem View Privilege performance to identify people who don’t need to be recalculated.  And a group to keep these people always calculated. This is helpful for certain power users who are always logging in. 

  • AI - Chad - make Maven change to generate swagger on demand only

  • AI - Chad discuss with Hubing needs for CI/Jenkins 
    • JDK8
    • PGP maven keys

  • AI - Chad update CI wiki once everything is finalized and works

 

DISCUSSION 

Current Work

  Vivek and Chris (as reported by Chris)

  • Google provisioner
    •  replacement for change log consumer, has a full sync
    • Has a mock server
    • Does not use Google SDK
    • Wiki not done yet.
    • Like Azure provisioner, there are group setting when you create groups
    • Those most likely to be customized
    • Can have defaults, can be overridden by metadata
    • See posting by Chris Dec 26, 2001 on Google core slack
    • Longer list of group setting s possible
    • Question: for settings that exist in change log consumer but not in new provisioner.?
    • Answer: we could add something to address that later



  • Stem View Privilege performance 
    • This is working, there is incremental and full daemon
    • Mysql was challenge
    • Chris tested:  loaded database with  Shilen’s script, with 20K groups, added privileges, 
    • In Grouper 2.5 if you turn on privileges, got slow response times, about 5 seconds
    • First time you go to calculate privileges, it will be slower
    • But after the 1st time, it will be handling incrementals and will be faster
    • This is for the 1st folder you go to
    • If you have a week  (that is configurable) of inactivity with Grouper, it will be slower again
    • There is a cache table
    • Good start, we can iterate on it.
    • Lesson learned: populate cache table using insert, select, delete queries, so don’t need to bring all back to java.  but  with mysql , insert with select locks the whole table and may cause other inserts and deletes to throw errors. So run in a loop . Oracle and postgres should not have this issue
    • Question: Is the only option to cache privileges based on   time last used? Or are there options, such as everyone in group always gets recalculated?
    • Chris: Can have a group where everyone can see all folders
    • Can change the default from one week of inactivity, to one month of inactivity, or one year of inactivity
    • Considered having a group of people it keeps, would be easy to implement
    • Do this for the users who come here pretty often
    • Document how to make a script to mark a bunch of people as “don’t need to recalculate”
    • AI: Chris look at making a script for Stem View Privilege performance to identify people who don’t need to be recalculated.  And a group to keep these people always calculated. This is helpful for certain power users who are always logging in.  
    • Noted that Duke and UNC don’t use mysql , so the slowness should not be an issue

 

  • Folder Privileges
    • There is now a view privilege
    • Like groups
    • Imply view

  

Chris

  • Loader Failsafes
    • Upgrade instruction talks about folder privileges and one that talks about UI changes
    • Hope to add standardization to UI menus  , including dropdown menus
    • Will note this in upgrade steps
    • Chad: Audit logs in different place and this can be confusing
    • For loader: adding failsafe options
    • Global defaults can be customized for each job
    •  There’s a new database table to track failsafes that have been triggered and approvals
    • Across all different types of jobs (loader, provisioner, or USDU)
    • Tracking and approvals done in one central group failsafe table
    • Chris will be fleshing out the UI for failsafes
    • Question: can I see a graph for job with loader errors?
    • yes
    • There is a method to mark different things as errors
    •    This can help with colors on graphs
    • Chris will work on group edit screen

  

Shilen

  • Next thing, new columns on member table: subject identifier1, 2, email0
  • Perhaps add email1 later
  • In the future add loader columns perhaps
  • But don’t have subject API query
  • It assumes they are in that table
  • Shilen added code for no duplicate subject identifiers
  • Version 3: subject sources in the database?
  •  

 

Chad

  • CI / jenkins
    • Githooks - dont know which branch or commit
    • Get the tag from git
    • New process, tag, and then go to testbed server and paste tag
    • JDK8
    • PGP maven keys


    • Travis could use apis to do creds and os’s etc
    • Jenkins is more DIY
    • Work with sysadmin (Hubing) to setup jvms and creds
    • Source code is not where jenkins file is
    • AI - Chad meet with Hubing and discuss  needs for CI/Jenkins 
    • AI - Chad update CI wiki once everything is finalized and works


  • Log4j upgrade
    • Difficulties with log4j properties file
    • Must change properties file to log4j2 format
    • System property to pass to use API bridge
    • Previous work was just in TomEE in container
    • Log4j_1 is still in Grouper
    • Log4j_1 is at End of Life
    • First attempt was API bridge
    • Change APIs to use log4j2
    • Container / GSH would set system property
  • Swagger JSON file line endings
    • Make the swagger generation a manual step
    • Clean compile swagger:generate
    • Check line endings when committing
    • “Docs” folder might not be in release container
    • AI - Chad - make Maven change to generate Swagger on demand only

  

  • Matt noticed some Log4j issues with remote config

 

Container OS

  • Rocky Linux?
  • More secure Linux
  • Redhat to debian ? more changes
  • Go Distroless? https://github.com/GoogleContainerTools/distroless
  • Small footprint
  • Not a complete OS, requires the outside kernel to do more things
  • You minimize other dependencies outside of the container
  • Tomcat running grouper, Shib SP on another container for password protection. Then proxies Grouper calls to other container
  • Perhaps possible for Grouper v 3
  • Need to keep Grouper easy to use but also keep security in mind
  • Perhaps open this discussion to broader community
  • Consider Docker Compose? 
  • Chris interested in the Rocky Linux model 
  • Shib SP versus straight Java SAML
  • Parsing XML, don’t need extra processes
  •     Unicon did work on this

 

Excited to release Grouper 2.66  Grouper Release Announcements

 

Next Grouper Call: Wed. Jan. 19, 2022

  • No labels