Child pages
  • 22-Jan-2020
Skip to end of metadata
Go to start of metadata

  

 

Attending 

  • Chris Hyzer, Penn, Chair
  • Chad Redman, University of North Carolina Chapel Hill
  • Shilen Patel, Duke
  • Carey Black, the Ohio State University
  • Bert Bee Lindgren, GA Tech
  • Vivek Sachdiva, independent
  •  Bill Thompson, Lafayette College
  • Jeff Williams, University of North Carolina Greensboro
  • Matt Wolfley, Unicon
  •   Action Items

 Grouper Action Items are here

 

DISCUSSION

  1. Administrivia


LOB discussion

  • In Grouper 2.5.0 patches we will support BLOBS in the database
  •   for Grouper 2.5 Large Object Database Type
  • https://spaces.at.internet2.edu/display/Grouper/Grouper+lob+database+type
  • Framework takes it out of database and uses Temp directory in case it takes up a lot of space
  • Could have a UI to help manage the table\
  • For 2.5 put this in API, eventually in UI
  • Vivek: issue with slowness in a CLOB he used
  • Suggestion: Only leverage CLOB for over 4000
  • Chris: could do this for attributes. 
    • Chris will think about this
  • Q: extend this to group descriptions? So it would be possible to use any generic string and overflow from short to long storage,  
  • Consider String for attributes, or lob as type.  Consider string / lobs

Bert

  • Performance of pspng will be priority after Grouper 2.5 is released, in a month 
  • Grouper Provisioning: PSPNG
  • In 2.5 patches, PSP performance will be a focus
  • For many sites with medium PSP provisioners, it can take a long time on change log and full sync
  • If Bert has time to help in about a month from now, that will be great. 


  • Bert  has worked to get Grouper 2.4 instrumented
    • Objects nest in tree structure
    • Full full sync and group full sync
    • Chain of instrumentation
    • Output on special performance log
    • Time threshold (10 seconds)
    • If takes more, then it summarizes itself
    • Grabbing those objects, full sync time can be saved as an attribute
    • To capture performance info
    • Applying this approach  to other places in Grouper would be helpful
    • Two phases: short term gains from this experience
    • Remote workshop for 2-3 hours with Chris, Jeff, Shilen, Bert
    • To see how to integrate this with direction for other provisioners
    • To increase performance, to increase data gathering around performance
    • May focus on this in 1-2 months
    •  
    • Working with Gettes
    • Will do some R&D in next few weeks and apply that to Grouper 2.5
    • Should use a library (perf4j)?
    • AI Bert Commit instrumented approach to branch from 2.4
    • [AI] Bert create wiki draft of instrumented approach
    • Could have per group stats?  Perhaps

Containers Strategy for Grouper 2.5

  • https://spaces.at.internet2.edu/display/Grouper/Grouper+packaging+and+versioning+strategy+for+2.5
  • Have a config that says auto-upgrade to 2.5, and container would auto-upgrade
  • Chris and Chad talking about how to compile JAVA
  • Whoever is ready to make a version of Grouper will put a tag on a branch and that will have a certain format
  • There will be a tag on a branch
  • A process will  publish Grouper to Maven Central (Chad working on this now)
  • To get the container to Docker Hub, in Internet2 Grouper that does the packaging they’ll change Docker file to say what version they want. They have CI / CD, to check out from GIT and run the installer.
  • New Installer Task for the packaging managers
  • Downloads jars from Maven, uses GIT files that have been exported, runs script or Java commands, or something to Merge into one folder, a web app. 
  • So no compiling, no tarballs, no patches
  • That container is listed as a release candidate, it can be tested, and deployed, then it will become the latest version. If a problem with it, we will move to the next number Grouper 2.5.8  can increment to Grouper 2.5.9
  • Working to get UI WS Daemon and SCIM running in one directory
  • There are still separate projects in GIT
  • Ant will be removed
  • Strategy to run a web  app in GIT
  • Bill Thompson has been documenting that
  • May need one small Ant task for 3rd  party dependency
  • Maturity Level 0 is running a container on a server
  • Maturity Level 1 , you might have a Docker File and can externalize config in other files, but still pwd and logs on server
  • Maturity Level  2, you have everything externalized, have pwd manager
  • Maturity Level 3, you are using orchestration
  • Maturity Level 4 might be working in cloud

For Maturity Level 0 , 

  • authentication can be a stumbling block
  • Some use Tomcat authentication
  • Problem because there is no way to dynamically tell Tomcat to use that, must change server XML,
  • Must change Web XML
  • Would like it to be a config option

So for Grouper 2.5, basic auth in the servlet filter. In addition, have filters based on source IP addresses and better auth for Web Services


Has been discussed on Slack:  should Grouper have authentication or if it should always be externalized


Other local customizations might need to be in the model. 

What is the methodology for injecting binary jars into containers? 

Currently : 

  1. Low Maturity  Level: if you have the files on the server that’s running the container   you can mount the files into the container.

 When you fire up Docker, you mount a source folder from outside Docker to a file inside Docker. Exception is that Chris Hubing, Internet2, has some built ins.  (conf, lib?) 

2. Higher maturity: you have  a Docker file; libs are in your source control, when it builds container, it copies from your source control into your container. You deploy your container, Not mounting anything to external file system


discussion:

  • Managing Tomcat config is like injecting a Jar
  • Matt: trying not to make JSP changes, concerned about the hooks and change log consumers
  • Vivek adding authentication, removing tomcats,   
  • making a few changes to Data Model 
  • For Grouper 2.5  there will be a tables , including a table for recent logins
  • There will be a GSH way to set a pwd
  • Config that asks  which of 3 approaches you want, basic auth, UI web service, or SCIM 
  • Will hash and encrypt the pwd into the database
  • No more tomcat XML files
  • Later on, do Jason Web Token (JWT) approach in server and client
  • JWT is more secure
  • Authentication approach would be selectable on a per account basis, flexible
  • https://spaces.at.internet2.edu/display/Grouper/Grouper+authentication+in+2.5

     Summary of work to date:

  • Simplified GIT structure
  • Chad working on Maven
  • Got web  app running in one dir

  • Still need to do installer  tasks for container installer
  • Another installer task to cover maturity level 0, with a wizard
  • Question: how does this impact major upgrades w DDL changes?
  • Answer: Don’t have GSH outside a container anymore
  • So spin  up a container , could be maturity level 0, start GSH and do registry, as you do now. In future, might have a config option  to do this automatically
  • Ideally automatic with some human intervention
  • Shilen: for just index adds or new tables, it’s fairly safe
    • Don’t too often have restructuring data in ways you have to be careful, but we need to consider those situations
  • When we get to that , we can look at why UI is not starting and try to absorb some of those errors
  • Allow person to go to admin screen, authenticate a user and sort it out
  • Or   “Automatically upgrade to v 2.6”
  •   always able to run one version back
  • If new container is running in older database, then it updates
  • n-1 approach
  • Chad: 
  • For web services, need to support many web clients, ORACLE etc.
  • They have basic authentication built in
  • Need to support basic authentication as well as allow JWT and other methods
  • Can have self serve accounts for web services
  • Local entity concept w privileges
  • Generate pwd, either basic auth or JWT private key
  • That account will be set for that authentication
  • Question: Do you want to enable JWT at all?
  • What is model for changing pwd?
  • For web services, not going to allow people to set their own pwd. (similar to DUO)
  • If Grouper is going to offer authentication, how much do we need to provide?


Chad


    • Jars are in maven  as snapshots and for Grouper 2.4.0
    • See tag, Start  a release process,  push to staging channel, administrator would release those manually, that’s described  on the packaging wiki page as steps for developer.
    • Suggestion to have a separate branch and call it release.
      •  Poms for maven now are -snapshot
      • Check in code? Or just local?
      • Can’t build from source code?
      • Should have official version pom somewhere 
      • That’s  a common workflow, Chad has read about
      • One branch  called release, for every release it merges master and changes the version
    • DECISION:  we don’t need a separate branch. Maven is the source of truth. 
    • What gets built is local and goes away
    • Travis CI will parse and convert
    • No snapshots in Maven

    • Suggestion to ask Vivek to move his work to Master
    • Removing ant
    • Remove 3rd party libs from github
    • Remove client from Grouper
    • Publish instructions for how to develop that way
    • May be a couple of speedbumps but will work them out
    • How to run tests without Ant?
    • Use Java or GSH


Shilen

  • Permission change log changes (waiting for patch from Chris)
  • Updates for views for enabled flag on groups
  • Couple weeks ago finished propagation for attributes disabled when group disabled
  • Web services stuff that Vivek had started
  • Queries for M. Gettes made things worse for Harvard 
  • https://lists.internet2.edu/sympa/arc/grouper-users/2020-01/msg00002.html
    • Undo for next patch?  Yes this would be OK 
    • Try to batch up the queries?
    • It’s a syntax thing
    • Hibernate and branching capabilities,  Branch at hibernate level?
    • Shilen will do some testing and try to replicate the problem
  • What else for Grouper 2.5?
  • Show something disabled in the UI? This is for future.

Issue Roundup 

Grouper WIKI pages updated


=======


SLACK

Jan 8 - Sudheer   ; using pspng to provision grouper groups to LDAP and if group description has new linefeeds in it, The provisioner is assuming the first line as actual group description and the remaining lines as LDAP attributes
(this got  addressed)

Jan 8 - J Crawford : does permission changes show up in the audit log?  GRP-2544 

Jan  8 - Carey : possible to have a rule set to "act as" the user who triggered the rule….
Rules would let me "link one to many"  and "decide later to add another" ( condition, source, target, etc....)
Two new features: Ignore direct memberships from "g:isa" ( To allow nested groups)

 Inspect indirect Memberships and create direct memberships when they are missing. ( and do it as the user who added the user to the other group. )


Jan 9 - Jeffrey C and Greg H: UI problem (fixed in patch?)

Jan 10 -Jeffrey C - bootstrap a grouper structure for an application that is org based.   don't want it to manage users, just the groups and stems. trick in the loader to allow it to manage groups and folders but not the actual memberships? 

Jan 10 - Rachel -possible to create Attributes over GWS

Jan 10 - Lacey V - provisioning groups to Active Directory (via PSPNG). AD admins are insisting that the samaccountname attribute match CN and samaccountname cannot contain colons. They also  prefer a flat group structure in AD, for performance reasons


Jan 10- Carey - Q #1) Is there a way to restrict "normal users' from assigning "EveryEntity" privileges?

Q #2) Is there a way to restrict "normal users' from assigning "EveryEntity" as a member of a group?

Jan 10 -Carey - a user added EveryEntity to a group in their user folder. And suddenly all users could see the other users "user folder".  It looks really odd.. and took me a bit to find the reason. " Should everyone be able to do that? "  


Jan 14 - Alex P - PSPNG limitation, We're trying to setup an attribute provisioner to write <subject username>@<subdomain> to an attribute, where subdomain is pulled from a group attribute. The latter part isn't a problem, but it looks like subject information isn't being passed into the jexl parser for provisionedAttributeValueFormat.   (CHRIS LOOKED AT THIS, SEEMS SOME TWEAKS ARE NEEDED TO PASS USER TO WHERE EXPRESSION LANGUAGE IS EVALUATED. Passed per group or per subject?   Bert will take a look at this)

Jan 14-ChrisH - The way audits are stored is not conducive to querying.

Jan 14 - Anthony Hill - Install issues, container is better

Jan 15 - J Babb - Oracle to Aurora Postgres

Jan 15 - Haverkamp - pathways for an unaudited membership?  

Jan 15 - PaulR -Performance issues w mass full syncs between grouper and AD.

Jan 17- J Craford - JVM options on the different grouper types, ui, ws, daemon? 

Jan 17 - Shilen -formatting the queries differently significantly helped performance on mysql.  But it turns out that the new queries don't work well for them but the old ones do. See email https://lists.internet2.edu/sympa/arc/grouper-users/2020-01/msg00003.html

Jan 17 ChrisH -Shib impersonate with grouper groups for who can impersonate and who can be impersonated. See here

Jan 22: J Williams : Upcoming change – Microsoft to disable use of unsigned LDAP port 389

Jan 22 - Alex  P -issue with a rule: a problem where a group creation failed in the UI

Jan 22 - Sudheer - Can we migrate folders,groups and group memberships from one instance to another

=======

Jiras


GRP-2560 Consider a fake user for grouper loading groups with no members

GRP-2559 add sql other job to easily run sql commands

GRP-2558 grouper custom composite should not return groups

GRP-2557 when filtering by users who have update priv, should also return admin priv

GRP-2556 invitation: option to not try and add user to external subjects source if they exis

GRP-2555 rules dont fire for add member in ui when go from disable to enable

GRP-2554 all user attributes in jexl for pspng group attribute resolver

GRP-2553 ui not compiling with patch

GRP-2552 Visualization: error when user graphs and something isn't accessible.

GRP-2551 Change log events for permission changes on subject

GRP-2550 Upgrade quartz schedule to 2.3.2 (resolved)

GRP-2549 Audit does not capture all direct Membership adds

GRP-2548 PSPNG function to strip carriage returns from strings.

GRP-2547 Remove full-sync at startup option

GRP-2546 background color on recent activity not correct for wide rows

GRP-2545 attribute to mark attributesDef with attributeName values to not put in PIT

GRP-2544 Updates to permissions do not seem to be tracked in the audit log

GRP-2543 add a way to change the morph pass

======

Grouper Users list Emails


Next Grouper Call: Wed Feb. 5, 2020

  • No labels