Child pages
  • 27-May-2020
Skip to end of metadata
Go to start of metadata

 

Attending 

  • Chris Hyzer, Penn, Chair
  • Chad Redman, University of North Carolina Chapel Hill
  • Shilen Patel, Duke
  • Carey Black, the Ohio State University
  • Jeff Williams, University of North Carolina Greensboro
  • Vivek Sachdiva, independent
  •  Emily Eisbruch, Internet2


  Action Items

 Grouper Action Items are here  

 

 

Grouper School June 2-3, 2020, ONLINE

https://incommon.org/grouper-school-virtual/

  • More than 30 people signed up 
  • Chris and Bill will be trainers
  • Modernized the GTE, Grouper Training Environment
  •   container w MSQL , LDAP , SHIB IDP,
  • Fire all that up, install the Grouper database, run GSH commands, make a snapshot and make that container 
  • EC2 instance with different containers 
  • Shell script commands that do docker things
  • Have Grouper  training content for containers now
  • Content from wiki plus exercises
  • Maturity levels, what you need to run Grouper
  • Hands on work
  • Maturity level -1 quickstart
     

DISCUSSION

Should we  list  the web browsers that Grouper supports on the specsheet? Or add the version number? 

You have to be on this version or up? From Java script , standard with  JQUERY,  

There are issues with IE

Decided specsheet is  OK as is 

Current work tasks, and next tasks

Vivek

  • Grouper Data configuration
  • Refactering common parts in config and put in abstract class for other use cases
  • Adding more validation, such as read only
  • Pre built classes for jobs , don’t want them to be changed
  • So read only 
  • Properties are showing without text, so creating text
  • For read-only w Expression language, remove checkbox
  • Screen that edits a daemon, a job that has a class , you are never going to change class that runs, 
    • so can change schedule but not the class
    • Hard coded, Not regex

  • Will be done in a few days
  • Then Vivek will start working on provisioning
  • External systems and daemons are similar
  • Vivek made a super class for those
  • Base config class, thing specific, then sub classes
  • Use base config as a super class of the provisioning config
  • Think of what a provisioner is , configuration of a provisioner
  • Pivoting from PSPNG , due to performance and other issues
  • New Framework with Base class, some common framework things, using the sync tables in the database now, interface with changelog in form of ESB, ESB knows the provisioner is getting messages, will send in bulk,  
  • will say this is not a change, or this is not relevant. 
  • Important design moment, we all need to take time to focus and get requirements 
  • Then migrate current provisioners 
  • For Grouper 2.6, everyone migrates to the new provisioning framework
  • At a high level, provisioner has a bunch of options, will get all memberships,
  • Ways to write your own spin
  • Maybe framework 
  • Share a general framework plan on the wiki with Grouper-users
  • Pull in the customer base, ask which services most interested in for a provisioner
  • Should we do a survey?
  • CON ID Framework
  • Adapter that goes to CON ID later?
  • Chad - inherited a custom provisioning system, hope to move to new Grouper provisioning approach in next 6 months. 
    • Did work with Azure provisioner
    • Implemented midpoint as part of CSP
    • Currently 2 projects at UNC, management leading toward Midpoint
    • Also deciding   LDAP or AD
  •  Chris: for basic account work, Grouper can do it, and can do access part,
    • Midpoint is still a good provisioning engine
    • LDAP and SQL data moves, for fewer dependencies and higher performance, might use Grouper provisioning
    • But to Box or Azure, might use Midpoint
    • Even if you have provisioning system, good to have hooks for troubleshooting access, Grouper can offer read only operations
    • Provisioner can read things, 
    • Bridge to Loader
    • Reports on who has access
    • To help deprovision
    • Interactions  to remove could go thru midpoint , but Grouper could at least know
  • Matt:
    • Agree with the goals
    • In larger  ITAP scope, each component can stand alone 
    • If designing as cleanline interface, lower complexity of understanding Grouper interface.
    • Redoing Grouper for DUO was challenging
    • MAVEN has helped
    • Loosely coupled interface is good
    • Message client interface could be good
    • Suggest that the provisioning framework support attribute mappings from each Subject Source to a set of “provisioner attributes” for that provisioner.

  • Chris
    • 2 camps:
      • Change log consumers tightly coupled 
      • Change log consumers loosely coupled
    • Option to run in Grouper loader demon , or run separately 
    • As we go forward, for troubleshooting and caching, it’s increasing difficult to support loosely coupled
    • Those using containers say they can spin up a container for each provisioner
    • But this results in too many containers, too many projects
    • Better to use Grouper  daemon and have fewer containers
    • Shilen agrees, need to keep it simple
    • For diagnostics , we know every daemon is working
    • If using message queues instead of daemon, need to do more monitoring

    • self contained library
    • Can have a maven dependency on Grouper,  this is done at UNC
    • Single dependency for communication
    • Goal: for simple provisioner, such as BOX, it should be very simple, Grouper will figure out the adds for you
    • Data structures to be used for all provisioners 

    • Subject API for identity or for account
    • Mappings between subject API and provisioners, good way to genericize 
    • Chris: with Grouper now, no easy way to have multiple accounts for a subject
    • A subject is an entity
    • So now subject API is an identity API
    • Carey: advantages to subject API being an account
    • Correlation level above subject API (An “Identity API” could be added that correlates Subject API’s )
    • Chris: In training next week , every person is one subject , don’t have multiple subject sources, 
    • Chris will work on the planning documentation to move this forward, LDAP provisioning as initial push
    • Vivek: the vision makes sense


Shilen

  • Fixed issue w Grouper recent memberships , related to display name
  • Grouper V 2.5.29
  • Socialize that more since it improves how loaders manage groups
  • Improved how Enabling / disabling memberships work, 
  • UI issues w immediate membership attribute assignments
  • Attributes on  memberships at Duke: mostly for card system, keep track of some attributes, use comment field
  • Fixed Issues w inheritance and resources and actions, 
  • Shilen will work on the USDU issue, 
  • Add column or add attributes?
  • Go straight to members table
  • Consider if subject is deleted don’t return it
  • If limbo, do some logic
  • Adding column is better to avoid performance issues
  • Flag for unresolvable and flag for deleted
  • Do we need to add timestamp, flag, or both?
  •  For PIT, don’t need to make a change

Chad

  • Wrote a script to parse out config properties and see which not used
  • Sent to the Grouper Dev list
  • A few duplicate properties , should be cleaned up
  • Huge number of unused properties
  • About half of total
  • Penn still has the LITE UI
  • Some not in current source code
  • But still in LITE UI
  • Some are dynamically built
  • Chad can add LITE UI to the mix and get smaller list
  • Best path is to remove 1st instance of duplicates
  • Put the others config properties that are rarely or not used   in separate file
  • Some are missed because there is a single JAVA class that is a bean window 
  • Great report, thanks Chad

  • Looked at PSPNG and docker containers
  • Jeff, testing harness for PSP NG was created by Bert , runs through a series of tests
  • https://spaces.at.internet2.edu/x/Ugh0C
  • Docker Compose, depends on environment variables being set.
  • One LDAP container
  • Using it with different volumes
  • Create a new docker container based on it that adds the eduperson schema
  • If we can maintain own subcontainer or keep Docker container where you build it.
  • Steps: you start sub container w a certain state, then you can run a JUNIT test
  • Now everything is like a GSH script, but that is not efficient
  • If Chad makes changes to PSPNG , what’s the best way to run the tests?
  • Use the README
  • Was run locally on mac
  • Chad has LINUX
  • Chad will try to run the test
  • Provisioning tests and LDAP loader tests
  • Best to have a Docker container

 

 

Issue Roundup

 

Wiki updates


Grouper Slack

Lacey   -  need to set the ServerName directive in grouper-www.conf to allow shib to generate endpoints over https when behind a load balancer with SSL offloading.  

  

Andy - using good ol' patch in our Dockerfile to fix up a few things.  It is safer than overwriting upstream files because we see when we have drifted far enough away from upstream to need review or rebase of patches.


Chris Hyzer -this immutable container thing is just an issue.  i would like to be able to make a 2.5.28 and not annouce it and do tests, and if i need to retag with 2.5.28 iw ould like to be able to do.  at that point, after announcing, immutable...

 

Chris Hyzer -Grouper v2.5.28 sha256:537747 is released. 

 

Shilen upgraded to 2.5.28 and love how easy upgrades are now once you get to 2.5

 

Andy -We have 2 custom changelog consumers that we have been using for a long time.  When trying to validate that they work correctly in v2.5,   I can't get DEBUG logs out of them  

 

Michael Gettes  5:13 PM

could we get a “startup” and “shutdown” entry written to the loader log for loader/ws/ui/scim?  happy to submit a jira.

 

Carey  -dev server v2.5.28 container   (  I am not clear on how to use the Basic authn built into Grouper.  )


Chris Hyzer  - could set a default folder for UI, I think we have one for WS, but fully qualified should do the trick, or a subject source 

Michael G  AD and LDAP -   If you see Error 12 it could be you are hitting a Result Set config issue with AD.  

Chris Hyzer  Security advisory GRP-2705 for Grouper

Ryan R Quick question, is there a process in Grouper that builds the resulting group if the group is a composite?    

 Michael  i run find bad membership dmn every 3 hours.

 Robin  Is there a different approach to change the "Institute of Higher Education" string on the mainpage and the footer than customizing the templates?

Erik C   after I figured out the nagging problem of hung changelog consumers and fast-forwarding the counter, I   keep getting these stuck full sync jobs as seen in my daemon jobs dashboard 

Andy  With 2.5.23, when I run the container as "gsh", I see the   slf4j warning

 

Andy    There are access logs being written inside the container at /opt/tomee/logs/.  These are caused by the access log valve in /opt/tomee/conf/server.xml.  That should be turned off or redirected somehow so that we don't write logs inside the container.  … Double logging issue

 

Paul C I'm working on fixing this same thing for the shib IdP.   In that case, they'll be sent to stdout using the mkfifo thing that we do in other places...

 

 Chris Hyzer  This is fixed: https://todos.internet2.edu/browse/GRP-2802

 uses unix "patch". 

 

Paul C  having trouble wiring the mkfifo pipe for tomcat access logs in the shib IdP.  Tomcat seems to ignore the existing pipe when it starts up and writes its own file with the same name as the pipe...  THIS WAS FIGURED OUT

 

Carey would like to use a loader job to load a tree of groups. ( on the order of 30K groups in the tree. ) 

Paul R  A number of us have been scratching our heads with a particular SQL loader job.  

 

Chris Hyzer   Grouper v2.5.29  is released.  dozen fixes.  no upgrade notes.   

Chris Hyzer  I updated   wiki to explain our versioning strategy: https://spaces.at.internet2.edu/pages/viewpage.action?pageId=163119272

Patrick H running version 2.5.22 and   have   users reporting errors while trying to assign values to attributes using the UI.

Some users will run into it one day and the next be fine. I’ll put the error in this thread.

 

Lacey Trying to use the custom shell hooks in 2.5.28, 

 

Andy  sent this to grouper-users mailing list in February.   I'm posting it here, too:

fix-pspng-log-entries.patch 

 

Chris Hyzer  copied the grouper demo server 2.4 database to one for 2.5 and set that up with the 2.5.29 container

 

Josh  Question about v2.5.29 using ENV variables in our Dockerfile...

If we have in our Dockerfile the ENV variable  

 

Andy  interested in having our git repo be environment agnostic.


Chris Hyzer  you have the dockerfile, docker run command (with env vars), system env vars, system files, some sort of secrets manager.  all those are option at your disposal 

 

Andy How can we manage our configuration if it is stored in the database?  I mean, one nice thing about baking the config into the image is that I know exactly what "version" of the config I'm running.

 

 Is it possible to export the config from the database (non-default values)?

 

Andy   when upgrading from v2.4 to v2.5 of Grouper, I see the following SQL logged during the daemon container startup 

 

Paul R Anyone have experience with the usdu utility and the -delete option?   

Carey With the new ldaptive subject API config ... how does one set a "init-param" value for  subjectIdToFindOnCheckConfig  ? (edited) 

 

Nicholas Roy Is the ability to configure a loader job on a folder that is different from the target of the loader output a feature , bug, or  “meh”?

 

Alex  secret management question: I see from the wiki that consuming secrets via env vars is discouraged. I'm curious about the security implications here.  

 

 

Josh getting CSRF errors when I set grouper-ui(2.5.29) properties for overriding org image and/or custom css file.   

 

Carey  where to find the stuff to customize that "Help" link ( In the upper right hand corner ) ?

 

Richard F  would be nice to be able to change where that help link goes. 

 

Chris Hyzer   

text config in database in 2.5.30, its already implemented and ready to go  

 

Alex   aurora cluster spun up in our eng apphosting env ,   trying the experimental database migration tool mentioned here. Getting a vague trace:

  

Brett  -request for  update   wiki  with screenshots from the new UI? I'm pretty new to attribute definitions and am having trouble translating to what the new UI has. https://spaces.at.internet2.edu/display/Grouper/Organizing+services+in+Grouper

 

Lacey -previously working subject db connection in the subject.properties file started failing  


Andy - a way to control access to the UI, like the etc:webServiceClientUsers group for WS?

 

Carey  I think the "container host name" is not helpful in the log information.   

 

Carey What causes this configuration  message on startup?

 

Carey -Questions about  Grouper daemon "other job" to run a script :

    Specifically about the GSH script part.

 

Josh - issue in 2.5.29 using a GROUPER_DATABASE_URL_FILE variable.   

 

Josh I see container environment variables for running Apache/Shib SP inside the container, but just noticed that those are "up to v2.5.27"...   wanting to "kick the tires" on that concept, but we're running 2.5.29.

 

Carey  Problem with WS authN in 2.5.29 based image.

 

JIRAS

 

Grouper Users list emails


RE: [grouper-users] creating Grouper USER Subjects, Black, Carey M., 05/22/2020

Re: [grouper-users] creating Grouper USER Subjects, Kevin Rooney, 05/22/2020

  •  
  • No labels