Attending
- Chris Hyzer, Penn, Chair
- Chad Redman, University of North Carolina Chapel Hill
- Shilen Patel, Duke
- Carey Black, the Ohio State University
- Bill Thompson, Lafayette College
- Vivek Sachdiva, independent
- Emily Eisbruch, Internet2
Intellectual Property reminder: http://www.internet2.edu/membership/ip.html
New Action Item from this call
- AI Chad document the solution (walled garden) to the OpenShift in a wiki to help the community
DISCUSSION
Grouper School June 2-3, 2020, ONLINE
https://incommon.org/grouper-school-virtual/
Went well, over 30 attendees, next online Grouper training likely in September 2020
- Successful virtual Grouper training
- too much content
- Having the next training in the Canvas LMS may be helpful
- People can do some exercises on their own
- Interest in templates during the training
- Configure the templates like the custom UI is
- Fill out your template declaratively w JSON
- Chris and Chad talked w N Carolina State
- Interested in a different kind of composite
- Attributes on things to make a complex composites
- Complex composites: important that access policy is transparent and discoverable,
- value of visualization
- Chad : complex composites could be added to visualization
- Keep implementing simple access governance and build on that
- Make it easy to implement more complex access governance
- Hope to do a proof of concept
- Easier to create
- Bill: recent features that we should show in training: grace periods
- Grouper has generic policy templates
- Might have templates specific to a target service
Provisioning
https://spaces.at.internet2.edu/display/Grouper/Grouper+generic+provisioner+framework
- Something for LDAP that is performant and works
- long term
- Starts w provisioner code
- Base provisioner
- Get configuration, can be standardized for UI
- Custom provisioner
- Will use provisioning attributes Vivek developed
- For a simple provisioner you may only need CRUD
- Config: generic to multiple provisioners and specific to a single target
- Diagram is helpful≠
- Difference from PSPNG: some of the mapping , the caching, interact w full and real time sync, now built into Grouper, and provisioner does not need to do it. All CRUD operations can be done in bulk if possible.
- Looking at provisioning history, PSP, PSPNG, using SPML, and Shib attribute resolver
- This is a step forward, Using minimal object model will be an advantage and take memory requirements down
- Logical architecture
- Maps to what Lafayette is doing w Rabbit MQ
- Lafayette: Constant provisioning, helpful to have a few framework classes, especially for SAS applications
- Using flavors of custom provisioners
- Chris: we have external system configuration, takes communications between Grouper and another system and groups those
- Could be Rest external system, for example
- Goal is to enable explosion of custom provisioners that plug into Grouper and can be shared in the community
- Summary: the generic configurations are there and Grouper might use them
- The external config class and the specific ones would be listed, might not be there for every provisioner
- Vivek will work on the config
- Q: How is change log consumer used with this?
- Change log consumers moved to ESB,
- Subclass that calls ESB
- Translates into an ESB event object module
- Puts in Java bean and allows filtering
- Hope to use that pattern , Change log will be like ESB workflow
- For messaging we don’t know status of the call
- Vivek: Perhaps use another queue to address that
- Still might not work for bulk operations
Vivek – Daemon configuration and Provisioning configuration
- Daemon jobs work is complete
- It’s in master
- New options: edit Daemon, delete daemon
- Gives screen to edit the properties
- If seeing loader type of job and you click edit you go to the loader screen where it’s being managed
- Can also add a daemon
- In future , could have more dropdowns, but now need to move to work on provisioning
- Vivek just started the provisioner config work
Chris – Recent memberships, templates, composites ng
Shilen – usdu, daemon scheduling
- Fix for scheduler issue that Michael reported
- Trigger set to acquired
- 2.5.30 will have fix
- Looking at USDU
- Added 2 more columns to member table,
- Columns for Resolvable, and for deleted
- Made DDL updates
- DDL utils no longer used
- Done for four databases
- For upgrade tasks, should we have a table so we don’t have to run check config every time?
- There is an attribute to keep track of what has been done so far
- Check that attribute when we start up to make start up faster
- Can be part of upgrade tasks
- Could roll back that attribute value if necessary
- AI Shilen make a JIRA for linking check config to upgrade tasks so check config isn’t run on every startup (DONE)
- Issue with UI and USDU, may have a bug fix
- Will need to verify
- This is all because of Loader issue and unresolvable that came up.
- Will set up so if when the loader tries to resolve in member table , and not resolvable, then it goes to the source
- Loader use Subject identifier column in member table, but for resolving must go to source
- Next step: Loader should resolve in same way as is done for provisioner
- Carey: Caching GUIDS from external systems back into Grouper
- Overlapping makes sense
- Shilen Next Task: Get a run script for simple provisioner, test Docker container w OPEN LDAP
Chad - pspng, docker compose, unit tests, o365?, etc
- Two PSPNG issues assigned to Chad
- Working on them
- One issue involves: Turns off full syncs
- Reaching out to Michael G
- Looking at OpenShift and why it’s not working
- Doesn’t run as root
- Starts as container w no record in ? password
- Can’t change file permissions
- If you have user name where groups are invalid it sets group to root
- If we make all writable by root, that would help
- Apache and Shib SP works outside the container
- Everything needs to be world readable and writable
- Can't specify a UID
- World writable , using user space partitioning
- Nothing else will use that space
- See doc on Grouper 2.5 as non root https://spaces.at.internet2.edu/display/Grouper/Grouper+Container+v2.5+running+as+non-root
AI Chad document the solution (walled garden) to the OpenShift in a wiki to help the community
Issue Roundup
Slack
Michael G Instrumentation error , Multiple assignments exist
Carey Black a * minor* problem with the 2.5.29 container.
If you enable: hooks.membership.class =
Ethan K Is there a way with the 2.5 container, to re-home the applications? using Openshift,
Chris Hyzer Grouper cant assume directory structures after all?
Michael G Chrome has a dark mode you can “force” on in chrome:// .
Sudheer We are seeing warning message in grouper_errro.log. I thought the warning is because DID is not listed as one of the subject attributes.
Carey Is it possible to do something like this Grouper+rules+use+case+-+Add+an+attribute+to+group+with+value+if+name+matches+a+pattern+or+two except.
When an attribute ( a specific attributeName ) is assigned to a group that is in a known stem branch, then have the Rule auto set a value on the attribute for the user.
Lacey Is there a reason you would want secret files (like passwords) linked in the classes dir vs just referencing run/secrets/secretname in the appropriate config files?
Sudheer seeing warning message in grouper_error.log and we are using subject identifier in loader job to load the group but not sure where to cache the subject identifier.
Carey - A docker container question, Are there "temporary file systems" that Grouper should use for transient file storage?
Carey - What Grouper features can use "SFTP", "S3", or require local file space for them to work?
Chris Hyzer -Everything in the container is transient and non persistent.
Erik C - Just when I feel like we've resolved sync issues, it seems like I keep getting snagged by more PSPNG trouble.
Carey v2.5 OTHER_JOB_usduDaemon debugging question....
I am not able ( yet ) to get the USDU to run to completion.
Carey - Is there a way to run the Loader job diagnostics in GSH? ( Like the Subject source Diagnostics can be run?)
Chris Hyzer here is our design for the new provisioning framework.
https://spaces.at.internet2.edu/display/Grouper/Grouper+generic+provisioner+framework
Michael - i have a loader job that runs every hour. We have just noticed it hasn’t run since 06/01. I have seen this a few times now. (on latest grouper, of course). if i manually start it - it will be “healed” and start running hourly again. I saw this with changelogtemp consumer.
Ross W attempting to import a configuration file from a Windows machine ( C:\Code\GrouperSettings\subject.properties ) via the form and grouper (2.5.29) is responding with "File name must be one of the following: grouper.cache.properties, grouper.client.properties, …..
Carey - Grouper+Subject+API+caching+improvements+in+2.4 :
If the subject source is ldap... and one of the identifiers are a multi-valued attribute.... does that "work" or should that be forbidden in the docs?
Carey - I think it would be wise if someone would document that as "supported". It is not clear at the moment and my assumption is that it would not work out well.
Do you already do that?
Christopher B
probably best to avoid, as ldap is not guaranteed to return values in any particular order, so could result in cache churn
unless grouper is imposing some ordering in the case of multiple results
Erik C - On Grouper 2.4 I can't seem to get any of the audit log functions to return anything within 60 seconds,
Carter S The only solution we found was truncating the audit log table. Not a great solution though
Carter S
We’re running Maris DB . trying to figure out what the appropriate timing is for audit logs.
Chris Hyze we have a design for fixing that with a table that indexes the data, but its not on our immediate radar.
Marwan
Is the “Veto if too many members” rule available only in relatively newer versions/patches of the API? Newer being >= 2.4.0
Richard F You may need to extend your proxy configuration for HTTPD. Tomcat may not be sending bytes back quick enough, and HTTPD is timing it out (or whatever you are using for a proxy).
Andy - I’m playing with the recent memberships (grace period) feature. We just upgraded to 2.5.29! … How does this work to remove the subject from the recentmemberships group after the expiration?
Ross W I had not realized that there was a firewall blocking my container from talking to my subject source and it kinda bricked my UI container.
Jeffrey C - I think something kicked off that cause around a 1.6 million record change, ChangeLog has been running for an hour already. All of these changes seem to be outside what we provision to LDAP, is there a way to forward skip if we accept that any changes that need to be provisioned will be skipped?
Andy Here is a gross way to get a unix timestamp for 'now' from Oracle
Should a recentmemberships group work if the base group is a composite? nvm, for some reason the changelog didn't work on it right away. After I created a test composite group and assigned recentmemberships attributes, the real group started being populated as well.
Andy I think there are probably 2 "bugs" right now: 1. unix timestamp math is off by tz offset in Oracle 2. doesn't support multiple recentmembership groups on a single base group
Carey Has anyone else had an issue with a v2.5.29+ container and an LDAP_GROUPS_FROM_ATTRIBUTES loader job?
Scott Koranda With Grouper configuration being held in the database, is it possible to make a configuration change using GSH?
Jeffrey C
I'm trying to determine if our DB is too small, We have quite a few groups that have over a million members, A changelog temp to changelog processes ran for just over 6 hours processing a 1.6 million entries.
Kevin R
looking for subject source api sasl_mech configuration docs/info/details if anyone knows where they might be. I'd be happy with just drop a keystore here and config the keystore password there.
Jeffrey C
I just tried to get grouper 2.5.29 running in my sandbox, I'm getting a slew of oracle errors.
Andy
PSPNG full sync slowness problem...
Jeffrey C
I've been fighting with my targetSystemUserCacheSize, however my groups can have over a million members. … I'm trying to find the sweet spot.
Jeffrey Crawford forward to by going to AWS.
Chris Hyzer There are a few things with recent memberships, and thanks for your time spent on it and figuring out the fix...
Shilen Patel Might be nice if an upgrade task fixes/moves the assignments?
Carey - Does this WebService Add+or+remove+grouper+privileges Only change privileges on Groups and Stems? Or does it support setting privileges on AttributeDefinitions?
Andy Morgan Related to Jeffrey's comments about targetSystemUserCacheSize - how much memory does each cache entry take, and should I really be trying to size my targetSystemUserCacheSize to match the total number of users in my target system?
Jonathan S - putting together design for our application’s permissions in grouper. I’ve been reading Grouper Deployment Guide and found this built-in application template: https://spaces.at.internet2.edu/display/Grouper/Grouper+new+template+wizard. I think this is close to what I need, but our application allows users to be in multiple workgroups.
Jeffrey - I'm trying to get GROUPER_WS_TOMCAT_AUTHN=true to work on our sandbox, not working, It's not requesting basic auth.
Ross W - Certificates for Shibboleth SP - Right now I have them in a slashRoot being mounted to the container. ….is there any way to do this with slashRoot?
Grouper Users Email List
- RE: [grouper-users] Grouper 2.5 on Openshift, Black, Carey M., 05/29/2020
- Re: [grouper-users] Grouper 2.5 on Openshift, Darren Boss, 05/29/2020
- RE: [grouper-users] Grouper 2.5 on Openshift, Black, Carey M., 05/29/2020
- RE: [grouper-users] Grouper 2.5 on Openshift, Black, Carey M., 05/30/2020
- Re: [grouper-users] Grouper 2.5 on Openshift, Oliver Trieu, 05/29/2020
- [grouper-users] grouper 2.5 gsh/database, T-Heetderks, 05/30/2020
- Re: [grouper-users] grouper 2.5 gsh/database, Hyzer, Chris, 05/30/2020
- Re: [grouper-users] grouper 2.5 gsh/database, Hyzer, Chris, 05/30/2020
- [grouper-users] Stem display_name column length, Sean Mason, 06/01/2020
- RE: [grouper-users] Stem display_name column length, Sean Mason, 06/01/2020
- RE: [grouper-users] Stem display_name column length, Black, Carey M., 06/01/2020
- RE: [grouper-users] Stem display_name column length, Sean Mason, 06/01/2020
- RE: [grouper-users] Stem display_name column length, Black, Carey M., 06/01/2020
- Re: [grouper-users] Stem display_name column length, Hyzer, Chris, 06/01/2020
- RE: [grouper-users] Stem display_name column length, Black, Carey M., 06/01/2020
- Re: [grouper-users] Grouper 2.5 on Openshift, Alex Poulos, 06/01/2020
- Re: [grouper-users] Grouper 2.5 on Openshift, Darren Boss, 06/01/2020
- RE: [grouper-users] Grouper 2.5 on Openshift, Black, Carey M., 06/01/2020
- <Possible follow-up(s)>
- RE: [grouper-users] Grouper 2.5 on Openshift, Hyzer, Chris, 06/01/2020
JIRAs
- GRP-2841 Grouper loader continues to add usdu deleted members to groups
- GRP-2840 "recent memberships" should work across group rename
GRP-2839n "recent memberships" has problems when the same group has multiple recent memberships lists
GRP-2838 recent memberships has timezone issues with oracle
GRP-2837 Quartz triggers stuck in ACQUIRED or ERROR states
GRP-2836
Configuration file upload fails on brow
GRP-2835 automate CSV group load
GRP-2834 support "dark mode" in chrome
GRP-2833 null pointer in ws servlet
GRP-2832 workflow , approval ... membership audit row does not show who approved the membership. ( It is blank. )
GRP-2831 Instrumentation throws an error
GRP-2830 grouper demo email smtp is broken and stopping registrations from happening
Next Grouper Call : Wed June 24, 2020