Child pages
  • 14-Nov-2019
Skip to end of metadata
Go to start of metadata

Note: this Grouper call was on a Thursday, instead of Wednesday due to Grouper Training Nov 12-13 at Temple University  

Attending 

  • Chris Hyzer, Penn, Chair
  • Chad Redman, UNC
  • Shilen Patel, Duke
  • Carey Black, the Ohio State University
  • Bill Thompson, Lafayette
  • Vivek Sachdiva, independent
  • Jeff Williams UNCG
  •  Emily Eisbruch, Internet2

Action Items

  

New Action Items from this call

  • AI Vivek add enabled date, disabled date and is enabled for Web Service
  • AI  Jeff  add info to PSP NG doc on   what’s possible around containers security.   
  • AI Jeff look into warnings on target system full, is the cache being used? 
  • AI Chad update libraries for 2.5 branch , quartz and hibernate
  •  AI Jeff respond to Oliver Trieu on AD Provisioning not working on delete and see if JIRA is needed
  • AI Chris work on different rule option for composite per request from Arizona, 
  • AI Chris will start a blob or clob table

Grouper Sessions at 2019 Technology Exchange 

 December 9-12, 2019 in New Orleans

https://meetings.internet2.edu/2019-technology-exchange/


========

Grouper BOF

Wed Dec 11, 2019 at 4pm

======

Provisioning and Access Management: Case Studies With Grouper and COmanage

Dec 10, 2019 at 10:20am

https://meetings.internet2.edu/2019-technology-exchange/detail/10005660/

=======

Running the InCommon Trusted Access Platform in the Cloud

Wed Dec 11, 2019 at 2:40pm

https://meetings.internet2.edu/2019-technology-exchange/detail/10005654/



 Discussion

Moving to Grouper 2.5 Branch

  • Chris make a 2.5 branch, master will be Grouper 2.5 , still one more round of 2.4 patches
  • Is Merging development (those patches) w both master and 2.4 problematic?
  • Chris will cherry pick
  • Developers doing commits Can lead to some out of sync
  • Suggestion for fewer 2.4 patches now that there’s a new branch.
  • Minor bug fixes can wait.
  • Developers just work on master branch.
  •  What is scope of 2.5?  See Grouper Roadmap: https://spaces.at.internet2.edu/x/_oXd
  • Nice if patches are not too risky
  • For web service Vivek is working on 
  • If we put work in WSDL , not too risky
  • Flags and delete enable dates is risky
  • Must do Database and WSDL changes
  • Hope to release Grouper 2.5   soon
  • Maybe just do DDL changes 
  • Must change the HQL
  • Discuss later 

Things Learned from Grouper Training at Temple in Nov 2019

  • Good feedback and JIRAs emerged from the November 2019 Grouper training at Temple
  • Chris needs to go thru the Docker ITAP containers and see what quickstart recipes might help. ( Feedback from the Training at Temple)
  • Supporting Docker is key moving forward, ideas are welcome on better ways to make it work
  • UI and subject source and searching could be improved
  • Backburner: container discussion with Chris Hubing

Current work tasks, and next tasks

Vivek – Unit tests, web service updates for 2.5

  • Web Services
  • Create new folders
  • Done in a few days, develop in master
  • Once we go live, go back to feature branches
  • Java Beans with no logic, different for every version of Grouper
  • To support legacy clients w old versions of SOAP
  • Then uses reflection magic to call the logic methods with whatever arguments you had from that WSDL. Copy all the Beans from 2.4 to 2.5
  • Adding, making a service to query audits
  • Audit record and search capability
  • Group add, such as give me all audits between Jan 1 - Jan 30,
  • Store 10 columns
  • Must be a read only admin or Grouper sysadmin
  • Have a configurable group?  Not at this time
  • Looking at group type fields, 
  • Three field types, attribute def name not there?
  • Grouper screen on the UI, it uses audit type
  • Might not have that for attributes yet
  • If “look only at these stems for my audit”
  • Can do for stem and group, but not for attribute name and attribute def
    • Don’t worry about this now
  • Vivek will look at OR versus AND 
  • AI Vivek add   enabled date, disabled date and is enabled for Web Service
  • Paging in web services needs enhancement
    • JFW: WebDev: “the one i was seeing slower results with was WsRestGetMembersRequest”
    • Can leave gaps if things are added or removed
    • Better to ask for next 10 records after XXX 
    • Start w get membership service
    • People have complained about current pagination
    • Jeff is looking into this at UNCG
    • Will defer decision on this until next call
  • Point  in time inputs and outputs

Chris –  Training, database configuration, SQL sync

  • Bill and Chris finished 2-day training at Temple
  • Between 20-30 students
  • Will be getting survey responses
  • Adjustments made since previous Grouper training were good
  • Informal discussions are valuable
  • Went thru all features of Grouper, how to change access policies, different types of setups, how to change from legacy access to using Grouper
  • Did not focus on installing Grouper
  • UNCG might host a future Grouper Training

Database Config

  • Chad using the database config work, liking this new config approach
  • Good first pass
  • Would be good to have a wizard
  • Confusing to figure out which database has which config
  •   overall config page should filter across all config files
  • Hard to know where Some params   are
  • Using at Penn in production
  • Iterating on SQL sync
  • Penn needs to sync from cloud to Oracle
  • Want to put pooling back in
  • Opens door to other syncs
  • Moving data around is a key
  • AI Chris will work on different rule option for composite per request from Arizona, 

 

Shilen – Bugs, 2.5 database changes

  • Updated the JIRA w notes on database changes for enable / disable changes would UI require    
  • add enable/disable dates on groups like memberships and permissions
  •  https://todos.internet2.edu/browse/GRP-849
  • Might be useful to have in UI ability to recover  group based on point in time
  • This is meant to solve a different problem
  • Scope …. 
  • What should be in 2.5.0  and what should be in a patch
  • Issue around checking and group tables
  • Needing to manage Group and Rule sets… 
  • UI is not that important for start of this work
  • When group is disabled run one hook?
    Or run deleted hook on all attributes, etc?
  • Shilen will start this work
  • Timeframe for this work? TBD
  • AI Chris will start a blob or clob table


Chad – Bugs, libraries in 2.5, gantt chart?  Job dependencies?

  • Folder Tree
  • Visually show job execution history, will work on this 
  • Errors  instead of warnings, changed in XML config, it may be an Eclipse thing,
  • Possibly need to just turn off checkstyle checking.
  • IntelligJ  works better than Eclipse for warnings versus errors
  • Reporting using commas csv library
  • UI for import export, open csv
  • There is a JIRA to merge   JIRA   2250

 

  • Sent Chad down path of how to unit test UI functions
  • https://site.mockito.org/
  •  Mockito, testing with GSH , mock up request and response, HTTP session
  • Can fake specific attributes, simulate a login
  • Could be a new direction for unit testing in the UI
  • Would go into a JUNIT test file, call in Mockito and simulate
  • Chris likes selenium approach for end to end testing
  • Chad Will work on folder menu item
  • AI Chad update libraries for 2.5 branch , quartz and hibernate
  •  Joins are now cross-joins for hibernate

Bill – GDG, training, pspng, dev env

  • Grouper Deployment Guide call , #2 on Wed Nov 20
  • Hoped for a 1st pass of GDG by TechEx, may be challenging to meet that 
  • After Nov 20 community call,   consolidate and create a work plan
  • Do a review of the input to GDG  at ACAMP
  •  Finalize the GDG after  ACAMP
  • May want other Grouper discussions at ACAMP too, to be discussed on a future call

 

Thoughts on creating UI template, inspired by question on Grouper users list from NDSU

Issue Roundup For Nov 14, 2019

JIRAs since Oct 30, 2019

  • GRP-2423 from training: we need a wiki of how to architect a prod deployment
  • GRP-2422 training class suggests we need a wiki on how to move from container vanilla to evolved container
  • GRP-2421 better error message if adding a group to itself as member
  • GRP-2420 add point in time from a subject view (and which memberships they are in)
  • GRP-2419 search for subject to add and select multiple (e.g. multiple groups
  • GRP-2418 when searching for assigned owners of attributes, allow search by attribute value
  • GRP-2417 free-form search on subjects should take subject source into account
  • GRP-2416 folder menu tree add options to limit attributeDef and attributeDefNames
  • GRP-2415 add database name from grouper loader properties to report config
  • GRP-2414 decryptInFileIfFile stack overflow since there is a config option for encoding
  • GRP-2413 Allow loader jobs to be triggered by another job completion, not time-based
  • GRP-2412 Folder menu tree to use "..." to indicate a truncated list of items
  • GRP-2411 simplify packages, ant files are large, make them more simple, tarball and compile,
    • Team, please add your ideas on simplifying to this JIRA 2411
  • GRP-2410 pspng full sync getting error about queue not exist
  • GRP-2409 real time sql sync
  • GRP-2408 assigning an attribute value from stem attribute assign page results in blank page
  • GRP-2407 grouper client db access does not connect to the right
  • GRP-2406 have gsh status script to check health before container start up
  • GRP-2405 "run daemon now" should schedule that job on the daemon server just like the "all daemons" screen
  • GRP-2404 attestation emails sent from nightly daemon are not sent out if configured to the admins/updaters of group
  • GRP-2403 database sync should be able to read from a separate (target) database than it writes to
  • GRP-2402 Morph.decryptIfFile should accept slashes in passwords
  • GRP-2401 database sync needs to update the logs in database as it runs
  •  GRP-2400 grouperClient has problems with grouper client properties in database on startup
  • GRP-2399 otherJob for table sync reports success when there is an exception
  • GRP-2398 scheduler check should (un)schedule grouper loader other job changes
  • GRP-2397 make log4j support EL or DB config, for encrypted passwords and logging dbug
  • GRP-2396 ws authn ng, with local entities, generate me some secrets,  more sophisticated, using hash, could work in concert with what people now have
    • Matt working on authN , not yet focusing on AuthZ, consider a cert based  approach?
    • Or something more lightweight 
  • GRP-2395 have an option to log in json
  • GRP-2394 workflow misspelling
  • GRP-2393 grouper claims not started but it is
  • GRP-2392 report daemon error
  • GRP-2391 Ability to advance a Change Log consumers last_sequence_processed value to force it past blocking errors that can be skipped
  • GRP-2390 grouper loader mistakenly uses db pool size of 5


Grouper-Users email list  



Slack Grouper Discussion since Oct 30, 2019 

 Michael G  - request (i believe there is a jira) to have a switch to turn off the automated fullsync within the PSPNG incremental.  I’d like to have my scheduled full-sync do the full-sync and not have the incremental try to be doing them as well. I need the incremental to be as fast as possible. 

Gasper:  Cal has some existing enhancements to the Google Apps/GSuite provisioner that I'm going to be baselining. We are  going to be reworking the provisioner to make batch request occur (extending on the existing Cal enhancements). wondering if any of the Google Apps/GSuite provisioner users   have wish list items that I should consider . I've identified a few like making the location of the appropriate attribute def/name customizable instead of fixed.

Bill T: EDUCAUSE Security Professionals Conference April 21–23, 2020 in Bellevue, Washington, plan for half day seminar, abstract submitted,

P Engle:  excited about the config-in-db concept

Michael Gettes - is it possible to cause an existing loader job to “quit”?  stop before subjobs and if doing subjobs to stop dispatching subjobs?

 G Haverkamp: WS authn issues, OAuth2-only mode

Chris H: an issue going to aws for us is our log4j mail config

Chris H : otherJob for SQL table sync https://todos.internet2.edu/browse/GRP-2398 scheduler check should (un)schedule grouper loader other job changes

Chris H -  Slashes in Passwords: https://todos.internet2.edu/browse/GRP-2402

Sudheer - configured attestation on a group to send emails to group admins but the emails are not being sent (I added "subjectApi.source.ldap.param.emailAttributeName.value = mail" in subject.properties).But when i pass a custom list of  email addresses the emails are sent). not seeing anything useful in the logs. How to troubleshoot? (NOTE THIS SEEMS TO HAVE BEEN FIXED)

In the future, Put a warning? 

Rachel Louden    issues with the complex demos in midpoint and grouper. Is there someone I could reach out to about this?

Matt/CAREY: A group supports adding members via two "bulk" processes in the UI. ( "Copy and Paste a list" and "file upload" )

Are there any stated expectations about when one or the other should be used?


J Crawford  

  •   if you set the subject.properties subject cache to have a directory of /var/tmp, and if you run both the UI and WS on a single tomcat. Is it a problem if they both share the same directory? Seems to work in test but they don't get much traffic.
  •   Take another pass at subject.properties subject cache ? 

 J Crawford --   gotten WS call to try and add an attribute and value to a stem/folder, however  keep getting the error: Unknown property WsAttributeDefNameLookup. 

Matt/CAREY: use the WSDL 


Sudheer:   warning and how to fix it?

  •   [ajp-nio-8009-exec-8] WARN  SubjectSourceCache.updateSubjectInCache(1114) - < grouper - 10.21.15.211 > - In subject source: umnldap the identifier: 'jrg' can find subject: 'y3453536, but the attribute for that identifier is not configured in the subject source.  In order for caching to be effective, please list all identifier attributes in the subject source. You can configure to suppress this log message in subject config.

Kurt McNew 

  •  receiving the following error when attempting to send to a grouper message queue:java.lang.RuntimeException: Cant find GrouperMessageQueue
  • Type from string: 'null', expecting one of: queue, topic,

Jeffrey Crawford  

  •    trying to add a new attribute that can be viewed on the UI and be available on the WS call.   when I add the new attribute to "subjectApi.source.ldap.attributes" it's not showing up on the UI or WS.  



Grouper Wiki Changes/ Updates recently (not a complete list)

New Comment on Attestation Using Reports page from Sudeer

Chris Edited SQL Database Provisioning:

Chad edited : Grouper Dev Environment


  Next Grouper Call: Wed. Nov 27 , day before Thanksgiving

  • No labels