Child pages
  • 30-Oct-2019
Skip to end of metadata
Go to start of metadata

  

Attending 

  • Chris Hyzer, Penn, Chair
  • Chad Redman, UNC
  • Shilen Patel, Duke
  • Carey Black, the Ohio State University
  • Bill Thompson, Lafayette
  • Vivek Sachdiva, independent
  • Jeff Williams UNCG
  • Jonathan Johnson, Unicon
  • Alex Poulos, U of Maryland, College Park
  • Emily Eisbruch, Internet2

Action Items

 

New Action Items from this call

  • AI Chris make a branch, master will be 2.5
  • AI  Vivek set up meeting with Chris to discuss the Grouper web services work. 
  • AI  Chad change statistics handling to make postgress tests work 
  • AI Chad   create a JIRA for the Loader Scheduling,  one job kicking off another,  
  • AI Chad reach out to Chris Hubing to be sure we are consistent w SHIB around  Amazon Corretto Java and Open JDK
  • AI Emily reschedule the Wed Nov 13 Grouper Call to Thurs Nov 14 (DONE)
  • AI JJ set up time to meet with Chad to review their work on office365-and-azure-ad-grouper-provision
  • AI  Jeff look at wiki for PSPNG and comments on Grouper Loader LDAPs to be more specific, it’s not obvious when you set it up , look at base config file
  • AI Jeff  look at Escaping search filter, Pete St. Onge email 
  •  AI Jeff  respond to   Oct 10 email from  Satya about PSP, suggest to stay on old JAVA until get on PSP NG
  • AI Chris reply to Creating our own template email, Richard Frovarp, 10/24/2019
  • AI Chad will work on "Browse folders" pane: no visual clue issue

 DISCUSSION

  • Welcome Alex and JJ to the call
  • U Maryland College Park is upgrading Grouper to 2.4, they had done some original work and will be doing much more Grouper work in coming period. 
    •  Interested in permission elevation feature so admin not alway running as super admin but elevates to higher permission level when needed,
    • User impersonation  feature for dev and QA 
    • https://spaces.at.internet2.edu/x/vQv9Bg
  • ChrisH: recently changed how priv work for admin users
  • To minimize # of priv on objects

CSP (Collaboration Success Program)

Chris represented Grouper at the CSP (Collaboration Success Program) gathering in Colorado last week  https://www.incommon.org/academy/cspdetails/

Current work tasks, and next tasks

Vivek – Bugs, attribute assignments from attributes screen, unit tests

  • Will start working on web services for Grouper 2.5 . 
  • AI  Vivek set up meeting with Chris to discuss the Grouper web services work. 
  • AI Chris make a branch , master will be 2.5
  • Grouper DDL  test issue, had to ignore ,  note there is switch around not checking config when doing DDL add  ? Tough issue
  •  Chris –  Bugs, attribute assign queries, training, database configuration
  • Stabilize the current ITAP container, should be done today
  • Penn now running in AWS, tracking down issues, loader has been a challenge, performance is better than with Oracle 


 Shilen – Bugs, 2.5 database changes

      • Working on bug fixes
      • Lazy subjects w loader
      • Test failures because of order by
      • Reporting not integrated  w diagnostics (patch still needed)
      • Fixing test cases, 
      • Note there are 2 wikis for tests….
      • Some tests still Failing , Vivek will take a look
      •   caching challenges … putting things in the flash cache that should not be there, called in a way not taking security into account 
      • If Chris will suggest the way to fix the caching Shilen will do it
      • Shilen will look at the latest commit 
      • Then will start on the database
      • Groups table, will need 3 tables:
        • Enabled or not, and enabled/disabled dates
        • Will need to have views
      • Remind Chris to make branch if needed

  ChadBugs, libraries in 2.5

      • Postgres
      • Major suites (not all tests, more reliable)
      • 4 errors
      • Permission stress test takes longer than it should but it’s OK if statistics are analyzed, probably can ignore this,  it’s only in the test
      • Hard to do statistics in platform neutral way
      • Hard to do statistics in middle of test
      • Chris: want to have the tests work
      • AI  Chad will make change to statistics handling to make postgress tests work 
      • Table sync, assumes metadata are all upper case, but in postgress its all lower, can just leave this until someone needs this to work in postgress
      • Best answer is to query database  
      • Create query and do metadata on the query
      • There is an open JIRA for this , see the test page  
      • Table sync on postgress works at Penn


      • Working on GANTT chart to see what loader jobs are running
      • See for previous day
      • Load a single group
      • Visualization, Chad has created 1st attempt
      • Could have mouse over and links
      • Built on D3 and module for GANTT charts
      • Looks good.
      • JJ is working on this as well,   Whisperer chart 
      • JJ will chat with Chad….
      • Building out the .csv
      • Good to have ability send a file and have graphs made
      • Grouper reporting may also be relevant

      • Loader dependencies and scheduling , have config field for a loader that says run this, or wait, some sleep time, issue of not running until next day if daemon gets bounced
      • Shilen : not sure what the default is for that…
      • Two types of dependencies: don’t run at same time and be sure it finishes for the day
      • Chad: would like feature that gets  one job to kick off when another is finished
      • Right now we only have time based and you hope one job stops before other starts, or you miss a day
      • Schedule type not a chron but instead a dependency schedule
      • Quartz supports something similar
      • OSU using Nagios, agree  these features would be useful
      • Shilen: could have config on a loader, after this finishes run these other jobs,
      • Then jobs could be kicked off , should not be too hard to implement
      • Is this worth including in 2.5?
      • Other  jobs , could be chained together
      • AI Chad will create a JIRA for the Loader Scheduling,  one kicking off another, etc 

Carey /Matt

      • Loader job issues
      • Sub plug in module
      • Separate grouper dependencies into one project
      • With  sub dependencies
      • Coming back to local database tables
      • Integration separation ,,, go get data and load it into database tables
      • Independent of code dependencies
      • Then use Grouper loader jobs in sequence order
      • Schedule this as another job , using another class to get data to local database tables, 
      • Runs jobs from inside the other job
      • Schedule for some year in the future so they never run 
      • Duke needs similar solution 
      • All REST based,  pull from LDAP to SQL table
      • Matt: Challenge, geting Grouper dependencies built and linked
      • don’t have good MAVEN repo
      • MAVEN dependencies
      • Chris: we should have MAVEN available
      • Do we have latest snapshots in MAVEN?
      • Chad: 
        • In MAVEN we have Grouper 2.4 release 
        • Then snapshot releases, but hard to match w specific patches
        • Hard to pinpoint , snapshot is whatever the current master commit is
        • Unless we do more frequent Grouper releases, there is a challenge with MAVEN
      • Other comments
        • Unicon dealing w that exact problem now
        • The big Docker image
        • Have also dealt w the MAVEN issue
        • Anything that can make the Docker image less complex would be helpful 
        • Each of the 3-4 parts had different bugs in latest patches.
        • Had to pull down 4 different images to get at right patch for things to be working
        • For example, I want Grouper 2.4.0 and API patch 67
        • Want to be able to pull this down
          •  
        •  the way patching works in Grouper project is unique
        • If we had a MAVEN release per API version
        • That would be a clean line for developers to base on
        • But that’s hard to get out of current Grouper project development style
        • Working in daemon backend consumer/ changelog 
        • OSU trying to move to full MAVEN environment
        • Hope to run the Docker containers in that environment
        • Would like it to be fully automatable
        • Not based on the ITAP container
        • ITAP overcomplicates the container by combining so much
        • JJ: 
        • Bert: GA Tech ‘s needs led to a large Docker image, had hooks and configs , did not want to maintain multiple images, such as 3 different source images, preferred large images
        • Alternative is different source images
        • JJ: yes like having one image, but the different parts of Grouper does not lend itself to this
        •  Some people do not use web services
        • Different orchestration environments
        •  
        • Chris: everyone will have different ideas on this
        • It’s important to be consistent 
        • Come BACK TO THIS ISSUE ON A FUTURE CALL
        •  
      • Chris: check out API in your development environment
      • Must keep Grouper client up to date
      • Grouper Client is part of the challenge
      •  

Bill – Grouper Deployment Guide (GDG), training, pspng, dev env

      • Grouper Deployment Guide is here: https://spaces.at.internet2.edu/display/Grouper/Grouper+Deployment+Guide+2.0
      • Notes from the Oct 23, 2019 GDG community call  are here https://spaces.at.internet2.edu/x/KRGJCQ
      • Call was great, excellent feedback, 
      • hope for published version of GDG on the wiki by TechEx
      • Next GDG community call is Wed Nov 20 , 11:30am - 1pm

      • Grouper Training at Temple, Bill getting the training environment working
      • Change to Grouper call from Nov 13 to Thurs Nov 14 at  11:30am 
      • AI Emily will reschedule the Wed Nov 13 Grouper Call to Thurs Nov 14

      • Bill will submit proposal for EDUCAUSE Security Professionals Conf Washington State in April , proposal by Nov 5. 




Issue Roundup 

Grouper-Users list  

      • Grouper 2.4 and office365-and-azure-ad-grouper-provisioner Oct 16, 2019 
        • Chad responded to this inquiry.
        • Get  the code into the mainline..
        • Open source the UNC code?
        • Using proxy, setting groups to private by default, expression language for display name
        • Coding not too difficult
        • Someone who wanted to change the code would not have too much effort
        • Unicon has had requests for this and done some enhancements
        • Unicon Had intended to contribute this back to the Grouper code base (as they did with Google groups)
        • Used old changelog consumer pattern
        • Thought of moving to PSPNG pattern
        • May be a rewrite 
        • Unicon does want to duplicate effort w UNC
        • UNC is also using changelog consumer, not PSPNG
        • UNC took from single group sync to a folder sync
        • Problems w deprovisioning since there is no deprov on folders
        • So need to do a report 
        • Syncing a single group is more straightforward
        • Don’t recommend customization of changing from group to a folder
        • Next steps: Start over and gather requirements? Want to harmonize the versions and make this available to the community?
        • Chad has not seen changes in past few months
        • Chad documented some of the work on UNC contribs page
        • Using UUID identifier and an  attribute for the provisioner
        • AI (JJ) set up time to meet with Chad to go over their work on  office365-and-azure-ad-grouper-provisioner 
        • Chris has added this to   Grouper Product Roadmap
      • AD Provisioning not working on delete, Oliver Trieu, 10/17/2019
        • Jeff followed up on this
        • Load balancing round robin gets things out of order potentially
        • Got some debug info
        • One instance where provisioning worked but then went to LDAP and it was not working right
        • Not sure what is going on. Hoping to get more info
        • AI  Jeff look at wiki for PSPNG and comments on Grouper Loader LDAPs to be more specific, it’s not obvious when you set it up , look at base config file
      • Change loader db name?, Richard Frovarp, 10/17/2019
        • Chris will reply to this 
      • Grouper, COmanage, and midPoint Training Workshops - Registration is Open!, Erin Murtha, 10/17/2019
        • Close to 30 attending
      • Escaping search filter?, Pete St. Onge, 10/24/2019
        • This is a common issue … 
        • AI Jeff  look at Escaping search filter?, Pete St. Onge email 
      • Substantial patches released, Hyzer, Chris, 10/24/2019
        • Config work
        • Attribute assignments looking at name
        • There will be one more patch and container today to fix database issue
        • Then should be back to stable


      • Creating our own template, Richard Frovarp, 10/24/2019
        • AI Chris reply to Creating our own template email, Richard Frovarp, 10/24/2019
      • "Browse folders" pane: no visual clue indicating truncated list, Dominique Petitpierre, 10/28/2019
        • After 30 it’s dot dot dot
        • 30 is configurable in Grouper Properties
        • UI function to return JSON, adding an extra folder if greater than 30
        • Might need a new icon, all 4 types of objects could be dot dot dot
        • AI Chad will work on "Browse folders" pane: no visual clue issue


Grouper-Dev list  

 Display a maintenance banner, Alex Poulos, 10/21/2019


Slack since Oct 16, 2019

  • Oct 17:  implementing  an "OtherJobBase" job.    would like to be able to get the java Properties for the job from the config files, Carey
  • Oct 22 Is there a way in the grouper UI to only show memberships from a subjectSource, J Crawford
  • Oct 22:  upgrading from 2.3 to 2.4 & moving to the containerized Grouper. Getting spotty UTF-8 behavior with latest build,  Paul Engle
  • Oct 23 : Is there a way to do a "Membership select statement" that  would honor the ACL's for a local entity? , IdP and the LDAP architecture,  Carey
  • Oct 25: access control for configuration editing can point to X-Forwarded-For but can't parse the header if it contains multiple IPs (which ours does, since it passes through multiple proxies). Should I file a Jira for that? , Paul Engle
  • Oct 26:   Created jira for dont set cookies in WS container. if path is different  some clients give superfluous warnings  https://todos.internet2.edu/browse/GRP-2381 Chris Hyzer
  • Oct 26: switching from oracle to aurora postgres, and moving from on prem (without ITAP container) to AWS ECS with ITAP container, Chris Hyzer
  • Oct 26: how do i change the max memory in the itap container for tomcat, Chris Hyzer
  • Oct 28: We're looking to bring up our cloud instance this week and having to decide between RDS postgres and Aurora, Thomas Elliot, 
  • Oct  29: issues with too drastic of a change in loader job,  groups could have a min size attribute, that would be honored by ui/ws/loader/gsh, or notification attribute,  ChrisHyzer
  • Oct 29: issues with config files and overlays in patch, M. Gettes
  • Oct 29: B-L, have the status messages for full-syncs changed?, Jeff  W

================

JIRAs  since Oct. 16 2019

  • GRP-2389 Bring the Google Apps Provisioners test environment and dependencies current
  • GRP-2388 add minimum group size to loader config
  • GRP-2387 Improve UI by allowing group member list to include subject source.
  • GRP-2386 Grouper reports missing from diagnostics
  • GRP-2385 UI config screen doesnt show config items which are not in the base config
  • GRP-2384 ui configuration changes do not immediately show up on screen
  • GRP-2383 database configuration can return no records and use default configuration
  • GRP-2382db config issues
  • GRP-2381dont set cookies in WS container. if path is different some clients give superfluous warnings
  • GRP-2380Morph was moved from external in grouperClient, adjust those projects
  • GRP-2379 does grouper-ui restrict by group work?
  • GRP-2378installer should ask about force installing patch if file is supposed to be new but exists
  • GRP-2377Problems with UTF-8 display in UI
  • GRP-2376add attestation from supervisor view
  • GRP-2375grouper configuration in database should show this as password
  • GRP-2374you should be able to edit config which has example (or remove edit button)
  • GRP-2373change installer to deal with new morphString features
  • GRP-2372add attestation to access policie
  • GRP-2371add something in WS response that indicates readonly mode
  • GRP-2370Attribute queries with order by failing in hsql
  • GRP-2369configuration.detect.errors does more than what documentation/base config says
  • GRP-2368deprovisioning report has duplicates on it. each group should display once and only once
  • GRP-2367viewing database config is slow
  • GRP-2366GrouperUtil.stripEnd should be GrouperUtil.stripSuffix
  • GRP-2365diagnostic classes should be removed from WS since they are in API
  • GRP-2364morphString should support EL scriptlets
  • GRP-2363v2.4 unit test fixes (round 1
  • GRP-2362GrouperSystem cant be found


Grouper Wiki (partial list)


Grouper Training in November


Grouper at Tech Ex in December in New Orleans

Wed.  Dec 11 , 4pm - 4:50pm

https://meetings.internet2.edu/2019-technology-exchange/detail/10005596/


Next Grouper Call: Thursday. Nov. 14, 2019 at 11:30am (changed from usual Wed slot due to collision w Grouper Training)

  • No labels