Child pages
  • 13-May-2020
Skip to end of metadata
Go to start of metadata

 

Attending 

  • Chris Hyzer, Penn, Chair
  • Chad Redman, University of North Carolina Chapel Hill
  • Shilen Patel, Duke
  • Carey Black, the Ohio State University
  • Jeff Williams, University of North Carolina Greensboro
  • Vivek Sachdiva, independent
  • Matt Wolfley, Unicon
  •  Emily Eisbruch, Internet2


  Action Items

 Grouper Action Items are here  

New Action Items from this call

  • AI Chris to create Wiki page for next steps on provisioning
  • AI Chris and Chad to discuss the header UI issue and  new tasks  (Note this page should be renamed to include the word header if it’s to be kept : Grouper v2.5 customize UI)



 DISCUSSION

Grouper Training June 2-3, 2020 - Online https://www.incommon.org/grouper-school-virtual/

Current work tasks, and next tasks


Vivek

  • Daemon configurations, made progress, 
  • Need to work on 
    • scheduling , including multiple instances
    • Work on validation 
    • And confirmation message for delete
    • Show scheduled jobs on a list
  • Chris has grouper loader job for scheduling jobs, Vivek can call that method
  • Things that are not multiple are configured in base, example weekly report
    • For those, they should not be available for add
  • Shilen: all could be done using config screen, but this screen makes it easier
  • Like external system config can all be done in the UI or manually, but it’s a pain
  • This approach makes it more reliable and easier
  • Could be helpful for custom composite also
  • Could this model be extended? 
  • Yes, Chris and Vivek discussing this, make a common utility class 
  • Some things here are specific to daemons
  • But much of the architecture can be generalizable to other features
  • Carey: are these classes implementable by deployers as well? 
  • Yes
  • Can UI support dynamically adding properties, so base class is all you need, 
  •  now that’s in Java, but we could expand
  • JSON can handle, if you add one property, it will ask you for others
  • If you are adding another job, you get 3 prompts, class , quartz and priority
  • If you make changes in config editor, and you come back to it in demon config, you will see the other properties and can edit them
  • Vivek will add ability to  add as disabled, default is enabled
    • In case for example, you are trying to get a framework in place before you run it

 

Chris  

  • Container work,  Grouper v2.5 customize container config files
  • Previously was one long library script
  • Was not clear at what point environment variables were set or things were edited
  • Now there is workflow of when container starts
  • Entry point script
  • Setting pipes
  • Script you can override , can have hooks at points in startup
  • Whatever you pass into container will run
  • If you are only running one thing, such as WS only,  it’s somewhat optimized 
  • Can overlay files, for Overlay, does a check first
  •  Can adjust files in a sub image
  • Can change docker entry point but not advised
  • Can add a custom shell hook
  • Comment: this flexibility is great, makes a lot of sense
  • There are cases where new images might change certain base files in ways where hook replacement would not work as well.
  • Changing environment variable is safer than tweaking a file
  • Convention around naming functions
  • Runs files at beginning, Run export all at end, then once it’s all done, call each one to unset each one, function export , and then un-export at end
  • Unit tests are built into the container
  • https://spaces.at.internet2.edu/display/Grouper/Grouper+v2.5+container+unit+tests
  • Tests that make a sub image
  • Would be good to expand these unit tests for other situations (such as WS only)
  • Shilen: morph decrypt possible? To handle secrets in local repo
  • Don't want to advertise how to decrypt w morph
  • Should there be a method to decrypt if you know the key?
  • Chris will work on this
  • If running container as single user, should user have write access to config settings?
  • Issue of what Grouper can write to
  • Issue with current container, Michael G, when run things as GSH, database config not being read, Chris will look into this
  • Not initialize hibernate correctly?

 

Shilen  

  • Fixing unit tests
  • For expirable cache was failing 40%
  • Added a sleep to the task
  • Added fix to WS for disabled attributes on disabled memberships
  • Fix for non grouper admins managing attributes
  • Small fixes to the google provisioner
  • Need to have it working for proof of concept at Duke
  • Performance issues as it is
  • Also look at cache???
  • Race condition  
  • Full sync at same time as incremental, problem w incremental getting missed
  • AI Chris to create Wiki page for next steps on provisioning


Chad 

    • Visualization bug fixed 
    • Problem involved new sessions and JSON spiders
    • New ticket re 3rd party security scanners
    • Set browser security header
    • Policy header
    •  
    • Grouper v2.5 customize UI  (  this wiki page should be renamed to include the word "header")
    • String to define what sites can put your page in an iframe
    • Content security policy header
    • Browser based things look at this header
    • Tomcat has built in header
    • If doing Load balancing can add the header there
    • Are there defaults we should set ?
    • Can be turned on at TOMEE level, without needing to set at the application level
    • Should we build this into Grouper?
    • This does not matter for web services
    • Put this in the Grouper UI filter? 
    • Can take care of this in the Apache config
    • Existing UI filter, set in UI properties
    • Only works for things that apply to that UI filter class, not images
    • More control at web.xml level
    • Want to keep web.xml as thin as possible
    • Could dynamically add the filter


  • AI Chris and Chad to discuss the header UI issue and   new tasks                      


Security Topic

  •   Chris will announce


Issue Roundup


InCommon Grouper Slack


Paul R   roadmap for the 2.6 and 2.7 release dates? 

Chris Hyzer we will shoot for yearly... so april 2021 and april 2022


Darren Boss 

    TAP container displaying accented characters that I wasn't having with a previous setup. 


Darren   when running a create groups from attributes type loader job, We have 100s of organizations and one of the organization name contains a &.   makes the loader choke. It's postgres that has the issue 


 covered group creation form attributes? Does the group that holds the loader config always have to live at the base of the structure it creates? Group names will always have the base of that group?

 

Sudheer   

 Can we create a loader job with SQL that looks like this:

SELECT USER_ID AS SUBJECT_ID_OR_IDENTIFIER FROM ALL_TABLES;


Christopher B is there WSDL for the SOAP web services?

 

Andy  M updated container from v2.4 to v2.5 with auto-DDL, and I saw the following SQL at the end of the DDL changes: "update grouper_ddl set db_version = 32, last_updated = '2020/04/30 13:55:49', history = '2020/04/30 13:55:49: upgrade Grouper from V31 to V62, 2020/04/17 07:18:57: upgrade Grouper from V30 to V31, ..."  Note the history saying, "from V31 to V62".  Shouldn't that be "V31 to V32"?


Lacey  point me in the right direction for Grouper/Confluence integration? 

 

Darren doing pretty good getting the regex implemented for my org loader until I hit our organizations from Québec  

 

Sudheer 

can we create a loader job with SQL that looks like this:

SELECT USER_ID AS SUBJECT_ID_OR_IDENTIFIER FROM ALL_TABLES;

 

Chris Hyzer  comanage and grouper containers and status endpoints for orchestration 


Darren  In my IdP setup the only file I needed to tweak was the access-control.xml file that contains the ip addresses/range for clients that can access the status url.

 

Shilen  ip address based restrictions on the status page will be there in the next release?  https://todos.internet2.edu/browse/GRP-2719


Darren  the IdP container and status, not grouper. If ip address configuration for the status page were in the next release that would be great.

 

Chad  any chance this would work in a modified web.xml?

 

Chris Hyzer   you are to modify the web.xml if you want... so if that is valid web.xml then it should work right?  :slightly_smiling_face:


Chris Hyzer  

  GSH scripts in 2.5.26+ will no longer absorb errors and continue processing. 

Chris Hyzer  

Implemented  auto assign relevant attributes idea  


Chris Hyzer  grace periods renamed to "recent memberships" to be more generic 


Chris Hyzer   in v2.5.26+ there will be a quick start (or maturity level -1).   


Ross     grouper-installer-2.5.X.jar intended to work on Windows/Docker Desktop? It fails trying to start "sh"


Sudheer     Is there a easy way to delete groups in bulk (like 100K groups) using SQL? deleting them from UI might take forever.

 

Jeffrey C   grouper shell would be better  


Michael G  if you are on latest grouper - you can delete from the UI an entire folder and everything beneath it and it will background the removal.  when deleting a folder - you can even be selective to some degree of what you delete as there is a form prompting you for what to delete.

 

Michael G     loader jobs used to sort the subjobs -   Now I see the loader jobs no longer sort the subjobs.   


Jeffrey C    get the red ERROR on the all daemon jobs page, f 


Sudheer    Can we add a rule on "root" folder that inherits privileges (eg. read privilege) of a subject/group to folders/groups under root? 

  

Sudheer    I tried  similar thing where i created a "globalReadAccess" group, added members to that group and tried to give read privileges to this group on root folder


Carey    There are times that the Grouper tools/features use UUID's of groups, or members, etc.. as attribute values.   But I think the UI does not have a way to search on those values.

 

Andy      storing my morphstring in a file separate from morphstring.properties, and the key has a unix newline (\n) as the last character in the file….   Does it actually only care about whitespace, not newlines?


Michael G   looking at the changes to get to 2.5.27.  in GRP-2749 - the only thing i am changing is the timeout.  I set it to 60000  .   Would you consider the passing in of an ENV var (PROXY_TIMEOUT) so an overlay isn’t needed?  Simplified config.  Maybe other ENV vars as well?


Alex P    

 curious problem in 2.5.23. I have a connector that tries to load a properties file from within its jar like this:

input = ClassLoader.getSystemResourceAsStream("grouper-oracle-connector.properties")

This fails when running the daemon through tomee. But works okay when I launch the job from GSH.  


Chris Hyzer   we should put something in gsh.sh that requires that it run as user "tomcat" in the container.  This is a security improvement..,  


Carey    giving i2incommon/grouper:2.5.27 a spin...     the "Miscellaneous > Configure > Configuration files" bombs out on the "grouper-ui.properties" file.   ( All other files render ok.)    


Carey     GRP-2749 "AKA: improve the tomcat and apache contexts"

 question : How does one change the /grouper part of the URL? 


Michael G    much like there is the option to delete: “Delete 1 folders if they are empty” in the UI - could we get the same option for empty groups?  Happy to file a Jira if so.


Alex P   move to prod deploy for 2.5 has run into hiccups: the auto DDL update is failing on grouper_groups_v


Scott K    using the image i2incommon/grouper:2.5.27 and trying to exercise the new environment variable functionality  


  Andy M    Will the Grouper container be updated to use Shib SP 3.1?


Andy M   In the 2.5.23 container, there seems to be an extra "classes" directory

 

Scott K   In the 2.5.27 container the SELF_SIGNED_CERT environment variable being “false” does not seem to prevent having HTTPS enabled 

 

Ryan R    Been getting   OOM exceptions this morning in our UI service -  

 

Scott K  

Is the encrypted value generated by edu.internet2.middleware.morphString.Encrypt when using the morph string   dependent on the version of Java used, 


Josh O   Is there a reference doc page for v2.5, configurable ENV variables?  Like GROUPER_LOG_TO_HOST, GROUPER_MAX_MEMORY, etc?

 

Shilen     have a graph of membership counts in our custom UI that i've been meaning to add to the grouper ui for a while - https://todos.internet2.edu/browse/GRP-2279

 

Chris Hyzer    

  if you PIT restore, then it deletes permanently what happened that you dont watn right?  maybe an option to just sync membership to now, but keep the bad stuff in PIT?

  

Andy     could use a "restore" feature too.   


Andy     had an admin accidently replace the membership when he meant to add new members.  

Christopher Bongaarts    be able to "git diff" as well as "git checkout" for comparing vs. restoring to a point in time

  

Lacey   errors in a previously working PSPNG log


Chris Hyzer  exciting container developments in the upcoming version v2.5.28:

 

Josh     trying out building v2.5.27 as a subcontainer with a Dockerfile in order to use the ENV variables.   when I start the container, the properties don't seem to be applied,  


Carey   Is there any supported/designed way to escape a colon character in a Group Name and/or Display Name?

   

Alex  looking over the env options in 2.28. I think I'll be able to get rid of a lot of my customizations, but one thing I don't see is the option to change the tomcat port.  


Andy   

  isn't there an endless list of options that might be injected as environment variables for customization?  Perhaps this deserves a different approach.  For example, can't docker map the container's port to a different port on the docker host?

 

Greg H   Arguably, with much of the environment setup hookable, very little needs to be readily passable into the container.


Andy    I was thinking of the arguments to start the container, like "docker run --detach --publish 80:80 --publish 443:443 ..."

 

On our dedicated VM running Docker, the container does all the work, so we map 443 to 443, but you can map 443 in the container to 8443 in the host or whatever port you want.


Alex     within our orchestration setup (AWS ECS) is handling all the port mapping for us, more or less.  

 

Grouper Wiki updates


JIRAS

Grouper-Users List

Re: [grouper-users] Upgrade process from Grouper 2.2.2 to 2.5, Robert Bradley, 05/13/2020

  • No labels