Versions Compared

Old Version 98

changes.mady.by.user Chris Hyzer (upenn.edu)

Saved on

compared with

New Version 108

changes.mady.by.user Chris Hyzer (upenn.edu)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Setting gsh.useLegacy = true in grouper.properties.
  • Using a command line argument  (gsh.sh -forceLegacyGsh)

Escape

Escape things in groovysh with single backslash.  e.g.

Code Block
attributeValueDelegate.assignValue(RuleUtils.ruleIfConditionElName(), "\${subject.sourceId != 'g:gsa'}");


API Compability

gsh is now a core part of the Grouper API and so is always compatible with the current release.

...

Command

Description

grouperSession = GrouperSession.startRootSession();
loaderGroup = GroupFinder.findByName(grouperSession, "stem:group");
loaderRunOneJob(loaderGroup);

Kick off the loader for one group (configured by group attributes)

loaderRunOneJob("MAINTENANCE_cleanLogs");

Kick off the loader by job name

loaderRunOneJob("CHANGE_LOG_changeLogTempToChangeLog");

Move change log entries from the temp table to the real table

loaderRunOneJob("CHANGE_LOG_consumer_grouperRules");
loaderRunOneJob("MAINTENANCE__rules");

Run the Grouper Rules daemon (the changelog or full version)

loaderRunOneJob("CHANGE_LOG_consumer_test");

Run a change log consumer

GrouperLoaderType.validateAndScheduleSqlLoad(group, null, false)Schedule SQL job
GrouperLoaderType.validateAndScheduleLdapLoad(attributeAssign, null, false)Schedule LDAP job
GrouperLoaderType.scheduleAttributeLoads();Schedule all attribute loader jobs

This query (in Oracle) will find jobs with no success in the last day and make a gsh script:

...

You can run the loader as a linux service

GrouperShell Variables (BeanShell only)

gsh has several variables that can be set to modify runtime behavior

Jobs not firing in daemon

  1. Stop all daemons

  2. Run these sqls from a file: fixLoaderScheduler.sql

    Code Block
    UPDATE GROUPER_QZ_TRIGGERS SET TRIGGER_STATE = 'WAITING';
DELETE FROM GROUPER_QZ_FIRED_TRIGGERS;
commit;



    Code Block
    gsh.sh -registry -runsqlfile fixLoaderScheduler.sql



  3. Restart daemons


It took a couple of hours to catch up on a few days of changes, but it seems to be back to normal. Thanks again, guys!

GrouperShell Variables (BeanShell only)

gsh has several variables that can be set to modify runtime behavior

Variable

Description

Variable

Description

GSH_DEBUG

Stack traces will be printed upon failure if true

GSH_DEVEL

Summaries of returned objects are not automatically printed if true

GSH_TIMER

Prints time spent evaluating each command if true

...

Code Block
# (1) Print tab-separated summary of all group members, and flags for direct, indirect, or both
# Depending on the results, you could use the data to create a scrutinized list of Ids to delete, then import it and delete in a loop

me = SubjectFinder.findByIdentifierAndSource("my-username", "pid", true);
session = GrouperSession.start(me);
// OR: session = GrouperSession.startRootSession(True)

group = GroupFinder.findByName(session, "tmp:my:group", true);

effectiveMembers = group.getEffectiveMembers();
immediateMembers = group.getImmediateMembers();

System.out.println(String.join("\t", "id", "name", "Effective", "Immediate"));

for (Member m: group.getMembers()) {
    System.out.print(m.getSubject().getId() + "\t" + m.getSubject().getName() + "\t");
    System.out.print(effectiveMembers.contains(m).toString() + "\t");
    System.out.println(immediateMembers.contains(m).toString() + "\t");
}



# (2) Get the immediate and effective members for a specific source ("pid" in this example), intersect them to find the redundant ones
# This has a dryRun flag, so you can test first

sources = new HashSet<Source>()
sources.add(SourceManager.getInstance().getSource("pid"))

effectiveUsers = group.getEffectiveMembers(Group.getDefaultList(), sources, null)
immediateUsers = group.getImmediateMembers(Group.getDefaultList(), sources, null)

# use retainAll() to find the intersection; i.e., users both as effective and immediate member
immediateUsers.retainAll(effectiveUsers)

System.out.println("There are " + immediateUsers.size() + " users having both direct + indirect memberships");

dryRun = true

for (Member m: immediateUsers) {
    if (dryRun) {
        System.out.println("Ok to delete " + m.getSubject().getId());
    } else {
        System.out.println("Deleting " + m.getSubject().getId());
        group.deleteMember(m, false);
    }
}

# (3) Get the groups this subject is a member of. Note that a group is a kind of subject, and has a toSubject() method to convert it.

import edu.internet2.middleware.grouper.membership.MembershipSubjectContainer

GrouperSession grouperSession = GrouperSession.startRootSession();

Group group = GroupFinder.findByName(grouperSession, "test:testGroup", true);
Subject subject = g.toSubject();

Set<MembershipSubjectContainer> msc = new MembershipFinder().addSubject(subject).findMembershipResult().getMembershipSubjectContainers();

for (MembershipSubjectContainer membershipSubjectContainer : msc) { println(membershipSubjectContainer.getGroupOwner().getName());}

//Note there are a few other options for the search. Add these to the MembershipFinder method chain before calling findMembershipResult():
//  - search immediate, effective, etc. (needs to import MembershipType)
import edu.internet2.middleware.grouper.membership.MembershipType
membershipFinder.assignMembershipType(MembershipType.IMMEDIATE) // options are IMMEDIATE|NONIMMEDIATE|EFFECTIVE|COMPOSITE
//  - retrieve specific groups based on pattern
membershipFinder.assignScope("%:test:%")
//  - Enabled status -- true means enabled only, false, means disabled only, and null means all
membershipFinder.assignEnabled(false)
// For other methods, refer to the Javadoc at http://internet2.github.io/grouper/master/grouper-parent/apidocs/edu/internet2/middleware/grouper/MembershipFinder.html

Misc


Configuration in the database

In v2.4.0 ui patch #56, and GSH must be run from the UI in WEB-INF/bin

Code Block
GrouperSession.startRootSession();

ADD
new edu.internet2.middleware.grouper.grouperUi.beans.config.GrouperDbConfig().configFileName("grouper.properties").propertyName("abc").value("123").store();

DELETE
new edu.internet2.middleware.grouper.grouperUi.beans.config.GrouperDbConfig().configFileName("grouper.properties").propertyName("abc").delete();

IMPORT
new edu.internet2.middleware.grouper.grouperUi.beans.config.GrouperDbConfigImport().configFilePath("d:/temp/temp/grouper.properties").store();


Misc

NoteNote: you cannot encrypt passwords with GSH since the passwords end up in the GSH history.  To encrypt passwords, issue the command:

...

v2.0: to sync up the point in time tables with regular tables, run this:

Code Block
new edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables()

To create missing group sets:

:

Code Block
new edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables()

To create missing group sets:

Code Block
new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().addAllMissingGroupSets();

Delete memberships not in transaction

Code Block
grouperSession = GrouperSession.startRootSession();
group = GroupFinder.findByName(grouperSession, "test:testGroup3", true);
for (membership : group.getImmediateMemberships()) {membership.delete();}
group.delete();

Note: in v2.4.0 patch 91+ ( unreleased at the time of writing ) you can use gsh to do simple sql tests through jdbc loader connection

Code Block
languagejava
gcDbAccess =
Code Block
new edu.internet2.middleware.groupergrouperClient.misc.AddMissingGroupSets().addAllMissingGroupSetsjdbc.GcDbAccess();

Delete memberships not in transaction

Code Block
grouperSession = GrouperSession.startRootSession();
group = GroupFinder.findByName(grouperSession, "test:testGroup3", true);
for (membership : group.getImmediateMemberships()) {membership.delete();}
group.delete(// "loaderConnection" is the string used in the grouper-loader.properties ( Example: db.warehouse.url --> "warehouse")
gcDbAccess.connectionName("loaderConnection").sql("select count(1) from test1").select(int.class);


See the WIKI for running the Grouper Report manually

...

Code Block
GrouperSession grouperSession = GrouperSession.startRootSession();
AttributeAssign attributeAssign = AttributeAssignFinder.findById("b629bd8170964663be507968752f4f17", true);
attributeAssign.delete(); = GrouperSession.startRootSession();
AttributeAssign attributeAssign = AttributeAssignFinder.findById("b629bd8170964663be507968752f4f17", true);
attributeAssign.delete();

NOTE: You can also use the AttributeAssignFinder.findById(String id, boolean exceptionIfNull)  to find attribute assignments from the logs too.
Example log "ERROR RuleEngine$3.callback(560) - - Error with daemon on rule: attributeAssignTypeId: 3d6ccb6c5a584f32919682ae154c0523". id="3d6ccb6c5a584f32919682ae154c0523". The returned AttributeAssign object will show you the stem/group that the attribute is attached to. (smile)

Grouper Builtin Messaging

...

Code Block
    String provisionTarget = "ad";
    GrouperSession grouperSession = GrouperSession.startRootSession();
    
    Set stemsToProvisionToSet = HibernateSession.byHqlStatic().createQuery("select s from Stem s, AttributeAssign aa, AttributeDefName adn, AttributeAssignValue aav where s.id = aa.ownerStemId and aav.attributeAssignId = aa.id and aa.attributeDefNameId = adn.id and aa.attributeAssignTypeDb = 'stem' and aa.enabledDb = 'T' and adn.extensionDb = 'provision_to' and aav.valueString = '" + provisionTarget + "'").listSet(Stem.class);
    for (Object stemObject : stemsToProvisionToSet) { Stem stem = (Stem)stemObject; System.out.println("provision_to assigned to stem: " + stem.getName());  }
    Set stemsToNotProvisionToSet = HibernateSession.byHqlStatic().createQuery("select s from Stem s, AttributeAssign aa, AttributeDefName adn, AttributeAssignValue aav where s.id = aa.ownerStemId and aav.attributeAssignId = aa.id and aa.attributeDefNameId = adn.id and aa.attributeAssignTypeDb = 'stem' and aa.enabledDb = 'T' and adn.extensionDb = 'do_not_provision_to' and aav.valueString = '" + provisionTarget + "'").listSet(Stem.class);
    for (Object stemObject : stemsToNotProvisionToSet) { Stem stem = (Stem)stemObject; System.out.println("do_not_provision_to assigned to stem: " + stem.getName());  }
    Set groupsToProvisionToSet = HibernateSession.byHqlStatic().createQuery("select g from Group g, AttributeAssign aa, AttributeDefName adn, AttributeAssignValue aav where g.id = aa.ownerGroupId and aav.attributeAssignId = aa.id and aa.attributeDefNameId = adn.id and aa.attributeAssignTypeDb = 'group' and aa.enabledDb = 'T' and adn.extensionDb = 'provision_to' and aav.valueString = '" + provisionTarget + "'").listSet(Stem.class);
    for (Object groupObject : groupsToProvisionToSet) { Group group = (Group)groupObject; System.out.println("provision_to assigned to group: " + group.getName());  }
    Set groupsToNotProvisionToSet = HibernateSession.byHqlStatic().createQuery("select g from Group g, AttributeAssign aa, AttributeDefName adn, AttributeAssignValue aav where g.id = aa.ownerGroupId and aav.attributeAssignId = aa.id and aa.attributeDefNameId = adn.id and aa.attributeAssignTypeDb = 'group' and aa.enabledDb = 'T' and adn.extensionDb = 'do_not_provision_to' and aav.valueString = '" + provisionTarget + "'").listSet(Stem.class);
    for (Object groupObject : groupsToNotProvisionToSet) { Group group = (Group)groupObject; System.out.println("do_not_provision_to assigned to group: " + group.getName());  }
    Set allGroups = new LinkedHashSet();
    Set allGroupsToProvision = new TreeSet();
    allGroupsToProvision.addAll(groupsToProvisionToSet);

    Set stemNamesToNotProvisionTo = new HashSet();
    Set stemNamesToProvisionTo = new HashSet();
    
    for (Object stemToProvision : stemsToProvisionToSet) { stemNamesToProvisionTo.add(((Stem)stemToProvision).getName()); }
    for (Object stemNotToProvision : stemsToNotProvisionToSet) { stemNamesToNotProvisionTo.add(((Stem)stemNotToProvision).getName()); }

    for (Object stemToProvision : stemsToProvisionToSet) { allGroups.addAll(((Stem)stemToProvision).getChildGroups(edu.internet2.middleware.grouper.Stem.Scope.SUB)); }
    
    Map groupToPaths.Scope.SUB)); }
    
    Map groupToPaths = new HashMap();
    for (Object groupObject : allGroups) { Group group = (Group)groupObject; if (allGroupsToProvision.contains(group)) {continue;} if (groupsToNotProvisionToSet.contains(group)) {continue;} List paths = new HashMapArrayList();
    for (Object groupObject : allGroups groupToPaths.put(group, paths); String currentName = group.getName(); paths.add(currentName);  while(true) { Group groupcurrentName = GrouperUtil.parentStemNameFromName(GroupcurrentName)groupObject;  if (allGroupsToProvisionGrouperUtil.containsisBlank(groupcurrentName)) {continue;} if (groupsToNotProvisionToSet.contains(group)) {continue;}break;} paths.add(currentName);  }   }
    
    for (Object groupObject : groupToPaths.keySet()) {Group group = (Group)groupObject; List paths = new ArrayList(List); groupToPaths.putget(group,); for (Object pathObject : paths); { String currentNamepath = group.getName(String)pathObject; if paths(stemNamesToProvisionTo.addcontains(currentName);  while(truepath)) { currentName = GrouperUtil.parentStemNameFromName(currentName); allGroupsToProvision.add(group); break; } if (GrouperUtilstemNamesToNotProvisionTo.isBlankcontains(currentNamepath)) { break;} paths.add(currentName);  } }   }
    
    for (Object groupObject : groupToPaths.keySet()allGroupsToProvision) { Group group = (Group)groupObject; List paths = (List)groupToPaths.get(group); for (Object pathObject : paths) { String path = (String)pathObject; if (stemNamesToProvisionTo.contains(path)) { allGroupsToProvision.add(group); break; } if (stemNamesToNotProvisionTo.contains(path)) { break; } } }
    
    for (Object groupObject : allGroupsToProvision) { Group group = (Group)groupObject; System.out.println("configured to provision to: " + provisionTarget + ": " + group.getName()); }

; System.out.println("configured to provision to: " + provisionTarget + ": " + group.getName()); }


Stem move

try this:

Code Block
GrouperSession.startRootSession();
stemFrom = StemFinder.findByName(grouperSession, "a:b", true);
stemTo = StemFinder.findByName(grouperSession, "a:c", true);
new edu.internet2.middleware.grouper.StemMove(stemFrom, stemTo).assignAlternateName(false).save();

Check health of database connection or run a query

(in 2.4.0 api patch 93+)

Code Block
gcDbAccess = new edu.internet2.middleware.grouperClient.jdbc.GcDbAccess();
gcDbAccess.connectionName("warehouse").sql("select count(1) from grouper_groups").select(int.class);