GrouperShell (gsh)

gsh is a command line shell for administering and interacting with the Grouper API. See architectural diagram.  It can be used in both a batch and interactive manner.  It is built on Java BeanShell

API Compability

gsh is now a core part of the Grouper API and so is always compatible with the current release.

Installation

When using the Grouper API source distribution, grouper.jar needs to be built before using gsh.sh for the first time:

cd $GROUPER_HOME
ant dist

Usage

For Windows use $GROUPER_HOME\bin\gsh.bat

Run gsh as an interactive shell:

$GROUPER_HOME/bin/gsh.sh

Read gsh commands from STDIN:

$GROUPER_HOME/bin/gsh.sh -

Read gsh commands from a script file:

$GROUPER_HOME/bin/gsh.sh /path/to/your/script.gsh

Run Grouper utilities:

$GROUPER_HOME/bin/gsh.sh <option>
args: -h,               Prints this message
args: -check,           Performs startup check and enters an
                        interactive shell
args: -runarg <command> Run command (use \\n to separate commands)
args: -main <class> [args...]
   class,               Full class name (must have main method)
   args,                args as required by main method of class
args: -initEnv [<configDir>]
       On Windows sets GROUPER_HOME and adds GROUPER_HOME/bin to path
       For *nix 'source gsh.sh' for the same result
       configDir optionally adds an alternative conf directory than
       GROUPER_HOME/conf to the classpath
args: (-xmlimport | -xmlexport | -loader | -test | -registry | -usdu |
       findbadmemberships)
                        Enter option to get additional usage for that
                        option
  -xmlimport,           Invokes XmlExporter
  -xmlexport,           Invokes XmlImporter
  -loader,              Invokes GrouperLoader
  -registry,            Manipulate the Grouper schema and install
                        bootstrap data
  -test,                Run JUnit tests
  -usdu,                Invoke USDU - Unresolvable Subject Deletion
                        Utility
  -findbadmemberships,  Check for membership data inconsistencies

Supported Commands

Grouper API methods

Any Grouper API method can be directly invoked just by referencing it, inclusive of the class in which it is defined. Methods return a java object which can be stored in a variable. For example, the following gsh session determines all of the groups to which a given subject belongs:

gsh 0% GrouperSession.startRootSession();
gsh 0% subj = findSubject("SD00125")
subject: id='SD00125' type='person' source='kitn-person' name='Barton, Tom'
gsh 1% sess = GrouperSession.start(subj)
edu.internet2.middleware.grouper.GrouperSession: 29c40f97-9fb0-4e45-88bc-a14877a6c9b5,'SD00125','person'
gsh 2% member = MemberFinder.findBySubject(sess, subj)
member: id='SD00125' type='person' source='kitn-person' uuid='d0fa765e-1439-4701-89b1-9b08b4ce9daa'
gsh 3% member.getGroups()
group: name='etc:sysadmingroup' displayName='Grouper Administration:SysAdmin Group' uuid='6f77fb36-b466-481a-84a7-7af609f1ad09'

Groups

Command

Description

addGroup(parent stem name, extension, displayExtension)

Add group to registry

delGroup(name)

Delete group from registry

getGroupAttr(group name, attr)

Get value of group attribute

getGroups(name)

Find all groups with a matching naming attribute value, returns a Set of groups

setGroupAttr(group name, attr, value)

Set value of group attribute

GroupFinder.findByName(grouperSession, name)

Find one group by name

GroupFinder.findByUuid(grouperSession, name)

Find one group by uuid

You can use GroupSave as an alternate way:

new GroupSave(grouperSession).assignName("stem1:a").assignCreateParentStemsIfNotExist(true).save();

Group Types

Command

Description

groupAddType(group name, type name)

Add type to group

groupDelType(group name, type name)

Delete type from group

groupGetTypes(group name)

Get group's types

groupHasType(group name, type name)

Check whether group had type

typeAdd(type name)

Create custom group type

typeAddAttr(type name, attr name, read, write, required)

Create custom group attribute. read and write must be an AccessPrivilege (e.g. AccessPrivilege.ADMIN)

typeAddList(type name, attr name, read, write)

Create a custom list. read and write must be an AccessPrivilege (e.g. AccessPrivilege.ADMIN).

typeDel(type name)

Delete group type

typeDelField(type name, field name)

Delete custom field from group type

typeFind(type name)

Find the group

typeGetFields(type name)

Get fields associated with the group type

Member change subject

Change subject of a Member object, e.g.:

grouperSession = GrouperSession.startRootSession();
oldSubject = findSubject("10021368");
member = MemberFinder.findBySubject(grouperSession, oldSubject);
newSubject = findSubject("10021366");
member.changeSubject(newSubject);

Command

Description

member.changeSubject(newSubject);

Change the subject of the member object.  If the subject is the same, its a no-op.  If the new subject does not have a Member object, then the existing member object simply gets new subject information.  If the new subject does have a member object, then all objects in the grouper registry which uses the old member, will be updated to the new member.  Then the old member object is deleted from the registry

member.changeSubject(newSubject,!Member.DELETE_OLD_MEMBER);

Change the subject, but dont delete the old member.  Do this if the way which deletes the old member doesnt work due to foreign keys.  This will do all the work it can, and the rest can be manual

member.changeSubjectReport(newSubject,Member.DELETE_OLD_MEMBER);

Dont do any of the work, just print a report to the screen of what will be done.  Dry-run.


Memberships

Command

Description

addComposite(group name, composite type, left group name, right group name)

Add composite membership.  e.g. CompositeType.UNION

addMember(group name, subject id)

Add member to the members list for the group.

addMember(group name, subject id, field)

Add member to the specified list for the group.

delComposite(group name)

Delete composite membership from group

delMember(group name, subject id)

Delete member from the members list for the group

delMember(group name, subject id, field)

Delete member from the specified list for the group

getMembers(group name)

Get members of group

hasMember(group name, subject id)

Check whether subject is member of the members list

hasMember(group name, subject id, field)

Check whether subject is member of the specified list

Privileges

Command

Description

grantPriv(group name, subject id, privilege)

Grant privilege on group. privilege must be an AccessPrivilege (e.g. AccessPrivilege.ADMIN)

grantPriv(stem name, subject id, privilege)

Grant privilege on stem. privilege must be a NamingPrivilege (e.g. NamingPrivilege.STEM)

hasPriv(group name, subject id, privilege)

Check whether subject has privilege on group. privilege must be an AccessPrivilege (e.g. AccessPrivilege.ADMIN)

hasPriv(stem name, subject id, privilege)

Check whether subject has privilege on strem. privilege must be a NamingPrivilege (e.g. NamingPrivilege.STEM)

revokePriv(group name, subject id, privilege)

Revoke privilege on group. privilege must be an AccessPrivilege (e.g. AccessPrivilege.ADMIN)

revokePriv(stem name, subject id, privilege)

Revoke privilege on stem. privilege must be a NamingPrivilege (e.g. NamingPrivilege.STEM)

Registry

Command

Description

registryInitializeSchema()

Will generate schema DDL for the DB, and wont drop before creating, will not run script

registryInitializeSchema(registryInitializeSchema.DROP_THEN_CREATE)

generate DDL for the DB, dropping existing tables, will not run script

registryInitializeSchema.WRITE_AND_RUN_SCRIPT)

generate DDL for the DB, not dropping, but will run the script after writing it to file

registryInitializeSchema(registryInitializeSchema.DROP_THEN_CREATE | registryInitializeSchema.WRITE_AND_RUN_SCRIPT)

generate DDL for the DB, drop existing grouper tables, and run the script after writing it to file

resetRegistry()

Restore registry to default state(delete data from all tables, install defaults)

registryInstall()

If the default Grouper data is not there, it will be added (e.g. root stem, default fields, etc)

Stems

Command

Description

addRootStem(extension, displayExtension)

Add top-level stem to the registry

addStem(parent stem name, extension, displayExtension)

Add stem to registry

delStem(stem name)

Delete stem from registry

obliterateStem(stem name, testOnlyBoolean, deleteFromPointInTimeBoolean)    (Grouper v2.0.2+)

Delete stem, and subobjects.

If testonly (true|false), then only
print a report.  This is not supported when deleteFromPointInTime is true.

If deleteFromPointInTime (true|false), then delete from point in time only.  Otherwise, point in time records are not deleted.

Note that point in time data can only be deleted after the actual objects have been deleted and those deletions have been processed by the changeLogTempToChangeLog job, which runs once a minute by default with the Grouper Daemon.  So if you want to completely delete a stem (including point in time data), then run this GSH command with deleteFromPointInTime=false, then wait a minute or so, then run the GSH command with deleteFromPointInTime=true.

GrouperSession must be open before calling...

getStemAttr(stem name, attr)

Get value of stem attribute

getStems(name)

Find all stems with a matching naming attribute value, returns a Set of stems

setStemAttr(stem name, attr, value)

Set value of stem attribute

StemFinder.findByName(grouperSession, name)

Find one stem by name

StemFinder.findByUuid(grouperSession, uuid)

Find one stem by uuid


grouperSession = GrouperSession.startRootSession();
stem = StemFinder.findByName(grouperSession, "a");
for(child : stem.getChildGroups(Stem.Scope.SUB)) { System.out.println("deleting: " + child.getName()); child.delete();  }
stemList = new ArrayList(stem.getChildStems(Stem.Scope.SUB));
Collections.sort(stemList);
Collections.reverse(stemList);
for(childStem : stemList) { System.out.println("deleting: " + childStem.getName()); childStem.delete(); }
stem.delete();

Delete stem and subcontents

Subjects

Command

Description

addSubject(id, type, name)

Add local subject to registry

findSubject(id)

Find a subject

findSubject(id, type)

Find a subject

findSubject(id, type, source)

Find a subject

getSources()

Find all Subject sources

System

Command

Description

sqlRun(file)

Execute each line of a sql file, just like ant would.  This can run the files generated by registryInitializeSchema()

sqlRun(string)

Executes a single sql statement

exit

Terminate shell

help()

Display usage information

history()

Print commands that have been run

history(N)

Print the last N commands that have been run

last()

Run the last command executed

last(N)

Execute command number N

p(command)

Pretty print results. This command is more useful when GSH_DEVEL is enabled

quit

Terminate shell

version()

Return version information

Unresolvable subject deletion utility (USDU)

usdu finds which memberships are with subjects which cannot be found in a subject source, and prints them on the screen
- if the usdu.DELETE option is passed in, then the memberships will be deleted
- a grouper session must be open when this command is run.

 For more information, see Unresolvable Subject Deletion Utility (USDU)

Command

Description

GrouperSession.startRootSession();
 usdu()

Sample call to find all unresolvable subjects in the registry and print details to the screen

usdu(usdu.DELETE)

Pass in that you want to delete memberships in the usdu call

usduBySource("schoolperson")

Work only in a specific subject source, pass in the sourceId from sources.xml

usduBySource("schoolperson", usdu.DELETE)

Work in a specific source and delete membeships

subject=SubjectFinder.findById("GrouperSystem")
 session=GrouperSession.start(subject)"
 memberSubject=SubjectFinder.findById("1234567")
 member=MemberFinder.findBySubject(session,memberSubject)
 usduByMember(member)

Work only with a specific member

usduByMember(member, usdu.DELETE)

usdu by member, and delete memberships

Find bad memberships

 This command will find membership records in the database which are invalid, and prints them on the screen, along with a GSH script that will fix the memberships.

 For more information, see Bad Membership Finder Utility

Command

Description

findBadMemberships()

complete findBadMemberships run



XML legacy

Command

Description

xmlFromFile(filename)

Load registry from XML in file

xmlFromString(xml)

Load registry from XML in string

xmlFromURL(url)

Load registry from XML at URL

xmlToFile(filename)

Exports registry to file

xmlToString()

Exports registry to string.

xmlUpdateFromFile(filename)

Update registry from XML in file

xmlUpdateFromString(xml)

Update registry from XML in string

xmlUpdateFromURL(url)

Update registry from XML at URL

XML export

There is an object: XmlExport which has various chaining methods, which should be ended with an exportTo() method.  You can export to file or string.

For more information, see Import-Export

Command

Description

XmlExport xmlExport.stem(stem)

The stem to export. Defaults to the ROOT stem.

XmlExport xmlExport.group(group)

The group to export

XmlExport xmlExport.relative(boolean)

If group or stem specified do not export parent Stems.

XmlExport xmlExport.includeParent(boolean)

If group specified, export from the parent stem

XmlExport xmlExport.childrenOnly(boolean)

If stem specified, export child stems and groups only - not the specified stem

XmlExport xmlExport.userProperties(file)

Properties file for extra settings for import

XmlExport xmlExport.grouperSession(grouperSession)

Operate within a certain grouper session (defaults to root session)

void xmlExport.exportToFile(file)

Export to an XML file

void xmlExport.exportToString(string)

Export to an XML string

 Examples:

gsh 1% new XmlExport().exportToFile(new File("c:/temp/export.xml"))
gsh 1% grouperSession = GrouperSession.start(SubjectFinder.findById("mchyzer"));
gsh 2% stem = StemFinder.findByName(grouperSession, "aStem");
gsh 3% new XmlExport().stem(stem).relative(true).userProperties(new File("C:/temp/some.props")).grouperSession(grouperSession).exportToFile(new File("c:/temp/export.xml"));

 -or- (without chaining)

gsh 3% xmlExport = new XmlExport();
gsh 4% xmlExport.stem(stem);
gsh 5% xmlExport.grouperSession(grouperSession);
gsh 6% xmlExport.exportToFile(new File("c:/temp/export.xml"))

XML import

There is an object: XmlImport which has various chaining methods, which should be ended with an importFrom() method.  You can import from file, string, or url.

For more information, see Import-Export

Command

Description

XmlImport xmlImport.stem(stem)

The Stem into which data will be imported. Defaults to the ROOT stem.

XmlImport xmlImport.updateList(boolean)

XML contains a flat list of Stems or Groups which may be updated.
Missing Stems and Groups are not created.

XmlImport xmlImport.userProperties(file)

Properties file for extra settings for import

XmlImport xmlImport.grouperSession(grouperSession)

Operate within a certain grouper session (defaults to root session)

XmlImport xmlImport.ignoreInternal(boolean)

Ignore internal attributes, including group and stem uuids.

void xmlImport.importFromFile(file)

Import from an XML file

void xmlImport.importFromString(string)

Import from an XML string

void xmlImport.importFromUrl(url)

Import XML from a URL

 Examples:

gsh 1% new XmlImport().importFromFile(new File("c:/temp/export.xml"))
gsh 1% grouperSession = GrouperSession.start(SubjectFinder.findById("mchyzer"));
gsh 2% stem = StemFinder.findByName(grouperSession, "aStem");
gsh 3% new XmlImport().stem(stem).updateList(true).userProperties(new File("C:/temp/some.props")).grouperSession(grouperSession).importFromUrl(new URL("http://whatever.xml"));

 -or- (without chaining)

gsh 3% xmlImport = new XmlImport();
gsh 4% xmlImport.stem(stem);
gsh 5% xmlImport.grouperSession(grouperSession);
gsh 6% xmlImport.importFromFile(new File("c:/temp/export.xml"))

Transactions

Transactions facilitate all commands succeeding or failing together, and perhaps some level of repeatable reads of the DB (depending on the DB).  If there is an open transaction and an exception is thrown in a command, GSH will shut down so that subsequent commands will not execute outside of a transaction.

Command

Description

help("transaction")

print help information

transactionStatus()

print the list of nested transactions

transactionStart("<GrouperTransactionType>")

start a transaction, or make sure one is already started
   Can use: "READONLY_OR_USE_EXISTING", "NONE", "READONLY_NEW",
"READ_WRITE_OR_USE_EXISTING", "READ_WRITE_NEW"

transactionCommit("<GrouperCommitType>")

commit a transaction
   Can use: "COMMIT_NOW", "COMMIT_IF_NEW_TRANSACTION

transactionRollback("<GrouperRollbackType>")

rollback a transaction
    Can use: "ROLLBACK_NOW", "ROLLBACK_IF_NEW_TRANSACTION

transactionEnd()

end a transaction
    Note if it was read/write, and not committed or rolled back, this will commit and end

Loader

Above, it describes how you can kick off the loader in daemon mode.  You can also execute one job with:

Command

Description

grouperSession = GrouperSession.startRootSession();
loaderGroup = GroupFinder.findByName(grouperSession, "stem:group");
loaderRunOneJob(loaderGroup);

Kick off the loader for one group (configured by group attributes)

loaderRunOneJob("MAINTENANCE_cleanLogs");

Kick off the loader by job name

loaderRunOneJob("CHANGE_LOG_changeLogTempToChangeLog");

Move change log entries from the temp table to the real table

loaderRunOneJob("CHANGE_LOG_consumer_grouperRules");

Run the Grouper Rules daemon

loaderRunOneJob("CHANGE_LOG_consumer_test");

Run a change log consumer

v1.6+ loader

Command

Description

loaderRunOneJobAttr(attirbuteDef)

Run an attribute definition loader job

GrouperShell Variables

gsh has several variables that can be set to modify runtime behavior

Variable

Description

GSH_DEBUG

Stack traces will be printed upon failure if true

GSH_DEVEL

Summaries of returned objects are not automatically printed if true

GSH_TIMER

Prints time spent evaluating each command if true

    Example:

gsh 4% GSH_DEVEL = true
gsh 5% subj = findSubject("SD00125")
gsh 6% sess = GrouperSession.start(subj)
gsh 7% member = MemberFinder.findBySubject(sess, subj)
gsh 8% p(member.getGroups())
group: name='etc:sysadmingroup' displayName='Grouper Administration:SysAdmin Group' uuid='6f77fb36-b466-481a-84a7-7af609f1ad09'

Misc

Note: you cannot encrypt passwords with GSH since the passwords end up in the GSH history.  To encrypt passwords, issue the command:

C:\mchyzer\isc\dev\grouper-qs-1.2.0\grouper>java -jar lib\morphString.jar
Enter the location of morphString.properties: conf/morphString.properties
Type the string to encrypt (note: pasting might echo it back):
The encrypted string is: ca8a15be4ad0fb45c6f1b3ca0cfd9c9e

v2.0: to sync up the point in time tables with regular tables, run this:

new edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables()

To create missing group sets:

new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().addAllMissingGroupSets();

Create a script from SQL

Here is an example to remove access from someone...  run a SQL to generate a GSH script, e.g. in oracle:

set linesize 1000;
set pagesize 1000;
select 'delMember("' || gmlv.GROUP_NAME || '", "' || gmlv.SUBJECT_ID || '");' as script
from grouper_memberships_lw_v gmlv where subject_id = '12345678' and gmlv.LIST_NAME = 'members';

Put that script in a text editor and remove extra whitespace (probably optional), and add this to the beginning:

grouperSession = GrouperSession.startRootSession();

Look at it and remove lines that dont apply...  then run in GSH

[appadmin@lorenzo bin]$ ./gsh.sh remove.script

Here is a more complicated example.  I want all groups in a certain folder which do not have an ADMIN privilege assigned to my application service principal, to assign that privilege.  Here is the query for oracle:

select 'grantPriv("' || gg.name || '", "someid/server.school.edu", AccessPrivilege.ADMIN);' as script 
from grouper_groups gg where gg.name like school:apps:appName:spaces:%' 
and not exists
(select (1) from grouper_memberships_lw_v gmlv where gg.name = gmlv.group_name and list_name = 'admins' 
and gmlv.subject_id = 'someid/server.school.edu');
Here is an example of removing privileges from a user on groups in oracle

set linesize 1000;
set pagesize 1000;
select 'revokePriv("' || gmlv.group_name || '", "' || gmlv.subject_id || '", AccessPrivilege.' ||
case
when gmlv.LIST_NAME = 'admins' then 'ADMIN'
when gmlv.LIST_NAME = 'readers' then 'READ'
when gmlv.LIST_NAME = 'viewers' then 'VIEW'
when gmlv.LIST_NAME = 'updaters' then 'UPDATE'
when gmlv.LIST_NAME = 'optins' then 'OPTIN'
when gmlv.LIST_NAME = 'optouts' then 'OPTOUT'
else gmlv.LIST_NAME
end  || ');'
  as script
from grouper_memberships_lw_v gmlv where subject_id = '12345678' and GMLV.LIST_TYPE = 'access'

This oracle script will remove privileges on folders for a certain user:

set linesize 1000;
set pagesize 1000;
select 'revokePriv("' || gs.name || '", "' || gm.subject_id || '", NamingPrivilege.' ||
case
when gf.NAME = 'stemmers' then 'STEM'
when gf.NAME = 'creators' then 'CREATE'
else gf.NAME
end  || ');'
  as script
from grouper_memberships_all_v gmav, grouper_fields gf, grouper_stems gs, grouper_members gm
where GMAV.FIELD_ID = GF.ID and gm.subject_id = '12345678' and GF.type = 'naming'
and GMAV.OWNER_STEM_ID = GS.ID and GMAV.MEMBER_ID = GM.ID

sdf