Grouper Web Services

Introduction

Grouper web services (grouper-ws) is a J2EE web application which exposes common Grouper business logic through SOAP and REST.  See Web Services FAQ. and architectural diagram.

To deploy the services, download the warfile and configure the property files (e.g. subject sources, databases, logging, etc).  Configure authentication.

Note: there is a command line and java API web service client called Grouper Client

To implement a web service client:

  1. Understand the object model.  All grouper-ws services are operations based on simple data structures.  The structures support Strings, ints, arrays, and structure references.
    1. Core web service API
    2. Example structure (only "getters" and "setters" are applicable properties)
    3. Each operation has many samples (authmated captures, versioned, and up to date).  Here is an example
    4. Most options has a sensible default (e.g. MemberFilter defaults to All members)
    5. Lookup objects in various (consistent) ways.  e.g. to delete a group, you can pass the name or uuid of the group.
  2. Decide if you are using SOAP or REST (this is real REST, not Axis HTTP/XML)
    1. Both SOAP and REST support the same API
  3. Inside SOAP and REST, each operation has two levels of complexity, the normal one, and the Lite one.
    1. Normal operation: can usually be batched (support a list of inputs, e.g. add multiple groups at once), supports complex inputs (arrays or structures)
    2. Lite operation: supports only inputs of scalars (no structures, no arrays... only Strings, ints, etc).  In REST this also means that the request can be sent via query string only
  4. If SOAP:
    1. Implement based on WSDL.  Note, you cant get the 1.4 WSDL out of a 2.0 server, since it has backwards compatible changes from 1.5 and 1.6.  You could get this 1.4 WSDL from SVN though...

Grouper client version

v1.4, v1.5, v1.6, v2.0 (server version) Endpoint

WSDL from server

WSDL from SVN

1.4

https://server.address/grouperWs/services/GrouperService

https://server.address/grouperWs/services/GrouperService?wsdl

1.4 WSDL

1.5

https://server.address/grouperWs/services/GrouperService

https://server.address/grouperWs/services/GrouperService?wsdl

1.5 WSDL

1.6

https://server.address/grouperWs/services/GrouperService

https://server.address/grouperWs/services/GrouperService?wsdl

1.6 WSDL, 1.6 WSDL in v2.0

2.0

https://server.address/grouperWs/services/GrouperService_v2_0

https://server.address/grouperWs/services/GrouperService_v2_0?wsdl

2.0 WSDL

    1. Note, if your servlet is not grouperWs (e.g. grouper-ws) then adjust accordingly.
    2. There is a sample Java client with sample calls
  1. If REST:
    1. Decide what format you want to send and receive data.  grouper-ws supports XHTML, XML, and JSON, as well as query strings for input (in URL or message body)
    2. There are many samples
  2. Understand versioning

https://spaces.at.internet2.edu/confluence/download/attachments/3014660/webServicePresentation.pptPresentation about Grouper web services

.NET client development guide

PHP client development guide

Guidelines For Working With Grouper Web Services

  1. There is a bug we are tracking with Axis, where if you skip String params, it will mix up the params.  So, if you are passing a param to a web service, make sure you pass empty strings for all null params before the param
  2. Code clients with a mindset that the service might change in subtle ways.  e.g. a result code might be added (check for success flag element, not success result code), an element might be added in a result object, another input element might be added to end of list, etc.  Expect elements to be added in data.  Note if you send the same version in the request, you will never get a response with a different structure.  Grouper WS are backwards compatible.
  3. Make sure there is a property in the client of the URL and version for the service.  The version of the service might change the URL (up to service deployer)...
  4. Note that Grouper WS can be setup with multiple instances.  If you have database replication (even readonly), then you can setup Grouper WS application servers is multiple data centers, and you can have the Grouper Client can failover among the nodes if there are errors or timeouts.

Operations

Features

Quick start

Note the WS is included in the Grouper Installer.  Checkout the appropriate projects under grouper-ws, read the README.txt in the grouper-ws/grouper-ws cvs directory

Build Script

Note the WS is included in the Grouper Installer.  (it will build and configure it). The build script for grouper-ws is pretty basic.  Generally just do the default (dist).  There is also an "ant grouper" target to build a new grouper jar, and "ant quick" to do everything but generate the Axis files (takes 3 minutes).

C:\mchyzer\isc\dev\grouper\grouper-ws>ant
Buildfile: build.xml

dist:

clean:
   [delete] Deleting directory C:\mchyzer\isc\dev\grouper\grouper-ws\build
    [mkdir] Created dir: C:\mchyzer\isc\dev\grouper\grouper-ws\build\grouper-ws
    [mkdir] Created dir: C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist
    [mkdir] Created dir: C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\grouper-ws

compile:
    [javac] Compiling 10 source files to C:\mchyzer\isc\dev\grouper\grouper-ws\build\grouper-ws
    [javac] C:\mchyzer\isc\dev\grouper\grouper-ws\src\grouper-ws\edu\internet2\middleware\grouper\webservices\GrouperSer
viceServlet.java:33: warning: [deprecation] getEPRForService(java.lang.String,java.lang.String) in org.apache.axis2.tran
sport.TransportListener has been deprecated
    [javac] public class GrouperServiceServlet extends AxisServlet {
    [javac]        ^
    [javac] 1 warning

generate-aar:
    [mkdir] Created dir: C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\webservices\classes
   [delete] Deleting directory C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\webservices\classes
    [mkdir] Created dir: C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\webservices\classes
     [copy] Copying 13 files to C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\webservices\classes
      [jar] Building jar: C:\mchyzer\isc\dev\grouper\grouper-ws\webapp\WEB-INF\services\GrouperService.aar
      [jar] Building jar: C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\grouper-ws.jar
    [mkdir] Created dir: C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\grouper-ws\WEB-INF\classes
    [mkdir] Created dir: C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\grouper-ws\WEB-INF\lib
     [copy] Copying 12 files to C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\grouper-ws\WEB-INF\classes
     [copy] Copying 30 files to C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\grouper-ws\WEB-INF\lib
     [copy] Copying 11 files to C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\grouper-ws
      [jar] Building jar: C:\mchyzer\isc\dev\grouper\grouper-ws\build\dist\grouper-ws.war

BUILD SUCCESSFUL
Total time: 22 seconds
The system cannot find the batch label specified - end

C:\mchyzer\isc\dev\grouper\grouper-ws>

 Notice the generate-aar target.  This is what makes the axis archive, which is all the classes needed for axis to determine the wsdl, along with the services.xml config file.

Axis is ~40 jars, though most of them are pretty axis specific.  There is an ant target which will compress most of these into one jar (axisBundle.jar).  Here is the ant help:

C:\mchyzer\isc\dev\grouper\grouper-ws>ant help
Buildfile: build.xml

help:
     [echo] Please ensure you have read the documentation -
     [echo] and created a build.properties file based on the template provided
     [echo]
     [echo] The following targets are available - type the appropriate name:
     [echo]
     [echo] 1) default (dist)
     [echo]      Simply builds, without cleaning, to the webapp.folder
     [echo] 2) clean
     [echo]      Clean the webapp folder, and classfiles, and build
     [echo] 3) generate-aar
     [echo]      Make the axis archive, which is the classfiles and services.xml that axis needs.  You need to do this i
f you ever change anything that changes the wsdl.  You can do this automatically in dist by setting a property in the bu
ild.properties
     [echo] 4) generate-axis-bundle-jar
     [echo]      Take all the bundlable axis jars (in lib/axis-bundle), unjar, and jar back up into one jar
     [echo]

BUILD SUCCESSFUL
Total time: 0 seconds
The system cannot find the batch label specified - end

C:\mchyzer\isc\dev\grouper\grouper-ws>

Subject attributes

 1. The default attribute names (comma separated) sent back for each request are specified in grouper-ws.properties under the key:

ws.subject.result.attribute.names

2. If the caller sets T to retrieve subject detail, then the attributes will be appended to that list in grouper-ws.properties key:

ws.subject.result.detail.attribute.names

3. If the caller specifies subjectAttributeNames in the request (comma separated), then those will be appended to the list (independent of the detail attributes).

So there are central settings, and caller settings that you need to design for and specify...

Note if subjectId and subjectIdentifier are filled in with the same value, it will find by subject id or identifier.

Logging requests and responses

You can do this via the client or a proxy.  If you must do this via the server, there is an experimental way to do this in v2.1.1+.  You should not do this in prod, only in a testing environment.

Add a new servlet filter mapping in the web.xml

<filter-mapping>
    <filter-name>Grouper logging filter</filter-name>
    <url-pattern>/*</url-pattern>
</filter-mapping>

Set the filter logger to log at debug level

log4j.logger.edu.internet2.middleware.grouper.ws.j2ee.ServletFilterLogger = DEBUG

You will see log entries like this

2012-05-03 09:13:18,575: [http-8088-1] DEBUG ServletFilterLogger.logStuff(98) -  - IP: 127.0.0.1, url: /grouperWs/servicesRest/v2_1_001/groups/aStem%3AaGroup/members, queryString: null, method: PUT, content-type: text/x-json; charset=UTF-8
request params:
request body: {"WsRestAddMemberRequest":{"actAsSubjectLookup":{"subjectId":"GrouperSystem"},"replaceAllExisting":"F","subjectLookups":[{"subjectId":"10021368"},{"subjectId":"10039438"}]}}
respone headers: (note, not all headers captured, and not in this order)
X-Grouper-resultCode: SUCCESS
X-Grouper-success: T
X-Grouper-resultCode2: NONE
HTTP/1.1 201
Content-Type: text/x-json; charset=UTF-8
response: {"WsAddMemberResults":{"responseMetadata":{"millis":"237","serverVersion":"2.1.1"},"resultMetadata":{"resultCode":"SUCCESS","resultMessage":"Success for: clientVersion: 2.1.1, wsGroupLookup: WsGroupLookup[pitGroups=[],groupName=aStem:aGroup], subjectLookups: Array size: 2: [0]: WsSubjectLookup[subjectId=10021368]\n[1]: WsSubjectLookup[subjectId=10039438]\n\n, replaceAllExisting: false, actAsSubject: WsSubjectLookup[subjectId=GrouperSystem], fieldName: null, txType: NONE, includeGroupDetail: false, includeSubjectDetail: false, subjectAttributeNames: null\n, params: null\n, disabledDate: null, enabledDate: null","success":"T"},"results":[{"resultMetadata":{"resultCode":"SUCCESS_ALREADY_EXISTED","success":"T"},"wsSubject":{"id":"10021368","name":"10021368","resultCode":"SUCCESS","sourceId":"jdbc","success":"T"}},{"resultMetadata":{"resultCode":"SUCCESS_ALREADY_EXISTED","success":"T"},"wsSubject":{"id":"10039438","name":"10039438","resultCode":"SUCCESS","sourceId":"jdbc","success":"T"}}],"wsGroupAssigned":{"description":"a group description","displayExtension":"a group","displayName":"a stem:a group","extension":"aGroup","name":"aStem:aGroup","typeOfGroup":"group","uuid":"d9094e4a7c6e4f399d7e1489c875b9f0"}}}

At some point we can make it more granular which requests get logged and give an option to format the request/response (indent, etc)

Fields and permissions

 If you want to check to see if a subject as a group permission, or to get a list of people with a certain permissions on a group, use hasMember or getMembers, and pass the name of the field (note this list depends on your configuration):

select name from grouper_fields where type != 'naming';

admins
description
displayExtension
displayName
extension
members
name
optins
optouts
readers
requireActiveEmployee
requireAlsoInGroups
updaters
viewers

Children pages

To do's (post 1.6.0)

  1. add logging filter
  2. fix javadoc warnings
  3. look into axis2 1.5, see if error fixed, see if samples/wsdl changes
  4. add move subject service
  5. improve auto-toString methods in resultMessage
  6. look at acegi
  7. add ip source filtering to grouper

See Also

Always Available Web Services

Grouper Failover Client

Grouper Diagnostics