The Grouper discovery client is a way to discover information about services using a failsafe cache. This is available in Grouper v2.1+ in grouperClient.jar. It simply downloads a file from a web server. The file can be a properties file that holds information about the URLs and other metadata about a service. There should be multiple discovery URLs, and it uses the GrouperClient failover client so that communication is always available if there are no shared components of the multiple discovery web servers. However, if there is only one server, and it has been downloaded once in the past, then it will be in a failsafe cache so that it will use that information instead of failing.
This client will be used for the GrouperClient WS and GrouperClient LDAP to make the service more highly available. It can used as a command line utility or Java library for other non-Grouper services.
Configuring
These settings in the grouper.client.properties configure the discovery client:
# path of a writable directory where files can be created or stored # for example, cache of discovery configuration, or failover state # dot is the current directory... note, this directory must exist # or it will be created (attempted) # if this is blank, none of these features will be used, and # no files will be saved grouperClient.cacheDirectory = . ######################################## ## Service discovery settings ######################################## # number of minutes to remember that a connection had errors grouperClient.minutesToKeepErrors = 2 # if you are using a discovery service, but a discovery properties # at a URL (preferably SSL with valid certificate) # you should have multiple discovery URLs hosted at independent locations # to add more, increment the integer grouperClient.urlOfDiscovery.0 = https://firstserver.school.edu/path/ grouperClient.urlOfDiscovery.1 = https://secondserver.school.edu/path/ # if your app has a slow startup time, and the initial connections are timing out # esp if you arent just using the command line client (e.g. if using it as a jar), # then add more time here grouperClient.secondsForClassesToLoad = 20 # this will save the failover state to a file so if the JVM is stopped, it # will be there when it starts again. # Set to 0 to store on every use (recommended if used command line) # or set to -1 to not store or read ever # grouperClient.cacheDirectory must be set grouperClient.saveFailoverStateEverySeconds = 60 # this will cache the discovery properties in memory or on disk # if you want to cache the discovery properties locally, put a directory here: # this is recommended especially if you are using the grouper client as a command # line application and the process is constantly restarting # note, this will be used for a failsafe cache if all discovery servers are unavailable # grouperClient.cacheDirectory must be set # set to 0 or -1 to not cache grouperClient.cacheDiscoveryPropertiesForSeconds = 120
Using as a Java library
Just include the grouperClient.jar and grouper.client.properties in the classpath, and call like this:
boolean throwExceptionIfNotFound = true; File configFile = edu.internet2.middleware.grouperClient.discovery.DiscoveryClient.retrieveFile("myConfigFile.properties", throwExceptionIfNotFound);
Note, if the URLs in the grouper.client.properties are:
https://firstserver.school.edu/path/https://secondserver.school.edu/path/
then the path of the file downloaded from the web server would be
https://firstserver.school.edu/path/myConfigFile.properties
-or-
https://secondserver.school.edu/path/myConfigFile.properties
depending on which server is available and responds successfully. See the FailoverClient documentation for more information about how this works
Using as a command line utility
C:\mchyzer\grouper\trunk\grouperClient\dist>java -cp grouperClient.jar edu.internet2.middleware.grouperClient.discovery.DiscoveryClient --operation=getFile --fileName=myConfigFile.properties C:\mchyzer\grouper\trunk\grouperClient\dist\myConfigFile_20120208_141052_088_Q8O0R25L.properties C:\mchyzer\grouper\trunk\grouperClient\dist>
See the Java library section above for more information, it just calls that method and will fail if not there.
How it works
Not that the file returned does not have the same name as the file requested, this is so that multiple processes can access the discovery client at the same time and never have contention.
- First see if the file is in the memory expirable cache. If so, use it
- Next, if file not found yet, check the file system cache directory, for all files that match this name. Note, there can be directories on the name, though all cache files are in the same parent directory. If there is a file in the directory, then get the newest one, and if it is new enough to be expired by the setting grouperClient.cacheDiscoveryPropertiesForSeconds, then cache it and use it
- Next, if file not found yet, try the URLs in the config file, one then the other. If there are no URLs throw an exception. Save the file to the filesystem with a temporary name, e.g. myConfigFile_20120208_141052_088_Q8O0R25L.properties.discoverytmp, then when done downloading, rename to myConfigFile_20120208_141052_088_Q8O0R25L.properties. This file has the date, time, millis, random unique string in it. Add the file to cache and use it.
- Next, if file not found yet, use the latest file on the file system, add to cache, and use it (failsafe part)
- If nothing found throw exception
Note: you cannot rename or edit the file returned from the DiscoveryClient since it will be reused by subsequent calls until it is expired.
Example
Get a couple of web servers, and add a text file. For example:
http://www.internet2.edu/grouper/temp/temp.txt
https://grouperdemo.internet2.edu/temp/temp.txt
Then set the base URLS in the grouper.client.properties:
grouperClient.urlOfDiscovery.0 = https://grouperdemo.internet2.edu/temp/ grouperClient.urlOfDiscovery.1 = http://www.internet2.edu/grouper/temp/
Then request the file, e.g. from command line:
C:\mchyzer\grouper\trunk\grouperClient\dist>java -cp grouperClient.jar edu.internet2.middleware.grouperClient.discovery.DiscoveryClient --operation=getFile --fileName=temp.txt C:\mchyzer\grouper\trunk\grouperClient\dist\temp_20120208_145738_982_Q8O2F8SB.txt C:\mchyzer\grouper\trunk\grouperClient\dist>
Open that file on the local machine and make sure the contents are correct:
C:\mchyzer\grouper\trunk\grouperClient\dist>more C:\mchyzer\grouper\trunk\grouperClient\dist\temp_20120208_145738_982_Q8O2F8SB.txt Hey, this is my file, internet2.edu
That came from internet2. Change the grouper.client.properties to make the internet2 URL wrong, and rename the result to be too old, and try again. Note, there is a stderr stack, but it is successful and the stdout still has the file contents ok
C:\mchyzer\grouper\trunk\grouperClient\dist>ren temp_20120208_145738_982_Q8O2F8SB.txt temp_20120208_145238_982_Q8O2F8SB.txt grouperClient.urlOfDiscovery.0 = https://grouperdemo.internet2.edu/temp/ grouperClient.urlOfDiscovery.1 = http://www.internet2.edu/grouper/temp2/ C:\mchyzer\grouper\trunk\grouperClient\dist>java -cp grouperClient.jar edu.internet2.middleware.grouperClient.discovery.DiscoveryClient --operation=getFile --fileName=temp.txt Feb 8, 2012 3:03:23 PM edu.internet2.middleware.grouperClient.failover.FailoverClient internal_failoverLogic WARNING: NON-FATAL error in failover connection: http://www.internet2.edu/grouper/temp2/, will try others in pool (1 tot al errors, 0 total timeouts, 0 total fatal errors): [THIS STACK]: java.lang.RuntimeException: this stack at edu.internet2.middleware.grouperClient.failover.FailoverClient.internal_failoverLogic(FailoverClient.java:721) at edu.internet2.middleware.grouperClient.failover.FailoverClient.failoverLogic(FailoverClient.java:521) at edu.internet2.middleware.grouperClient.failover.FailoverClient.failoverLogic(FailoverClient.java:503) at edu.internet2.middleware.grouperClient.discovery.DiscoveryClient.retrieveFileFromDiscoveryServer(DiscoveryClient.java:614) at edu.internet2.middleware.grouperClient.discovery.DiscoveryClient.retrieveFile(DiscoveryClient.java:308) at edu.internet2.middleware.grouperClient.discovery.DiscoveryClient.main(DiscoveryClient.java:130) java.lang.RuntimeException: Problem with url: http://www.internet2.edu/grouper/temp2/temp.txt, and local file: C:\mchyzer\grouper\trunk\grouperClient\dist\temp_20120208_150615_695_Q8O2RBNU.txt.discoverytmp at edu.internet2.middleware.grouperClient.discovery.DiscoveryClient$1.logic(DiscoveryClient.java:755) at edu.internet2.middleware.grouperClient.discovery.DiscoveryClient$1.logic(DiscoveryClient.java:614) at edu.internet2.middleware.grouperClient.failover.FailoverClient$1.call(FailoverClient.java:631) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.lang.RuntimeException: Expected 200, but received response code: 404 at edu.internet2.middleware.grouperClient.discovery.DiscoveryClient$1.logic(DiscoveryClient.java:723) ... 7 more C:\mchyzer\grouper\trunk\grouperClient\dist\temp_20120208_150323_221_Q8O2NMKU.txt C:\mchyzer\grouper\trunk\grouperClient\dist>more temp_20120208_150323_221_Q8O2NMKU.txt This is my file, from grouper build server
If you rename one to be 20 minutes old the old files will not be automatically deleted yet, however, rename two to be more than 20 minutes old, and it will automatically delete the ones more than 20 minutes old except the newest one over 20 minutes old.
C:\mchyzer\grouper\trunk\grouperClient\dist>dir temp* Volume in drive C is OS Volume Serial Number is 4651-8CE1 Directory of C:\mchyzer\grouper\trunk\grouperClient\dist 02/08/2012 02:10 PM 35 temp_20120208_141052_088_Q8O0R25L.txt 02/08/2012 02:57 PM 35 temp_20120208_145238_982_Q8O2F8SB.txt 02/08/2012 03:03 PM 42 temp_20120208_150323_221_Q8O2NMKU.txt 3 File(s) 112 bytes 0 Dir(s) 270,957,670,400 bytes free C:\mchyzer\grouper\trunk\grouperClient\dist>ren temp_20120208_145238_982_Q8O2F8SB.txt temp_20120208_143238_982_Q8O2F8SB. txt C:\mchyzer\grouper\trunk\grouperClient\dist>dir temp* Volume in drive C is OS Volume Serial Number is 4651-8CE1 Directory of C:\mchyzer\grouper\trunk\grouperClient\dist 02/08/2012 02:10 PM 35 temp_20120208_141052_088_Q8O0R25L.txt 02/08/2012 02:57 PM 35 temp_20120208_143238_982_Q8O2F8SB.txt 02/08/2012 03:03 PM 42 temp_20120208_150323_221_Q8O2NMKU.txt 3 File(s) 112 bytes 0 Dir(s) 270,956,908,544 bytes free C:\mchyzer\grouper\trunk\grouperClient\dist>java -cp grouperClient.jar edu.internet2.middleware.grouperClient.discovery.DiscoveryClient --operation=getFile --fileName=temp.txt C:\mchyzer\grouper\trunk\grouperClient\dist\temp_20120208_150615_884_Q8O2RBNV.txt C:\mchyzer\grouper\trunk\grouperClient\dist>dir temp* Volume in drive C is OS Volume Serial Number is 4651-8CE1 Directory of C:\mchyzer\grouper\trunk\grouperClient\dist 02/08/2012 02:57 PM 35 temp_20120208_143238_982_Q8O2F8SB.txt 02/08/2012 03:03 PM 42 temp_20120208_150323_221_Q8O2NMKU.txt 02/08/2012 03:06 PM 42 temp_20120208_150615_884_Q8O2RBNV.txt 3 File(s) 119 bytes 0 Dir(s) 270,956,711,936 bytes free https://spaces.at.internet2.edu/display/Grouper/Grouper+Web+ServicesC:\mchyzer\grouper\trunk\grouperClient\dist>
Note this file was deleted: temp_20120208_141052_088_Q8O0R25L.txt, since it was 20 minutes old, and the one from 14:32.
To do
- Add basic authn for discovery
- Add in an end date for discovery information so things can be transitioned in the future?