Page History

...

Enabled for UI, WS and Daemon
Data will be kept in the folder etc:attribute:instrumentationData
Collect counts of servlet requests, group adds/deletes, membership adds/deletes, folders adds/deletes
UI and WS can start a new thread when the servlet first initializes, Daemon can start a new thread when it starts.
The new thread (a single thread executor) will enable stat collection (i.e. set some static variable)
Grouper api and ui code will update various static lists of timestamps indicating when each operation is done
A config option will determine how often the thread will go through the timestamps in memory and update the grouper database. Lower means fewer gaps when the process is killed.
Another config option will specify the increments to keep counts of. E.g. if we're keeping counts by 10 minutes or hour or day.
When the UI thread starts up, check to see if an "<ENGINE_NAME>_instrumentation.dat" file exists in the logs directory. This file will contain the uuid of this instance.
If it doesn't exist, create it and create a corresponding attribute in grouper, e.g. etc:attribute:instrumentationData:instrumentationDataInstances:theuuid (def = etc:attribute:instrumentationData:instrumentationDataInstancesDef)
The <ENGINE_NAME>_instrumentation.dat file should have a trivial update whenever the thread flushes to the database just in case the system is cleaning old files.
There will be a group used for assignments - etc:attribute:instrumentationData:instrumentationDataInstancesGroup.
There will be a single assign multi valued attribute - etc:attribute:instrumentationData:instrumentationDataInstanceCounts (def = etc:attribute:instrumentationData:instrumentationDataInstanceCountsDef)
There will also be other attributes (def = etc:attribute:instrumentationData:instrumentationDataInstanceDetailsDef) - etc:attribute:instrumentationData:instrumentationDataInstanceLastUpdate, etc:attribute:instrumentationData:instrumentationDataInstanceEngineName, etc:attribute:instrumentationData:instrumentationDataInstanceServerLabel
So etc:attribute:instrumentationData:instrumentationDataInstances:theuuid will be assigned to etc:attribute:instrumentationData:instrumentationDataInstancesGroup. And on that assignment will live assignments with actual data (instrumentationDataInstanceCounts, instrumentationDataInstanceLastUpdate, instrumentationDataInstanceEngineName, instrumentationDataInstanceServerLabel)

The value of the assignment on the assignment (instrumentationDataCounts) will be like:

Code Block
{"startTime" : 1486753200000, "duration" : 600000, "UI_REQUESTS" : 30, "API_GROUP_ADD" : 5, "API_GROUP_DELETE" : 3}

There may be multiple values added each time it runs. For example, if the database is updated every hour and the increment is every 10 minutes, then it could add 6 of these.

Code Block

{"startTime" : 1486753200000, "duration" : 600000, "UI_REQUESTS" : 30, "API_GROUP_ADD" : 5, "API_GROUP_DELETE" : 3}
{"startTime" : 1486753800000, "duration" : 600000, "UI_REQUESTS" : 300, "API_GROUP_ADD" : 2, "API_GROUP_DELETE" : 6}
{"startTime" : 1486754400000, "duration" : 600000, "UI_REQUESTS" : 3000, "API_GROUP_ADD" : 1, "API_GROUP_DELETE" : 2}etc

The TIER instrumentation daemon will sends these to TIER.

Code Block

"instances" : [ { "uuid" : "uuid1", 
                  "engineName" : "grouperUI", 
                  "serverLabel" : "ui-01"
                  "lastUpdate" : 1488825739828, 
                  "newCounts" : [{"startTime" : 1486753200000, "duration" : 600000, "UI_REQUESTS" : 30, "API_GROUP_ADD" : 5, "API_GROUP_DELETE" : 3}, 
                                 {"startTime" : 1486753800000, "duration" : 600000, "UI_REQUESTS" : 300, "API_GROUP_ADD" : 2, "API_GROUP_DELETE" : 6}, 
                                 {"startTime" : 1486754400000, "duration" : 600000, "UI_REQUESTS" : 3000, "API_GROUP_ADD" : 1, "API_GROUP_DELETE" : 2}] 
                }, 
                { "uuid" : "uuid2", 
                  "serverLabel" : "ui-02"
                  "engineName" : "grouperUI", 
                  "lastUpdate" : 1488825739829
                },
                { "uuid" : "uuid3",
                  "serverLabel" : "ws-01"
                  "engineName" : "grouperWS",
                  "lastUpdate" : 1488825739829
                },
                { "uuid" : "uuid4",
                  "serverLabel" : "ws-02"
                  "engineName" : "grouperWS",
                  "lastUpdate" : 1488825739829
                },
                { "uuid" : "uuid5",
                  "serverLabel" : "daemon-01"
                  "engineName" : "grouperLoader",
                  "lastUpdate" : 1488825739829
                }
              ]

An attribute will be created for each collector (e.g. etc:attribute:instrumentationData:instrumentationDataCollectors:OTHER_JOB_tierInstrumentationDaemon). This will be assigned to another group (etc:attribute:instrumentationData:instrumentationDataCollectorsGroup). And that assignment will have the time the collector was last updated (etc:attribute:instrumentationData:instrumentationDataCollectorLastUpdate).
The values won't be audited (user audit or point in time audit)
The cleanLogs daemon will delete counts older than 30 days (configurable).
Adding more counts
- There is a built in enum of types - InstrumentationDataBuiltinTypes
- You can also dynamically configure other types when starting the instrumentation thread.
- InstrumentationThread.startThread(GrouperContext.retrieveDefaultContext().getGrouperEngine(), customTypes);
- Maybe there needs to be a way to add more types after startup so that the PSP (or something else) can add to it?

...

Page tree

Versions Compared

Old Version 34

New Version 35

Key