Versions Compared

Old Version 23

changes.mady.by.user Shilen Patel (duke.edu)

Saved on

compared with

New Version 24

changes.mady.by.user Shilen Patel (duke.edu)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  • Data will be kept in the folder etc:attribute:instrumentationData
  • Collect counts of servlet requests, group adds/deletes, membership adds/deletes, folders adds/deletes
  • UI can start a new thread when the servlet first initializes
  • The new thread (a single thread executor) will enable stat collection (i.e. set some static variable)
  • Grouper api and ui code will update various static lists of timestamps indicating when each operation is done
  • A config option will determine how often the thread will go through the timestamps in memory and update the grouper database.  Lower means fewer gaps when the process is killed.
  • Another config option will specify the increments to keep counts of.  E.g. if we're keeping counts by 10 minutes or hour or day.
  • When the UI thread starts up, check to see if an "<ENGINE_NAME>_instrumentation.dat" file exists in the logs directory.  This file will contain the uuid of this instance.
  • If it doesn't exist, create it and create a corresponding attribute in grouper, e.g. etc:attribute:instrumentationData:instances:theuuid (def = etc:attribute:instrumentationData:instancesDef)
  • The <ENGINE_NAME>_instrumentation.dat file should have a trivial update whenever the thread flushes to the database just in case the system is cleaning old files.
  • There will be a group used for assignments - etc:attribute:instrumentationData:instrumentationDataGroup.
  • There will be a single assign multi valued attribute - etc:attribute:instrumentationData:instrumentationDataCounts (def = etc:attribute:instrumentationData:instrumentationDataCountsDef)
  • There will also be other attributes (def = etc:attribute:instrumentationData:instrumentationDataDetailsDef) - etc:attribute:instrumentationData:instrumentationDataLastUpdate, etc:attribute:instrumentationData:instrumentationDataEngineName, etc:attribute:instrumentationData:instrumentationDataServerLabel
  • So etc:attribute:instrumentationData:instances:theuuid will be assigned to etc:attribute:instrumentationData:instrumentationDataGroup.  And on that assignment will live assignments with actual data (instrumentationDataCounts, instrumentationDataLastUpdate, instrumentationDataEngineName, instrumentationDataServerLabel)

  • The value of the assignment on the assignment (instrumentationDataCounts) will be like:  

    Code Block
    {"startTime" : 1486753200000, "UI_REQUESTS" : 30, "API_GROUP_ADD" : 5, "API_GROUP_DELETE" : 3}

  • There may be multiple values added each time it runs.  For example, if the database is updated every hour and the increment is every 10 minutes, then it could add 6 of these.

    Code Block
    {"startTime" : 1486753200000, "UI_REQUESTS" : 30, "API_GROUP_ADD" : 5, "API_GROUP_DELETE" : 3}
{"startTime" : 1486753800000, "UI_REQUESTS" : 300, "API_GROUP_ADD" : 2, "API_GROUP_DELETE" : 6}
{"startTime" : 1486754400000, "UI_REQUESTS" : 3000, "API_GROUP_ADD" : 1, "API_GROUP_DELETE" : 2}etc

  • The TIER instrumentation daemon will sends these to TIER and delete the values from Grouper.

    Code Block
    "instances" : [ { "uuid" : "uuid1", 
                  "engineName" : "grouperUI", 
                  "serverLabel" : "ui-01"
                  "lastUpdate" : 1488825739828, 
                  "newCounts" : [{"startTime" : 1486753200000, "UI_REQUESTS" : 30, "API_GROUP_ADD" : 5, "API_GROUP_DELETE" : 3}, 
                                 {"startTime" : 1486753800000, "UI_REQUESTS" : 300, "API_GROUP_ADD" : 2, "API_GROUP_DELETE" : 6}, 
                                 {"startTime" : 1486754400000, "UI_REQUESTS" : 3000, "API_GROUP_ADD" : 1, "API_GROUP_DELETE" : 2}] 
                }, 
                { "uuid" : "uuid2", 
                  "serverLabel" : "ui-02"
                  "engineName" : "grouperUI", 
                  "lastUpdate" : 1488825739829,
                  "newCounts" : [] 
                },
                { "uuid" : "uuid3",
                  "serverLabel" : "ws-01"
                  "engineName" : "grouperWS",
                  "lastUpdate" : 1488825739829,
                  "newCounts" : []
                },
                { "uuid" : "uuid4",
                  "serverLabel" : "ws-02"
                  "engineName" : "grouperWS",
                  "lastUpdate" : 1488825739829,
                  "newCounts" : []
                },
                { "uuid" : "uuid5",
                  "serverLabel" : "daemon-01"
                  "engineName" : "grouperLoader",
                  "lastUpdate" : 1488825739829,
                  "newCounts" : []
                }
              ]

  • The values for instrumentationDataCounts, instrumentationDataEngineName, and instrumentationDataLastUpdate woninstrumentationDataLastUpdate, and instrumentationDataServerLabel won't be audited (user audit or point in time audit)

  • Code should be reusable for WS, loader, etc.

...