Bad membership finder will be re-written in v2.5.X to address performance issues. In some databases it takes hours and hours to complete. In some database crashes (mysql). And the fixes for composites take too long.
Bad membership finder change log consumer
- Look at change log entries, run every 10 minutes
- If a composite was changed more than 10 (configurable) minutes ago
- If not, dont indicate done with that part of change log, send it back (this wont cause an error right? maybe we need a setting in change log to only send messages X minutes old so it doesnt think there is a problem)
- And if it has not be processed before the event occurred (keep track in marker and name/value pair attribute on composite)
- Process it
- If a message from Grouper built in messages comes in to process a composite, process if more than 10 minutes old
- And if it has not be processed after the event occurred (keep track in marker and name/value pair attribute on composite)
- Process it
Processing
- See if we can process multiple groups at once, if so, batch it up, if not, one at a time
- Perhaps later consider how big the group is (once we cache an approx group size)
- Compare what members should be in group with the composite, and fix it if necessary
- If there is a fix it should cause message to go back and process group again to see if fixed
Bad membership finder daemon
- Asynchronous daemon intended to run nightly
- Can run at same time as incremental
- Process all groups (in batches? individually?)
- If there is a fix needed, send a message to the incremental to process it
- No need to run full multiple times
- No need to split out this part of daemon from other parts of bad membership daemon