After several days of loading and reloading our QA ldap server from our grouper database, we have a few statistics on LDAPpc's performance and behavior. These may be useful for the work starting up to improve Grouper and LDAPpc performance issues. 

Environment

• MACE Grouper 1.2.0 rc 4
• Subject API 0.3
• LDAPpc 1.0 
• Oracle 10g
  • person source for LDAPpc
  • group source for MACE Grouper
• SunOne LDAP server
• Grouper app server and DB hardware runs on Red Hat Enterprise Server 3 on quad processor 64-bit SunFire X4100 intel boxes with 4GB RAM and a nice quick SAN disk. The app server runs apache 2.2.4, tomcat 5.x and Sun jdk 5.x. The db server runs Oracle 10g. Our SunOne LDAP server runs under Solaris on a substantially less beefy box--a SunFire 280R dual UltraSparcIII with 2GB RAM, without a SAN connection. All servers are on the same subnet of a Gigabit Ethernetwork.
• 10,487 groups in MACE Grouper's EAB and Courses stems:
  • 704 EAB groups
  • 9,783 course groups
• Other details of our environment and statistics are linked at the bottom of this page

Configuration

We provision MACE Grouper from our existing groups registry, and then provision our LDAP group and person objects using LDAPpc. Due to performance considerations, we have created a local SQL person registry in the MACE grouper database to contain the essential person information needed to provision groups into LDAP. We provision the isMemberOf attribute on the Person objects in LDAP as well as the hasMember attribute on the Group objects in LDAP. For testing purposes, we are provisioning just EAB and Courses.

Quality Results 

For the most part, LDAPpc provisioned groups as we expected. There were a couple quirks that we have yet to resolve:

• Person objects' isMemberOf attribute contains only the name of the group, not the dn of the group that we anticipated. We _think_ this will work for us, but we want to make sure this is as designed.
  example:
  isMemberOf=COURSE:PHYS:0200:2007-Fall:S01:Student
  instead of
  isMemberOf=brownGroupRdn=COURSE:PHYS:0200:2007-Fall:S01:Student
• Groups that are children of other groups get the parent group dn in an isMemberOf attribute. Although not anticipated, we like this for the story it tells.
  example:
  brownGroupRdn=COURSE:PHYS:0200:2007-Fall:S01:Student
  isMemberOf=COURSE:PHYS:0200:2007-Fall:S01:Learner
  isMemberOf=COURSE:PHYS:0200:2007-Fall:S01:All
• Groups that are parents of other groups get the child group dn in the hasMember attribute, along side the dns of the person objects that belong to the group either directly or indirectly. We had planned for the roll-up of the direct and indirect person members, but the group members are not needed, and we're not sure if they are desired. I think that some applications may not be able to correctly understand the difference between persons (brownUuid) and groups (brownGroupRdn) listed in the hasMember attribute. From our perspective, we are flattening all groups, so we'd rather have just the person members listed in hasMember.
  example:
  brownGroupRdn=COURSE:PHYS:0200:2007-Fall:S01:All
  hasMember=brownUuid=foo
  hasMember=brownUuid=bar
  hasMember=brownGroupRdn=COURSE:PHYS:0200:2007-Fall:S01:Learner
  hasMember=brownGroupRdn=COURSE:PHYS:0200:2007-Fall:S01:Student

Performance Results 

We ran numerous trials with various configurations. The results were not always repeatable, perhaps in part because our QA LDAP server was not isolated for these tests, so we may have witnessed interference from other tasks hitting the LDAP server. But there were some general patterns:

• LDAPpc's initial run provisioning our 10,487 groups takes approximately 5 hours.
• The main constraint is I/O on the ldap server.
• Reducing or even eliminating logging on the ldap server does not help the performance bottleneck in a substantial way; runtime decreased from 5 hours to 4:50, but the I/O bottleneck actually became worse, probably because SunOne wasn't busy logging, so it hit the disk even harder for LDAP reads & writes. We're moving the SunOne LDAP DB and logs from the local file system to a high performance SAN, and we'll report on how that impacts performance.
• Numerous errors show up in the LDAPpc logs during the run, with strangely variable results that we have not entirely resolved. Some runs get more errors than others, and some incremental runs (after the initial run) show fewer of some types of errors. But not always, which is unexpected. The results tend to show these counts:
  • 10,487 Group Synchronizer entries: For tracking purposes, we indicate each group synced by LDAPpc.
    example:
    [edu.internet2.middleware.ldappc.synchronize.GroupSynchronizer] GroupSynchronizer -- synching COURSE:GEOL:0310:2007-Fall:S01:All
  • 878 Group Synchronizer Errors: not exactly sure what this indicates, but it appears in the early part of the run, when LDAPpc is synchronizing the group objects.
    example:
    [edu.internet2.middleware.ldappc.synchronize.GroupEntrySynchronizer] MEMBER[[ UUID = 9b604e75-daef-4ca0-bd4a-f3d163f633e7 ][ SUBJECT ID = 000426199 ][ SUBJECT SOURCE ID = brownjndi ]] Subject not found :: No results: searchSubject searchValue: 000426199
  • 2209 EntryNotFoundExceptions: Not clear yet why LDAPpc isn't finding these entries; the members appear in LDAP. The exeption comes up at the end of the LDAPpc run, when LDAPpc is populating the person objects' isMemberOf attribute.
    example:
    [edu.internet2.middleware.ldappc.GrouperProvisioner] edu.internet2.middleware.ldappc.EntryNotFoundException :: Subject not found using [ subject id = SIS:CHIN:0700:2007 Fall:S01:Student ][ source = g:gsa ][ filter = [base=ou=groups,dc=brown,dc=edu][scope=2][filter=(brownGroupRDN=0)] ] :: MEMBER[ UUID = 924d6f80-d093-493c-a83f-aa5ef9a13f5a ][ SUBJECT ID = fe771039-839e-4fa0-9e3e-d278c4666e84 ][ SUBJECT SOURCE ID = g:gsa ][ SUBJECT [ NAME = SIS:CHIN:0700:2007 Fall:S01:Student ][ ID = fe771039-839e-4fa0-9e3e-d278c4666e84 ] ]

      Questions or comments?  Contact us.