At the November TAC F2F, we discussed having a matrix of best practices by which to evaluate registered sites to help set expectations and create peer pressure. This is a preliminary set of suggested criteria.

The Best Practice Matrix might also be thought of as the "Community Wall of (Fame | Shame)." Emphasizing "Fame" or "Shame" would be a conscious decision on our part. It seems both approaches would be appropriate and effective in practice. For example, the IDs of entities requesting the unsupported WAYF might be published on the "Wall of Shame" while a list of those entities supporting SAML V2.0 would be published on the "Wall of Fame."

Policy / Non-Technical

• POP Available
• Security Incident Contact Registered
  • Does this also imply adherence to the recommended incident response process?
• Process for facilitating attribute release to SPs
• Release of basic attributes to SPs in some TBD automated fashion (with or without consent)

Deployment Practices

• Accessing the (unsupported) InCommon WAYF
• Deploying an Unsupported Version of Shibboleth / OS / etc.
• Expired Certificates in Metadata
• SAML 2.0 Support
  • IdPs with TLS-protected HTTP-Redirect SSO
  • SPs that support SAML 2.0 should indicate so in metadata
  • SPs with TLS-protected HTTP-POST ACS and an encryption key
• SAML 1.1 Support
  • SPs with TLS-protected HTTP-POST ACS
• Support for SAML 2.0 persistent NameIDs or eduPersonTargetedID
  • Perhaps support for other attributes are worth noting?
• Full saml2int conformance
• Consent-based support for particular attributes (i.e., no admin involvement needed)
• Keys of less than a certain age
  • We should consider what, if any, age is actually "too old"
• Appropriate error pages
  • Perhaps subjective, but I'd start with having actual contact info for users and a reasonable indication of what to do, maybe not using the Shibboleth logo?

Implementation Support

• InCommon Implementation Profile conformance
  • Could call out Metadata IOP as a subset, but my guess is few products would support that without the rest
  • Could identify "exceptions to conformance" to highlight specific missing capabilities or could break profile into separate features in the matrix

Meeting Notes

Meeting Notes - April 21, 2011