Per-Entity Metadata Pilot

Per-Entity Metadata: A Pilot Study

In June 2013, the InCommon Technical Advisory Committee convened the Metadata Distribution Working Group. The output of that Working Group included an initial set of Phase 1 Recommendations (the implementation of which is now complete) and another set of Phase 2 Recommendations (whose implementation is a work-in-progress).

One of the Working Group’s Phase 2 Recommendations is to conduct a pilot study of per-entity metadata:

RECOMMENDATION: Conduct a pilot study that explores the utility of [signed, per-entity metadata] as an alternative to metadata aggregates, and evaluate current implementations of this model to discover problems or identify new requirements.

The only SAML implementation known to support dynamic metadata exchange of per-entity metadata via the Metadata Query Protocol is the Shibboleth SP software. If you know of other implementations that perform dynamic metadata exchange, please post your experiences to the metadata-support mailing list.

It is well known that Shibboleth, simpleSAMLphp, and other Metadata Client Software can automatically refresh the entire InCommon metadata aggregate. We expect that these software implementations can also refresh per-entity metadata but one of the goals of this pilot study is to demonstrate this capability. If you know of other metadata client software or SAML implementations with the ability to automatically refresh per-entity metadata, please document this fact on the metadata-support mailing list.

Contents

Overview

This Per-Entity Metadata Pilot Study will last for six (6) months. Here is a rough timeline:

1. [Sep 2014] A beta instance of a Metadata Query Server will be deployed.

   • initially, there will be support for per-IdP metadata only

2. [Nov 2014] The beta instance of the Metadata Query Server will be upgraded.

   • if all goes well, support for per-SP metadata will be added

3. [Jan 2015] The future of the Metadata Query Server will be determined.

4. [Mar 2015] The beta instance of the Metadata Query Server will be decommissioned.

If the Pilot Study is successful, a production instance of a Metadata Query Server may be deployed. In any case, the beta instance of the Metadata Query Server is guaranteed to be decommissioned at the end of the Pilot Study period.

The Metadata Query Server draws from two disparate metadata sources: InCommon metadata and eduGAIN metadata. The source metadata is refreshed every few hours using the secure metadata refresh process outlined on the Metadata Consumption wiki page. When an entity descriptor is first requested, it is extracted from the source metadata, signed, and cached for future use. Each entity descriptor is signed with a private key generated specifically for this Pilot Study.

Metadata Query Server

To support this pilot study, a Metadata Query Server that implements the Metadata Query Protocol has been deployed. This Metadata Query Server is an instance of Ian Young's mdq-server reference implementation of the Metadata Query Protocol. If you find a bug in the server implementation, please record the issue in the project's issue tracker.

This deployment of the Metadata Query Server is monitored 24x7. If a problem is detected, we will do our best to restore normal server operation as soon as possible.

Protocol Overview

A typical request to the Metadata Query Server is a GET request at a URL constructed from the following base URL:

• http://mdq-beta.incommon.org/global

To construct a metadata location, append the URL-encoded entityID of the desired entity to the base URL as specified in the Metadata Query Protocol. For example, for entity https://webauth.cmc.edu/idp/shibboleth the full URL is:

• http://mdq-beta.incommon.org/global/entities/https%3A%2F%2Fwebauth.cmc.edu%2Fidp%2Fshibboleth

When you click the above link, the Metadata Query Server returns an HTML page suitable for browsing by a human. For machine-readable XML metadata, use some other (non-browser) HTTP client. For instance, you can use curl at the command line to retrieve per-entity metadata in machine-readable form:

# Fetch a signed entity descriptor
$ /usr/bin/curl --silent http://mdq-beta.incommon.org/global/entities/https%3A%2F%2Fwebauth.cmc.edu%2Fidp%2Fshibboleth

The output of the above command is a single entity descriptor, signed with a 2048-bit private key. See the next section for instructions how to obtain an authentic copy of the corresponding metadata signing certificate.

Bootstrapping Trust

To ensure the security of your metadata refresh process, you must verify the XML signature on each and every entity descriptor you consume. To do that, you need an authentic copy of the metadata signing certificate. The certificate must be obtained securely since all subsequent operations depend on it.

To obtain an authentic copy of the metadata signing certificate for the Metadata Query Server, perform the following steps:

1. Download a copy of the metadata signing certificate via a secure channel
2. Compute the SHA-1 and SHA-256 fingerprints of the metadata signing certificate so obtained
3. Compare the computed fingerprints to the actual fingerprints obtained via an independent secure channel

The latter two steps guarantee the integrity of the metadata signing certificate so obtained.

You may check the integrity of the downloaded certificate in a variety of ways. For example, on a GNU/Linux system, you could use curl and openssl to perform the first two steps of the bootstrap process:

# Step 1: Download a copy of the metadata signing certificate via a secure channel
$ MD_CERT_LOCATION=https://ds.incommon.org/certs/mdq-beta-cert.pem
$ MD_CERT_PATH=/path/to/mdq-beta-cert.pem
$ /usr/bin/curl --silent $MD_CERT_LOCATION > $MD_CERT_PATH

# Step 2: Compute the SHA-1 and SHA-256 fingerprints of the metadata signing certificate
$ /bin/cat $MD_CERT_PATH | /usr/bin/openssl x509 -sha1 -noout -fingerprint
SHA1 Fingerprint=F0:1D:23:CC:D8:4D:01:53:93:14:26:0A:C6:18:1A:70:BA:B0:00:E9
$ /bin/cat $MD_CERT_PATH | /usr/bin/openssl x509 -sha256 -noout -fingerprint
SHA256 Fingerprint=CA:2B:2D:DA:C5:0D:AB:CA:4C:94:43:A4:F7:EF:09:2C:B7:3B:84:07:2B:6F:05:F0:D1:36:A8:74:2D:6C:B5:32

Step 3: The final step is to compare the computed fingerprints to the actual fingerprints. The latter are displayed on this authoritative web page:

• https://ops.incommon.org/mdq_beta_cert.html

If the computed fingerprints match the actual fingerprints, you are done. You may now safely use the certificate to verify the signature on the metadata file.

Configuration

The Shibboleth SP is the only known SAML implementation that supports Dynamic Metadata Exchange. The following Dynamic MetadataProvider works with Shibboleth SP 2.4 (or later).

<!--
  The following MetadataProvider dynamically requests IdP metadata just-in-time.
  This is called Dynamic Metadata Exchange.
-->
<MetadataProvider type="Dynamic" ignoreTransport="true">

  <!-- Substitute an URL-encoded entityID into the request URL -->
  <Subst> http://mdq-beta.incommon.org/global/entities/$entityID </Subst>
  <!-- NOTE: Remove leading and trailing whitespace from the above URL -->

  <!--
    Require a validUntil XML attribute on the EntityDescriptor element
    and make sure its value is no more than 14 days into the future 
  -->
  <MetadataFilter type="RequireValidUntil" maxValidityInterval="1209600"/>
 
  <!-- Verify the signature on the metadata file -->
  <MetadataFilter type="Signature" certificate="mdq-beta-cert.pem"/>
 
</MetadataProvider>

If you find that some other configuration works better, please share your config on the metadata-support mailing list.