The new InCommon Metadata Distribution Service, based on the Metadata Query (MDQ) protocol, is now available as a Release Candidate. This new service supports realtime per-entity metadata query. It eliminates the need for metadata consumers to load the entire metadata aggregate and saves a ton of system resources and start up time. 

InCommon will transition to this new service in the coming months. To begin working with it, visit the new InCommon Metadata Service Wiki.

Operationally, structural changes to metadata are first introduced into the Preview Aggregate and subsequently synchronized with the Main Aggregate and the Fallback Aggregate, in that order. Time between synchronization events depends on the nature of the structural change.

Metadata consumers choose exactly one of the three aggregates in the pipeline depending on the immediate requirements of their deployment.

Multiple metadata aggregates allow InCommon to deploy changes to metadata more quickly, easily, and safely.

Preview Metadata Aggregate

The Preview Metadata Aggregate helps manage the introduction of potentially breaking changes into InCommon metadata. Before such a change is deployed to the Main Aggregate, it is first introduced in preview mode. Any issues that surface are addressed before the change is synced with the Main Aggregate.

The Preview Aggregate is designed for deployments on the leading edge, such as test and dev deployments. Such deployments are strongly encouraged to consume the Preview Aggregate instead of the Main Aggregate.

Main Metadata Aggregate

In the best possible world, a deployment would configure itself to refresh its metadata store from the Main Metadata Aggregate and that would be the end of it. The problem is that metadata aggregates are brittle by their very nature, that is, a small change to metadata may have unexpected effects downstream. If this happens, a deployment can “fall back” to a previous version of metadata that is known to be backward compatible.

Fallback Metadata Aggregate

The Fallback Metadata Aggregate comes into play when a breaking change is inadvertently introduced into InCommon metadata. When a change is made to the Main Aggregate, and that change breaks a downstream metadata process, an affected deployment can temporarily migrate to the Fallback Aggregate. This gives the deployment time to adjust to the breaking change.

A deployment should consume the Fallback Aggregate only when it has to, that is, when it is unable to consume the Main Aggregate. Consuming the Fallback Aggregate is a temporary measure while a deployment reacts to a breaking change introduced into InCommon metadata. See the article Using the fallback aggregate for more information.

The Fallback Aggregate is transient in the sense that backward compatibility is provided for a limited, predetermined period of time. This forces deployments to adjust to breaking changes to metadata albeit in a controlled environment.

IdP-only Aggregate

This aggregates contains only the IdP entities. It is designed for use by service providers (SP) to streamline operations. SPs have no need to parse the over 5,000 SP entites in the InCommon metadata aggregate. Loading them only wastes system resource and slows systems start up time. See IdP-only metadata aggregate for more information.

Comparing the differences between versions of metadata aggregates

Differences between the various metadata aggregates are recorded and archived daily:

1. Vertical diff between two consecutive instances of the Main Aggregate

2. Horizontal diff between the Main Aggregate and the Preview Aggregate

3. Horizontal diff between the Fallback Aggregate and the Main Aggregate

The vertical diff captures changes to metadata over time. The horizontal diffs record the flow of metadata through the preview-main-fallback pipeline.

To obtain a diff of consecutively published metadata aggregates, visit the InCommon metadata diff history web page. Alternatively, you may requested via email.