Page History

Requirements

• Docker or other OCI server OR Java 11 .x or higher (Tested with Amazon Corretto 11)
• %TODO% G memory
• RDBMS for persistent configuration storage:
  • Postgres PostgreSQL (version 14 is the presumed default.)
  • Tested with latest docker images of mariadbMariaDB, Mysql MySQL and SQL Server.  
  • SQLserver.  An example docker-compose and application.yml examples for each DB can be found in the testbed directory in the Git repository. 

Running

The Shibboleth IdP UI is built as a Java Spring Boot ( https://spring.io/projects/spring-boot ) application. It can be run as a standalone web application that has Tomcat embedded in it. The same WAR file can be deployed into an external servlet container (standalone Tomcat, Jetty etc..). It can also be deployed using a Docker image.

Deployment Considerations

Choosing which method to run the Shibboleth IdP UI is largely dependent on your own infrastructure and knowledge base.  The UI was designed with the mindset that it would run alongside the Shibboleth IdP and integrate with it via a shared volume.  This is only one possible method of integration and the UI does not force any conventions on how the data is shared with the IdP.  For an example of using a shared docker volume to integrate with the IdP look at the integration testbed in the Git repository. 

Configuration Files

Regardless of which method is used to run the Shibboleth IdP UI a configuration file(s) is required. Much of the behavior of the UI can be set and controlled through properties files which can be in one or both of the following formats:

• application.properties , a Spring property file similar to the ones used in the Shibboleth IdP – a simple text file with a property name, equals sign, and the property value, one per line.
• application.yml , a YAML format file

Choosing which format to use is largely a personal choice as functionally they are identical. One caveat is in the case that there is both a properties and yml file, the properties file will be read first. 

The ShibUI comes with a basic example of both, with the example application.properties file having the core settings for authentication, database connection information, users file, directory/location settings for where the UI should write out the metadata files and metadata-providers.xml file it manages, etc. The example application.yml file contains all the settings that impact the information, options, list elements, etc. that are actually shown in the UI. 

There is no technical reason that you need to follow the examples and keep the distinction; you could manage everything through a single application.properties or application.yml if you wanted. On the other hand, it can be a convenient distinction to keep the core "internal/baked-in settings" distinct from the "front-end/UI" settings. 

Authentication

There are three methods for authenticating users. The first two involve choosing an encryption scheme for the password from one of the following:

• noop -- clear text password follows
• bcrypt -- the following value has been encrypted with the $2a$ Bcrypt algorithm (limitation of the underlying Spring library currently incorporated is that only the $2a$ Bcrypt algorithm is supported.)

1. A single root user. This is the default and will generate a random password displayed in the console for the root user at runtime.  The name and password for this user can also be set in application.properties with

shibui.default-rootuser=root
shibui.default-password={ENCRYPT_SCHEME}somepassword

2. Users defined via a file with a format of:

username,{ENCRYPT_SCHEME}password,firstname,lastname,ROLE_VALUE,email

Where ROLE_VALUE is one of:

• ROLE_ADMIN -- No limits, can do anything the Shibboleth IdP UI supports. Currently, Metadata Provider configuration (and filter configuration) requires ROLE_ADMIN access
• ROLE_USER -- These users can only add individual metadata sources and modify metadata sources that they created.  When creating a new SP entry, that SP metadata will not be active until a user with ROLE_ADMIN approves it.
• ROLE_ENABLE – These are "enhanced" ROLE_USER users that have the ability to enable/publish SP metadata but cannot access/configure Metadata Provider (and filter) configurations

Note: you must have at least one user with ROLE_ADMIN.

Then instruct the UI where to find that file in application.properties

shibui.user-bootstrap-resource = file:/full/path/to/users/file.txt

3. User authenticated via SAML

The Shibboleth IdP UI includes a Java-based SAML SP based on the Pac4j ( https://www.pac4j.org ) library (SAML support built on top of the Shibboleth Consortium's OpenSAML software).  Configuration requires a few external steps and configuration of application.properties.

...

Configure the following settings in application.properties or application.yml

shibui:
  default-password: "{noop}password"
  user-bootstrap-resource: file:/opt/shibui/conf/users.csv
  roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_ENABLE,ROLE_PONY
  pac4j-enabled: true
  pac4j:
    keystorePath: "/opt/shibui/conf/samlKeystore.jks"
    keystorePassword: "password"
    privateKeyPassword: "password"
    serviceProviderEntityId: "https://shibui.local/shibui"
    serviceProviderMetadataPath: "/opt/shibui/conf/sp-metadata.xml"
    identityProviderMetadataPath: "/opt/shibboleth-idp/metadata/idp-metadata.xml"
    forceServiceProviderMetadataGeneration: false
    callbackUrl: "https://shibui.local/shibui/callback"
    maximumAuthenticationLifetime: 3600000
    simpleProfileMapping:
      username: urn:oid:0.9.2342.19200300.100.1.1
      firstName: urn:oid:2.5.4.42
      lastName: urn:oid:2.5.4.4
      email: urn:oid:0.9.2342.19200300.100.1.3
      groups: urn:oid:2.5.4.15  # businessCategory
      roles: urn:oid:1.3.6.1.4.1.5923.1.1.1.7  # eduPersonEntitlement

Deployment Examples

Embedded Tomcat

...

Create a users.csv with:

root,{bcrypt}$2a$10$V1jeTIc0b2u7Y3yU.LqkXOPRVTBFc7SW07QaJR4KrBAmWGgTcO9H.,first,last,ROLE_ADMIN,user1@example.org

Start the database

docker run --rm --name postgres-db -p 5432:5432 -e POSTGRES_PASSWORD=shibui -e POSTGRES_USER=shibui -d postgres

Create a basic Shibboleth IdP UI configuration with database settings

spring:
  profiles:
    include:
  datasource:
    platform: postgres
    driver-class-name: org.postgresql.Driver
    url: jdbc:postgresql://localhost:5432/shibui
    username: shibui
    password: shibui
  jpa:
    show-sql: false
    properties:
      hibernate:
        dialect: org.hibernate.dialect.PostgreSQL95Dialect
        format_sql: true
shibui:
  user-bootstrap-resource: file:users.csv
  roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_PONY

Run the war and tell it where to find your configuration files

java -Xmx1g -jar shibui-1.16.0.war --spring.config.additional-location=file:application.yml

...

Running under https is accomplished by adding the following to your configuration file

server:
  ssl:
    key-store: keystore.p12
    key-store-password: password
    key-store-type: pkcs12
    key-alias: tomcat
    key-password: password
  port: 8443

• A method for providing generated metadata to the associated IDP:
  • Shared filesystem
  • Git push/pull
  • file transfer
  • MDQ

Downloads

Pre-built docker images are on Docker Hub.

Releases for the Java war file can be found in the Internet2 GitHub project repository.

Quick Start Demo

Using Docker:

docker run \
    -p 8080:8080 \
    -p 9090:9090 \
  i2incommon/shib-idp-ui:latest

As a standalone app:

VERSION=1.16.0
wget https://github.internet2.edu/TIER/shib-idp-ui/releases/download/v$VERSION/shibui-$VERSION.war
java -jar -Dshibui.default-password={noop}abcd1234 shibui-$VERSION.war

Once up, the application can be accessed at http://localhost:8080/. The demo includes an in-memory H2 database, and a default administration user. For the docker demo, the user is "root" and password is "password". For the standalone app, there is no default password. The demo above sets it to "abcd1234" by setting application property shibui.default-password.

API documentation from Swagger is available on port 9090. This port is exposed in this demo, but is not for user access in production.

Deployment Considerations

The principal output from the Shib IDP UI application are XML files. Metadata provider records in the UI correspond to MetadataConfiguration resources, which contain pointers to the actual entity metadata to consume. Metadata source records in the UI define individual entities, and each entity corresponds to a single XML file defining its metadata. The entity files are generally to be used with a LocalDynamicMetadataProvider, and the entity file names follow its convention of lower case hex-encoded SHA-1 digest of the entityID, suffixed with ".xml" The locations in the filesystem to save the output are defined by UI application properties, and have no defaults. See {Configuration} for details on how to define these.

The IDP reads metadata provider and entity metadata from these files, so they must be accessible by the IDP. There are a number of ways these files can get from the UI to the IDP:

• UI running as a web application on the same server as the IDP, and saved to local filesystem
• Shared filesystem between the UI and the IDP
• Git push/pull jobs
• file transfer jobs

Running

The Shibboleth IdP UI is built as a Java Spring Boot (https://spring.io/projects/spring-boot) application. It can be run as a standalone Java application (using an embedded Tomcat server), or as a Java web application running under an installed Tomcat or Jetty web application server. Both the standalone and web application methods can be used either directly on a web server, or packaged inside a Docker image. Internet2 supplies pre-built Docker images for released versions. The deployment method is largely dependent on your own infrastructure and knowledge base.

Configuration

Regardless of which method is used to run the Shibboleth IdP UI, configuration is required. Much of the behavior of the UI can be set and controlled through property files which can be in one or both of the following formats:

• application.properties - a Spring property file similar to the ones used in the Shibboleth IdP – a simple text file with a property name, equals sign, and the property value, one per line.
• application.yml - a YAML format file

Choosing which format to use is largely a personal choice as functionally they are identical. One caveat is in the case that there is both a properties and yml file, the properties file will be read first.

The ShibUI comes with a basic example of both, with the example application.properties file having the core settings for authentication, database connection information, users file, directory/location settings for where the UI should write out the metadata files and metadata-providers.xml file it manages, etc. The example application.yml file contains all the settings that impact the information, options, list elements, etc. that are actually shown in the UI.

There is no technical reason that you need to follow the examples and keep the distinction; you could manage everything through a single application.properties or application.yml if you wanted. On the other hand, it can be a convenient distinction to keep the core "internal/baked-in settings" distinct from the "front-end/UI" settings.

The application properties are Spring configuration files, and follow its built-in methods for where it looks for these files:

• The classpath root
• The classpath /config package
• The current directory (note, in the Internet2 Docker image the "current directory" will be the working directory which is /opt/shibui)
• The /config subdirectory of the current directory
• Immediate child directories of the config/ subdirectory
• File name option passed to the executable jar --spring.config.location=file://{absolute-path-to-file}
• Directory name passed to the executable jar --spring.config.location=file://{absolute-path-to-directory}
• Java property -Dspring.config.location
• Environment variable SPRING_CONFIG_LOCATION
• Property spring.config.additional-location

Spring properties also utilize profile designations, which target the reading of an environment-specific configuration file. For example, when passing Java system property -Dspring.profiles.active=prod, setting environment variable SPRING_PROFILES_ACTIVE=prod, or setting property spring.profiles.active=prod in application.properties, application startup will additionally load file application-prod.properties, using the same search order as for application.properties.

Database properties

The Shib IDP UI can be integrated with a variety of back end databases to store registry information. The distribution includes JDBC drivers for PostgreSQL, MySQL, MariaDB, and SQL Server. With no custom database configuration, the application will default to an embedded in-memory H2 database, which will be cleared out on application exit. Database-specific properties to set are:

PostgreSQL:

spring:
  datasource:
    platform: postgres
    driver-class-name: org.postgresql.Driver
    url: jdbc:postgresql://db-hostname:5432/db-name
    username: shibui
    password: shibui
  jpa:
    properties:
      hibernate:
        dialect: org.hibernate.dialect.PostgreSQLDialect

MySQL:

spring:
  datasource:
    platform: mysql
    driver-class-name: com.mysql.cj.jdbc.Driver
    url: jdbc:mysql://db-hostname:3306/db-name
    username: shibui
    password: shibui
  jpa:
    properties:
      hibernate:
        dialect: org.hibernate.dialect.MySQL8Dialect

MariaDB:

spring:
  datasource:
    platform: mysql
    driver-class-name: org.mariadb.jdbc.Driver
    url: jdbc:mariadb://db-hostname:3306/db-name
    username: shibui
    password: shibui
  jpa:
    properties:
      hibernate:
        dialect: org.hibernate.dialect.MariaDB103Dialect

SQL Server:

spring:
  datasource:
    platform: sqlserver
    driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver
    url: jdbc:sqlserver://db-hostname:1433
    username: sa
    password: Password1
  jpa:
    properties:
      hibernate:
        dialect: org.hibernate.dialect.SQLServerDialect

Authentication

There are three methods for authenticating users. The first two involve choosing an encryption scheme for the password from one of the following:

• noop -- clear text password follows
• bcrypt -- the following value has been encrypted with the $2a$ Bcrypt algorithm (limitation of the underlying Spring library currently incorporated is that only the $2a$ Bcrypt algorithm is supported.)

1. A single root user. The default user is root, but can be set to another username via property shibui.default-rootuser. there is no default password in the war file, but it can be defined via property shibui.default-password. NOTE: The Internet2 docker image does include a users.txt which has a default root password of password.

2. Users defined in a bootstrap file, and imported at startup by defining property shibui.user-bootstrap-resource=file:/full/path/to/user-file. In the Internet2 image, a sample users.txt file, with user root with password password, is included in the app folder, and imported in the default startup command.

The file is a comma-separated list of users to bootstrap, with a format of:

username,{ENCRYPT_SCHEME}password,firstname,lastname,ROLE_VALUE,email

Where ROLE_VALUE is one of:

• ROLE_ADMIN -- No limits, can do anything the Shibboleth IdP UI supports. Currently, Metadata Provider configuration (and filter configuration) requires ROLE_ADMIN access

• ROLE_USER -- These users can only add individual metadata sources and modify metadata sources that they created.  When creating a new SP entry, that SP metadata will not be active until a user with ROLE_ADMIN approves it.

• ROLE_ENABLE – These are "enhanced" ROLE_USER users that have the ability to enable/publish SP metadata but cannot access/configure Metadata Provider (and filter) configurations

Note: you must have at least one user with ROLE_ADMIN.

import bcrypt

password = 'passw0rd123'

bytes = password.encode('utf-8')
salt = bcrypt.gensalt()

hash = bcrypt.hashpw(bytes, salt)
print(hash)

3. User authenticated via SAML

The Shibboleth IdP UI includes a Java-based SAML SP based on the Pac4j ( https://www.pac4j.org ) library (SAML support built on top of the Shibboleth Consortium's OpenSAML software).  Configuration requires a few external steps and configuration of application.properties.

1. Configure, in a users file, at least one user that will match up with a Shibboleth IdP supplied user identifier with ROLE_ADMIN (password is a required field in the file but is irrelevant/ignored in the user file)

2. Create a copy of the Shibboleth IdP's metadata and place within the file system where the UI can access

3. Create the Metadata for the SP and place within the file system where the UI can access

4. Configure the following settings in application.properties or application.yml

shibui:
  default-password: "{noop}password"
  user-bootstrap-resource: file:/opt/shibui/conf/users.csv
  roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_ENABLE,ROLE_PONY
  pac4j-enabled: true
  pac4j:
    keystorePath: "/opt/shibui/conf/samlKeystore.jks"
    keystorePassword: "password"
    privateKeyPassword: "password"
    serviceProviderEntityId: "https://shibui.local/shibui"
    serviceProviderMetadataPath: "/opt/shibui/conf/sp-metadata.xml"
    identityProviderMetadataPath: "/opt/shibboleth-idp/metadata/idp-metadata.xml"
    forceServiceProviderMetadataGeneration: false
    callbackUrl: "https://shibui.local/shibui/callback"
    maximumAuthenticationLifetime: 3600000
    simpleProfileMapping:
      username: urn:oid:0.9.2342.19200300.100.1.1
      firstName: urn:oid:2.5.4.42
      lastName: urn:oid:2.5.4.4
      email: urn:oid:0.9.2342.19200300.100.1.3
      groups: urn:oid:2.5.4.15 # businessCategory
      roles: urn:oid:1.3.6.1.4.1.5923.1.1.1.7 # eduPersonEntitlement

Entity Metadata Files

Entity metadata is not written to XML files by default. To periodically write the generated metadata files to a target directory, define:

shibui.metadata-dir=/path/to/metadata/output/directory
shibui.taskRunRate=milliseconds-or-duration

Property shibui.metadata-dir is an absolute path to the output directory. Entity filenames will be constructed by combining the directory and the base name derived from the SHA-1 digest of the entityID, adding the path separator if needed.

The shibui.taskRunRate property (default 30000 or every 30 seconds) sets the output to be generated at timed intervals. The property can accept a number which represents milliseconds, or a Java Duration such as PT20M.

$ echo -n "urn:test:foobar" | openssl sha1
d278c9975472a6b4827b1a8723192b4e99aa969c

Metadata Provider File

The metadata providers defined in the UI are not written to an external XML file by default. To periodically write the generated configuration to a filesystem resource, define:

shibui.metadataProviders.target=file:/path/to/provider-file.xml
shibui.metadataProviders.taskRunRate=milliseconds-or-duration

Property shibui.metadataProviders.target is the filename for the output xml. It expects a Java Resource, thus requires file: before the path.

Similar to the entity task run rate, the shibui.metadataProviders.taskRunRate property defaults to 30000 or every 30 seconds and can accept a number which represents milliseconds, or a Java Duration.

Metadata External Provider File

TODO

Other properties

The full set of properties used in the UI, with their defaults, are embedded in the war file application.yml and application.properties.

Spring properties can also be utilized to define server behavior. Logging levels, JDBC tuning, or HTTPS are common ones to use.

Enabling HTTPS Listener

Add these Spring properties to your configuration file:

server:
  ssl:
    key-store: keystore.p12
    key-store-password: password
    key-store-type: pkcs12
    key-alias: tomcat
    key-password: password
  port: 8443

Spring boot supports keystores in either the PKCS12 or JKS format. An example command to generate a new keystore:

keytool -genkeypair -alias tomcat -keyalg RSA -keysize 2048 -storetype PKCS12 -keystore keystore.p12 -validity 3650

Deployments Examples

Internet2 Docker image - base image with mounted configuration

In this example, properties include shibui.user-bootstrap-resource=file:/opt/shibui/users.csv, along with connections to an existing database.

docker run -d \
    -p 8080:8080 \
    -e SPRING_PROFILES_ACTIVE=prod \
    -v $PWD/conf/application.properties:/opt/shibui/application.properties \
    -v $PWD/conf/prod/application.properties:/opt/shibui/application-prod.properties \
    -v $PWD/conf/users.csv:/opt/shibui/users.csv \
    -v $PWD/output/providers:/var/log/providers \
    -v $PWD/output/metadata:/var/log/metadata \
  i2incommon/shib-idp-ui:latest

There are many ways to deploy an application with Docker and the following example is only meant to show a simplistic approach for demonstration purposes.  A docker compose example of a fully functioning Shibboleth IdP UI with full integration with the Shibboleth IdP can be found in the smoke-test testbed in the Git repository.

Standalone war file

This example assumes all the files are placed in the same directory that the war will be run from and that the PostgreSQL database is running locally.

1. Download the war from https://github.internet2.edu/TIER/shib-idp-ui/releases/
2. Create a users.csv with:
   root,{bcrypt}$2a$10$V1jeTIc0b2u7Y3yU.LqkXOPRVTBFc7SW07QaJR4KrBAmWGgTcO9H.,first,last,ROLE_ADMIN,user1@example.org
3. Start the database
   docker run --rm --name postgres-db -p 5432:5432 -e POSTGRES_PASSWORD=shibui -e POSTGRES_USER=shibui -d postgres

4. Create a basic Shibboleth IdP UI configuration with database settings
   
   

   Code Block
   language yml title startup - standalone war file
   spring: profiles: include: datasource: platform: postgres driver-class-name: org.postgresql.Driver url: jdbc:postgresql://localhost:5432/shibui username: shibui password: shibui jpa: show-sql: false properties: hibernate: dialect: org.hibernate.dialect.PostgreSQL95Dialect format_sql: true shibui: user-bootstrap-resource: file:users.csv roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_PONY

   
   

5. Run the war and tell it where to find your configuration files
   java -Xmx1g -jar shibui-1.16.0.war --spring.config.additional-location=file:application.yml

6. You can access the application at http://localhost:8080 and login with root/password

7. Running under https is accomplished by adding the following to your configuration file

   Code Block
   server: ssl: key-store: keystore.p12 key-store-password: password key-store-type: pkcs12 key-alias: tomcat key-password: password port: 8443

   
   

Tomcat web application

This example assume Tomcat but the procedure for Jetty or others will be similar.  Also assumes configuration files will be placed in /opt/shibui/ and that the Postgres database is already running.

Java Servlet Container

...

Create a /opt/shibui/users.csv with:

root,{bcrypt}$2a$10$V1jeTIc0b2u7Y3yU.LqkXOPRVTBFc7SW07QaJR4KrBAmWGgTcO9H.,first,last,ROLE_ADMIN,user1@example.org

Create a basic Shibboleth IdP UI configuration with database settings

spring:
  profiles:
    include:
  datasource:
    platform: postgres
    driver-class-name: org.postgresql.Driver
    url: jdbc:postgresql://localhost:5432/shibui
    username: shibui
    password: shibui
  jpa:
    show-sql: false
    properties:
      hibernate:
        dialect: org.hibernate.dialect.PostgreSQL95Dialect
        format_sql: true
shibui:
  user-bootstrap-resource: file:/opt/shibui/users.csv
  roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_PONY

Tell Tomcat where to find the configuration files by adding the following to $CATALINA_HOME/bin/setenv.sh

export JAVA_OPTS="$JAVA_OPTS -Dspring.config.additional-location=file:/opt/shibui/"

...

Docker

There are many ways to deploy an application with Docker and the following example is only meant to show a simplistic approach for demonstration purposes.  A docker compose example of a fully functioning Shibboleth IdP UI with full integration with the Shibboleth IdP can be found in the smoke-test testbed in the Git repository.  

1. Download the war from https://github.internet2.edu/TIER/shib-idp-ui/releases/ and place it in your Tomcat webapps dir.  Ex. /opt/tomcat/webapps/

1. Create a users.csv with:

   code

   Create a /opt/shibui/users.csv with:

   
   root,{bcrypt}$2a$10$V1jeTIc0b2u7Y3yU.LqkXOPRVTBFc7SW07QaJR4KrBAmWGgTcO9H.,first,last

   ,ROLE_ADMIN,user1@example.org

   Start the database

   Code Blockdocker run --rm --name postgres-db -p 5432:5432 -e POSTGRES_PASSWORD=shibui -e POSTGRES_USER=shibui -d postgres

   ,ROLE_ADMIN,user1@example.org

2. Create a basic Shibboleth IdP UI configuration with database settings

   Code Block
   language yml title tomcat webapp properties
   spring: profiles: include: datasource: platform: postgres driver-class-name: org.postgresql.Driver url: jdbc:postgresql://localhost:5432/shibui username: shibui password: shibui jpa: show-sql: false properties: hibernate: dialect: org.hibernate.dialect.PostgreSQL95Dialect format_sql: true shibui: user-bootstrap-resource: file:/opt/shibui/users.csv roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_PONY

Start the Shibboleth IdP UI

Code Block

3. 
   
   

docker run -p 8080:8080 -v application.yml:/opt/shibui/application.yml -v users.csv
4. Tell Tomcat where to find the configuration files by adding the following to $CATALINA_HOME/bin/setenv.sh
   export JAVA_OPTS="$JAVA_OPTS -Dspring.config.additional-location=file:/opt/shibui/
users.csv i2incommon/shib-idp-ui
5. "
6. You can access the application at http://localhost:8080 and login with root/password
7. Running under https is accomplished using the standard Tomcat or Jetty SSL connectors.