Requirements
- Docker or other OCI server OR Java 11 or higher (Tested with Amazon Corretto 11)
- 4-8 G memory
- RDBMS for persistent configuration storage:
- PostgreSQL (version 14 is the presumed default)
- Tested with latest docker images of MariaDB, MySQL and SQL Server.
- docker-compose and application.yml examples for each DB can be found in the testbed directory in the Git repository.
- A method for providing generated metadata to the associated IDP:
- Shared filesystem
- Git push/pull
- file transfer
- MDQ
Downloads
Pre-built docker images are on Docker Hub.
Releases for the Java war file can be found in the Internet2 GitHub project repository.
Quick Start Demo
Using Docker:
docker run \ -p 8080:8080 \ -p 9090:9090 \ i2incommon/shib-idp-ui:latest
As a standalone app:
VERSION=1.16.0 wget https://github.internet2.edu/TIER/shib-idp-ui/releases/download/v$VERSION/shibui-$VERSION.war java -jar -Dshibui.default-password={noop}abcd1234 shibui-$VERSION.war
Once up, the application can be accessed at http://localhost:8080/. The demo includes an in-memory H2 database, and a default administration user. For the docker demo, the user is "root" and password is "password". For the standalone app, there is no default password. The demo above sets it to "abcd1234" by setting application property shibui.default-password
.
API documentation from Swagger is available on port 9090. This port is exposed in this demo, but is not for user access in production.
Deployment Considerations
The principal output from the Shib IDP UI application are XML files. Metadata provider records in the UI correspond to MetadataConfiguration resources, which contain pointers to the actual entity metadata to consume. Metadata source records in the UI define individual entities, and each entity corresponds to a single XML file defining its metadata. The entity files are generally to be used with a LocalDynamicMetadataProvider, and the entity file names follow its convention of lower case hex-encoded SHA-1 digest of the entityID, suffixed with ".xml" The locations in the filesystem to save the output are defined by UI application properties, and have no defaults. See {Configuration} for details on how to define these.
The IDP reads metadata provider and entity metadata from these files, so they must be accessible by the IDP. There are a number of ways these files can get from the UI to the IDP:
- UI running as a web application on the same server as the IDP, and saved to local filesystem
- Shared filesystem between the UI and the IDP
- Git push/pull jobs
- file transfer jobs
Running
The Shibboleth IdP UI is built as a Java Spring Boot (https://spring.io/projects/spring-boot) application. It can be run as a standalone Java application (using an embedded Tomcat server), or as a Java web application running under an installed Tomcat or Jetty web application server. Both the standalone and web application methods can be used either directly on a web server, or packaged inside a Docker image. Internet2 supplies pre-built Docker images for released versions. The deployment method is largely dependent on your own infrastructure and knowledge base.
Configuration
Regardless of which method is used to run the Shibboleth IdP UI, configuration is required. Much of the behavior of the UI can be set and controlled through property files which can be in one or both of the following formats:
- application.properties - a Spring property file similar to the ones used in the Shibboleth IdP – a simple text file with a property name, equals sign, and the property value, one per line.
- application.yml - a YAML format file
Choosing which format to use is largely a personal choice as functionally they are identical. One caveat is in the case that there is both a properties and yml file, the properties file will be read first.
The ShibUI comes with a basic example of both, with the example application.properties file having the core settings for authentication, database connection information, users file, directory/location settings for where the UI should write out the metadata files and metadata-providers.xml file it manages, etc. The example application.yml file contains all the settings that impact the information, options, list elements, etc. that are actually shown in the UI.
There is no technical reason that you need to follow the examples and keep the distinction; you could manage everything through a single application.properties or application.yml if you wanted. On the other hand, it can be a convenient distinction to keep the core "internal/baked-in settings" distinct from the "front-end/UI" settings.
The application properties are Spring configuration files, and follow its built-in methods for where it looks for these files:
- The classpath root
- The classpath /config package
- The current directory (note, in the Internet2 Docker image the "current directory" will be the working directory which is /opt/shibui)
- The /config subdirectory of the current directory
- Immediate child directories of the config/ subdirectory
- File name option passed to the executable jar
--spring.config.location=file://{absolute-path-to-file}
- Directory name passed to the executable jar
--spring.config.location=file://{absolute-path-to-directory}
- Java property
-Dspring.config.location
- Environment variable
SPRING_CONFIG_LOCATION
- Property
spring.config.additional-location
Spring properties also utilize profile designations, which target the reading of an environment-specific configuration file. For example, when passing Java system property -Dspring.profiles.active=prod
, setting environment variable SPRING_PROFILES_ACTIVE=prod
, or setting property spring.profiles.active=prod
in application.properties, application startup will additionally load file application-prod.properties
, using the same search order as for application.properties.
Database properties
The Shib IDP UI can be integrated with a variety of back end databases to store registry information. The distribution includes JDBC drivers for PostgreSQL, MySQL, MariaDB, and SQL Server. With no custom database configuration, the application will default to an embedded in-memory H2 database, which will be cleared out on application exit. Database-specific properties to set are:
PostgreSQL:
spring: datasource: platform: postgres driver-class-name: org.postgresql.Driver url: jdbc:postgresql://db-hostname:5432/db-name username: shibui password: shibui jpa: properties: hibernate: dialect: org.hibernate.dialect.PostgreSQLDialect
MySQL:
spring: datasource: platform: mysql driver-class-name: com.mysql.cj.jdbc.Driver url: jdbc:mysql://db-hostname:3306/db-name username: shibui password: shibui jpa: properties: hibernate: dialect: org.hibernate.dialect.MySQL8Dialect
MariaDB:
spring: datasource: platform: mysql driver-class-name: org.mariadb.jdbc.Driver url: jdbc:mariadb://db-hostname:3306/db-name username: shibui password: shibui jpa: properties: hibernate: dialect: org.hibernate.dialect.MariaDB103Dialect
SQL Server:
spring: datasource: platform: sqlserver driver-class-name: com.microsoft.sqlserver.jdbc.SQLServerDriver url: jdbc:sqlserver://db-hostname:1433 username: sa password: Password1 jpa: properties: hibernate: dialect: org.hibernate.dialect.SQLServerDialect
Authentication
There are three methods for authenticating users. The first two involve choosing an encryption scheme for the password from one of the following:
- noop -- clear text password follows
- bcrypt -- the following value has been encrypted with the $2a$ Bcrypt algorithm (limitation of the underlying Spring library currently incorporated is that only the $2a$ Bcrypt algorithm is supported.)
1. A single root user. The default user is root
, but can be set to another username via property shibui.default-rootuser
. there is no default password in the war file, but it can be defined via property shibui.default-password
. NOTE: The Internet2 docker image does include a users.txt which has a default root password of password
.
2. Users defined in a bootstrap file, and imported at startup by defining property shibui.user-bootstrap-resource=file:/full/path/to/user-file
. In the Internet2 image, a sample users.txt file, with user root
with password password
, is included in the app folder, and imported in the default startup command.
The file is a comma-separated list of users to bootstrap, with a format of:
username,{ENCRYPT_SCHEME}password,firstname,lastname,ROLE_VALUE,email
Where ROLE_VALUE is one of:
ROLE_ADMIN -- No limits, can do anything the Shibboleth IdP UI supports. Currently, Metadata Provider configuration (and filter configuration) requires ROLE_ADMIN access
ROLE_USER -- These users can only add individual metadata sources and modify metadata sources that they created. When creating a new SP entry, that SP metadata will not be active until a user with ROLE_ADMIN approves it.
ROLE_ENABLE – These are "enhanced" ROLE_USER users that have the ability to enable/publish SP metadata but cannot access/configure Metadata Provider (and filter) configurations
Note: you must have at least one user with ROLE_ADMIN.
To encrypt a password using Bcrypt, there are online Bcrypt encryption tools. For security, if you want to encrypt offline, the following python script can be used.
import bcrypt
password = 'passw0rd123'
bytes = password.encode('utf-8')
salt = bcrypt.gensalt()
hash = bcrypt.hashpw(bytes, salt)
print(hash)
3. User authenticated via SAML
The Shibboleth IdP UI includes a Java-based SAML SP based on the Pac4j ( https://www.pac4j.org ) library (SAML support built on top of the Shibboleth Consortium's OpenSAML software). Configuration requires a few external steps and configuration of application.properties.
Configure, in a users file, at least one user that will match up with a Shibboleth IdP supplied user identifier with ROLE_ADMIN (password is a required field in the file but is irrelevant/ignored in the user file)
Create a copy of the Shibboleth IdP's metadata and place within the file system where the UI can access
Create the Metadata for the SP and place within the file system where the UI can access
Configure the following settings in application.properties or application.yml
shibui: default-password: "{noop}password" user-bootstrap-resource: file:/opt/shibui/conf/users.csv roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_ENABLE,ROLE_PONY pac4j-enabled: true pac4j: keystorePath: "/opt/shibui/conf/samlKeystore.jks" keystorePassword: "password" privateKeyPassword: "password" serviceProviderEntityId: "https://shibui.local/shibui" serviceProviderMetadataPath: "/opt/shibui/conf/sp-metadata.xml" identityProviderMetadataPath: "/opt/shibboleth-idp/metadata/idp-metadata.xml" forceServiceProviderMetadataGeneration: false callbackUrl: "https://shibui.local/shibui/callback" maximumAuthenticationLifetime: 3600000 simpleProfileMapping: username: urn:oid:0.9.2342.19200300.100.1.1 firstName: urn:oid:2.5.4.42 lastName: urn:oid:2.5.4.4 email: urn:oid:0.9.2342.19200300.100.1.3 groups: urn:oid:2.5.4.15 # businessCategory roles: urn:oid:1.3.6.1.4.1.5923.1.1.1.7 # eduPersonEntitlement
Entity Metadata Files
Entity metadata is not written to XML files by default. To periodically write the generated metadata files to a target directory, define:
shibui.metadata-dir=/path/to/metadata/output/directory shibui.taskRunRate=milliseconds-or-duration
Property shibui.metadata-dir
is an absolute path to the output directory. Entity filenames will be constructed by combining the directory and the base name derived from the SHA-1 digest of the entityID, adding the path separator if needed.
The shibui.taskRunRate
property (default 30000 or every 30 seconds) sets the output to be generated at timed intervals. The property can accept a number which represents milliseconds, or a Java Duration such as PT20M.
Each entity corresponds to a single XML file defining its metadata. To calculate the SHA-1 base name from an entityID, the following can be run on the command line:
$ echo -n "urn:test:foobar" | openssl sha1
d278c9975472a6b4827b1a8723192b4e99aa969c
Metadata Provider File
The metadata providers defined in the UI are not written to an external XML file by default. To periodically write the generated configuration to a filesystem resource, define:
shibui.metadataProviders.target=file:/path/to/provider-file.xml shibui.metadataProviders.taskRunRate=milliseconds-or-duration
Property shibui.metadataProviders.target
is the filename for the output xml. It expects a Java Resource, thus requires file:
before the path.
Similar to the entity task run rate, the shibui.metadataProviders.taskRunRate
property defaults to 30000 or every 30 seconds and can accept a number which represents milliseconds, or a Java Duration.
Metadata External Provider File
TODO
Other properties
The full set of properties used in the UI, with their defaults, are embedded in the war file application.yml and application.properties.
Spring properties can also be utilized to define server behavior. Logging levels, JDBC tuning, or HTTPS are common ones to use.
Enabling HTTPS Listener
Add these Spring properties to your configuration file:
server: ssl: key-store: keystore.p12 key-store-password: password key-store-type: pkcs12 key-alias: tomcat key-password: password port: 8443
Spring boot supports keystores in either the PKCS12 or JKS format. An example command to generate a new keystore:
keytool -genkeypair -alias tomcat -keyalg RSA -keysize 2048 -storetype PKCS12 -keystore keystore.p12 -validity 3650
Deployments Examples
Internet2 Docker image - base image with mounted configuration
In this example, properties include shibui.user-bootstrap-resource=file:/opt/shibui/users.csv
, along with connections to an existing database.
docker run -d \ -p 8080:8080 \ -e SPRING_PROFILES_ACTIVE=prod \ -v $PWD/conf/application.properties:/opt/shibui/application.properties \ -v $PWD/conf/prod/application.properties:/opt/shibui/application-prod.properties \ -v $PWD/conf/users.csv:/opt/shibui/users.csv \ -v $PWD/output/providers:/var/log/providers \ -v $PWD/output/metadata:/var/log/metadata \ i2incommon/shib-idp-ui:latest
There are many ways to deploy an application with Docker and the following example is only meant to show a simplistic approach for demonstration purposes. A docker compose example of a fully functioning Shibboleth IdP UI with full integration with the Shibboleth IdP can be found in the smoke-test testbed in the Git repository.
Standalone war file
This example assumes all the files are placed in the same directory that the war will be run from and that the PostgreSQL database is running locally.
- Download the war from https://github.internet2.edu/TIER/shib-idp-ui/releases/
- Create a users.csv with:
root,{bcrypt}$2a$10$V1jeTIc0b2u7Y3yU.LqkXOPRVTBFc7SW07QaJR4KrBAmWGgTcO9H.,first,last,ROLE_ADMIN,user1@example.org - Start the database
docker run --rm --name postgres-db -p 5432:5432 -e POSTGRES_PASSWORD=shibui -e POSTGRES_USER=shibui -d postgres Create a basic Shibboleth IdP UI configuration with database settings
startup - standalone war filespring: profiles: include: datasource: platform: postgres driver-class-name: org.postgresql.Driver url: jdbc:postgresql://localhost:5432/shibui username: shibui password: shibui jpa: show-sql: false properties: hibernate: dialect: org.hibernate.dialect.PostgreSQL95Dialect format_sql: true shibui: user-bootstrap-resource: file:users.csv roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_PONY
Run the war and tell it where to find your configuration files
java -Xmx1g -jar shibui-1.16.0.war --spring.config.additional-location=file:application.yml- You can access the application at http://localhost:8080 and login with root/password
Running under https is accomplished by adding the following to your configuration file
server: ssl: key-store: keystore.p12 key-store-password: password key-store-type: pkcs12 key-alias: tomcat key-password: password port: 8443
Tomcat web application
This example assume Tomcat but the procedure for Jetty or others will be similar. Also assumes configuration files will be placed in /opt/shibui/ and that the Postgres database is already running.
Download the war from https://github.internet2.edu/TIER/shib-idp-ui/releases/ and place it in your Tomcat webapps dir. Ex. /opt/tomcat/webapps/
Create a /opt/shibui/users.csv with:
root,{bcrypt}$2a$10$V1jeTIc0b2u7Y3yU.LqkXOPRVTBFc7SW07QaJR4KrBAmWGgTcO9H.,first,last,ROLE_ADMIN,user1@example.orgCreate a basic Shibboleth IdP UI configuration with database settings
tomcat webapp propertiesspring: profiles: include: datasource: platform: postgres driver-class-name: org.postgresql.Driver url: jdbc:postgresql://localhost:5432/shibui username: shibui password: shibui jpa: show-sql: false properties: hibernate: dialect: org.hibernate.dialect.PostgreSQL95Dialect format_sql: true shibui: user-bootstrap-resource: file:/opt/shibui/users.csv roles: ROLE_ADMIN,ROLE_NONE,ROLE_USER,ROLE_PONY
- Tell Tomcat where to find the configuration files by adding the following to $CATALINA_HOME/bin/setenv.sh
export JAVA_OPTS="$JAVA_OPTS -Dspring.config.additional-location=file:/opt/shibui/" - You can access the application at http://localhost:8080 and login with root/password
- Running under https is accomplished using the standard Tomcat or Jetty SSL connectors.