This is the status of the work. Requirements 1 to 9 are based on TIER Docker Container Specification. Further requirements (from number 10 on) are based on the agreed Statement of Work.

RequirementDescription (requirement + optionally solution comments in italics)State
1. Base Linux ImageBase Linux Image
  1. All TIER developed containers must be based on the current Centos image.  As of March, 2018, this is Centos 7.
    1. Source: standard maintained Centos 7 docker image
    2. (Under Discussion) potential use of Centos 7 image from Dockerhub that includes what is needed to use systemd as init (instead of supervisord).  We may enable this option if obtaining/implementing the logging changes we need to supervisord are hard - - https://hub.docker.com/r/centos/systemd/
    3. When build pipelines are published for production, they must include a yum update step.
  2. TIER Ancillary Containers may be based on other Linux distributions
    1. An ancillary container is a non-TIER Component container (e.g., a database, message queue manager, etc.) that is used as part of a docker-compose to enable the TIER Component to function.
    2. TIER Ancillary Containers may be based on other non-Centos Linux distributions as long as the container in question is published by the organization maintaining the product and and TIER-related changes are marginal.
done
2. Servlet EngineTomcat will be used whenever a servlet engine is needed.done
3. Java DistributionZulu should be used.done
4. Database
  1. If a relational database is provided within a container, MARIADB will be used.
  2. In general, database support is normally handled externally by the user or via a TIER-maintained MARIADB container.

Note: The midPoint repository can be attached to the midPoint server in a flexible way. It can be either deployed in an (alternative) Docker container, or be provided externally either on premises or in the cloud.

done
5. Multi-Process ContainerSupervisord will be used whenever a container needs more than one process.done
6. TIER BeaconRun the TIER Beacon code on a regular interval as specified in the documentation.  Unless the component has its own scheduling mechanism for running external code, this requirement will usually result in the need to support cron and run supervisord in the container.done

7. Container Configuration

a) Standard Data

  1. Containers may receive configuration data via the environment as described below for Secret Data (7.b)
  2. Configuration data may be mounted into the container from external storage
  3. Configuration data may be "burned" into the container while it is being built.
  4. There are many trade-offs between 2 and 3, some environments will choose to enable the end user to build their containers using either method.
done

7. Container Configuration

b) Secret Data

  1. The preferred mechanism to support data that must be protected (e.g., passwords, keys, etc.) is Swarm-mode Docker Secrets.
  2. Docker secrets are read-only to the application.
  3. Secret Processing - Docker Secrets are processed using one of the two mechanisms described below
    1. Secrets/Pointers-to-Secrets are passed in the environment using the syntax described below.  Either a single value may be supplied or, with the _FILE suffix, a file pointing to a docker swarm secret location 
      1. COMPONENT_DATABASE_PASSWORD=foobar
      2. COMPONENT_DATABASE_PASSWORD_FILE=/run/secrets/my_password_file
      3. Container startup scripts
        1. Start-up scripts process the environment and do whatever setup is needed to make secrets usable in the application.
        2. If the environment contains both a _FILE and name-only variables, the _FILE form is to be used.
      4. Documentation/comments for each attribute is required.
    2. A naming convention is developed for all application files that will exist in /run/secrets.  Scripting within the container appropriately processes these files, linking them to the application components as appropriate.  Documentation/comments re: the naming convention and files is required.
done
8. Container Orchestration
  1. Containers designed for compatibility/ease of use with Docker SWARM mode using Docker Stack Deploy and Compose files.
  2. Work to not preclude the use of other orchestration frameworks.
  3. Secrets are automatically mounted in /run/secrets by docker stack deploy using a compose file.
currently using docker-compose up command, compatibility with other orchestration frameworks is out of scope of SoW
9. Logging
  1. All logs from all elements within a container are written to stdout
  2. Goal: easily parsable records; future work is likely to include json formatted logs
  3. Lines (records) within each log file start with the following format
    1. Component Name (e.g., Shibboleth IdP, Grouper Loader, etc.)

    2. Native logfile name (e.g., Catalina.out, shibd.log, etc.)

    3. Environment (e.g., Prod, Dev, Test)

    4. A user supplied token via the environment

    5. The text of the logfile line, without modification.
  4. Records within a line are separated by the semi-colon character.  Semicolons are not permitted in the first four fields and must be removed if present.
  5. Spaces also need to be removed from the (c) Environment and (d) User Supplied fields of each record. If anyone remembers why we need ro remove these spaces, please comment here.
  6. Example Records

    1. supervisord;console;testing;Build:1.2.3;2018-04-02 18:27:30,778 CRIT Set uid to user 0

    2. tomcat;catalina.out;testing;Build:1.2.3;2018-04-02 18:27:32,915 [main] INFO  org.apache.coyote.http11.Http11NioProtocol- Initializing ProtocolHandler ["https-jsse-nio-443"]

    3. Timestamps in logs must default to UTC.  Documentation should exist to assist users with changing this default to a local timezone.  The default of UTC instead of EST or PST seems logical given that many future campus deployments will include components deployed in multiple timezones for redundancy.

done

10. Shibboleth integrationUsers can be authenticated to midPoint using Shibboleth.done
11. CI/CD setupQA, automated testing setupdone

(the CI tests can be - of course - developed further, if needed)
12. UpgradesDetect and report incompatibilities of DB schema. Point to the upgrade documentation.done
13. Design of a reference solutionProvide a design of a sample deployment of midPoint in the university environment. Other outputs of "Analysis/design" SoW sections (e.g. providing consultations) are intangible.done (the sample deployment can be developer further, if needed; see "TODO" points in the description of the demo/complex scenario)

The following table maps individual sections of the Statement of Work to the above items.

SoW sectionReference
Analysis/designItem 13
Docker fundamentalsItems 1, 3, 7
LoggingItem 9
CI/CD setupItem 11
Repository configurationItem 4
Shib integrationItem 10e
UpgradesItem 11
DocumentationAll the items
Add-onsItems 5, 6

Besides that, Analysis/design is carried out "as needed", by providing consultations as necessary.

  • No labels