Date: Thu, 28 Mar 2024 20:58:01 +0000 (UTC) Message-ID: <134252964.7001.1711659481145@ip-10-10-7-29.ec2.internal> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_7000_917868039.1711659481143" ------=_Part_7000_917868039.1711659481143 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
This is an example of a CI/CD pipeline based on Jenkins and AWS = Elastic Container Service (ECS) that will build a Grouper Docker container = based on the InCommon Trusted Access Platform containers (https://hub.docker.c= om/u/i2incommon). If the build is successful, it will push the new imag= e to a private AWS Elastic Container Registry (ECR). In this case, the buil= d host runs with an IAM role that with an IAM policy attached to it that al= lows it access to push to this repo.
import = groovy.json.JsonSlurperClassic // IMPORTANT: to be able to determine the right S3 bucket to pull the confi= g from (dev, staging or prod) // The branch name MUST end with either -dev, -staging, or -prod pipeline { agent { // This is important, it tells Jenkins what worker nodes to use // Nodes with this label have an IAM role that is allowed to assume= a role // in the ICP account that has a very stripped down IAM policy label 'icp_ops' } environment { // AWS Docker Repo for Collab Platform maintainer =3D "************.dkr.ecr.us-east-2.amazonaws.com" // name of container, uses the FQDN of the collab host imagename =3D "${env.JOB_NAME.tokenize('/')[1]}" // tag, should be based on branch and is defined in the build stage= below tag =3D "" // environment, used to determine which S3 bucket to pull configura= tion from // e.g. dev is s3://****-dev-host-configs, staging is s3://****-sta= ging-host-configs // prod is s3://****-prod-host-configs shortenv =3D env.BRANCH_NAME.substring(env.BRANCH_NAME.lastIndexOf(= "-") + 1, env.BRANCH_NAME.length()) =20 // temp variable which will be replaced soon s3url =3D "s3://****-${shortenv}-host-configs" // The role to assume that is allowed to push to the ICP AWS Docker= repo and read s3 bucket role_to_assume =3D "arn:aws:iam::************:role/ti-prod-jenkins" // Not super important, just a session name to associate with assum= ing role, value doesn't seem to matter role_session_name =3D "***-ops" // placeholder for AWS credentials that will be populated when the = sts-assume-role // command is issued. This uses the IAM role of the running Jenkins= , currently "jenkins-agent-icp-ops" // in AWS account ************* (also known as TI Prod) // This role is able to assume the role "ti-prod-jenkins" in accoun= t ************=20 AWS_ACCESS_KEY_ID =3D "" AWS_SECRET_ACCESS_KEY =3D "" AWS_SESSION_TOKEN =3D "" region =3D "us-east-2" } stages { stage('Setting build context') { steps { script { if (env.BRANCH_NAME =3D=3D "master") { tag =3D "latest-test" } else { tag =3D env.BRANCH_NAME } sh 'mkdir -p tmp && mkdir -p bin' dir('tmp') { git([url: "https://github.internet2.edu/docker/util= .git", credentialsId: "jenkins-github-access-token"]) sh 'rm -rf ../bin/*' sh 'mv ./bin/* ../bin/.' } } } } stage('Clean') { steps { script { try { sh 'bin/destroy.sh >> debug' } catch (error) { def error_details =3D readFile('./debug'); def message =3D "BUILD ERROR: There was a problem b= uilding the Base Image. \n\n ${error_details}" sh "rm -f ./debug" handleError(message) } } } } stage('Build') { steps { script { // This will return a JSON object with credentials that= can be parsed and used to carry out the AWS // operations needed to build and push the container to= the Collab AWS account def stsjson=3D sh ( script: "aws sts assume-role --role-arn ${role_to_a= ssume} --role-session-name ${role_session_name}", returnStdout: true ).trim() def ststoken =3D new JsonSlurperClassic().parseText(sts= json) AWS_ACCESS_KEY_ID =3D "${ststoken.Credentials.AccessKey= Id}" // print "access key =3D ${AWS_ACCESS_KEY_ID}" AWS_SECRET_ACCESS_KEY =3D "${ststoken.Credentials.Secre= tAccessKey}" // print "secret key =3D ${AWS_SECRET_ACCESS_KEY}" AWS_SESSION_TOKEN =3D "${ststoken.Credentials.SessionTo= ken}" // print "session token =3D ${AWS_SESSION_TOKEN}" try { docker.withRegistry("${maintainer}/${imagename}") { try { baseImg =3D docker.build("${imagename}", "-= -no-cache --build-arg AWS_ACCESS_KEY_ID=3D${AWS_ACCESS_KEY_ID} --build-arg = AWS_SECRET_ACCESS_KEY=3D${AWS_SECRET_ACCESS_KEY} --build-arg AWS_SESSION_TO= KEN=3D${AWS_SESSION_TOKEN} --build-arg s3url=3D${s3url} --build-arg ENVIRON= =3D${shortenv} .") } catch (error) { handleError(error) } } } catch (error) { handleError(error) } } } } stage('Test') { steps { script { try { echo "need to create tests" //sh 'bin/test.sh 2>&1 | tee debug ; test ${= PIPESTATUS[0]} -eq 0' } catch (error) { handleError(message) } } } } stage('Push') { steps { script { withEnv(["AWS_ACCESS_KEY_ID=3D${AWS_ACCESS_KEY_ID}", "A= WS_SECRET_ACCESS_KEY=3D${AWS_SECRET_ACCESS_KEY}", "AWS_SESSION_TOKEN=3D${AW= S_SESSION_TOKEN}" ]) { =20 def ecrlogincmd=3D sh ( script: "aws ecr get-login --no-include-email -= -region ${region}", returnStdout: true ).trim() =20 sh "${ecrlogincmd}" docker.withRegistry("https://${maintainer}") { baseImg.push("${tag}") } } } } } stage('Notify') { steps { echo "${imagename}:${tag} pushed to ${maintainer}" slackSend color: "good", message: "${maintainer}/${imagenam= e}:${tag} pushed to ECR" } } } post { always { echo "Done Building." } failure { slackSend color: "good", message: "Build failed" handleError("BUILD ERROR: There was a problem building ${mainta= iner}/${imagename}:${tag}.") } } } def handleError(String message) { echo "${message}" currentBuild.setResult("FAILED") slackSend color: "danger", message: "${message}" //step([$class: 'Mailer', notifyEveryUnstableBuild: true, recipients: '= chubing@internet2.edu', sendToIndividuals: true]) sh 'exit 1' }
The following is a Dockerfile that is used by the above build script tha= t layers in the necessary local configurations for running Grouper needed b= y the Internet2 Collaboration Platform that houses spaces.internet2.edu. The files= that contain secrets are located in an encrypted S3 bucket and the build h= ost runs with an IAM role that allows it to access this bucket. This exampl= e is for a Grouper UI container. The AWS RDS (Relational Database Service) = is used by this service.
FROM i2= incommon/grouper:4.1.6 ARG s3url ARG ENVIRON ARG AWS_ACCESS_KEY_ID ARG AWS_SECRET_ACCESS_KEY ARG AWS_SESSION_TOKEN ENV ENV=3D$ENVIRON ENV USERTOKEN=3DICP ENV S3_BUCKET=3D$s3url/grouper.at.internet2.edu # Bring in public IdP signing cert from AWS ADD https://s3.us-east-2.amazonaws.com/****-metadata-public/certs/icmp_sign= ing.crt /etc/pki/tls/certs/ # Install and Upgrade pip, install awscli for getting making calls to AWS. RUN yum -y install python3-pip && \=20 pip3 install --upgrade pip && \ pip3 install awscli # Bring in sp cert and private key from AWS S3 RUN aws s3 cp $S3_BUCKET/shib/sp-cert.pem /etc/shibboleth && \ aws s3 cp $S3_BUCKET/shib/sp-key.pem /etc/shibboleth # Clears the Default SSL config for Apache RUN cp /dev/null /etc/httpd/conf.d/ssl-enabled.conf =20 # Copy in the files that congigure SAML, Apache, and Tomcat Catalina COPY container_files/shibboleth/ /etc/shibboleth/ COPY container_files/httpd/grouper.conf /etc/httpd/conf.d COPY container_files/tomee/server.xml /opt/tomee/conf/ # Copy in mariadb client for reading external DBs COPY container_files/lib/ /opt/grouper/grouperWebapp/WEB-INF/lib/ # Copy in the Grouper properties files in Git COPY container_files/grouper/ /opt/grouper/grouperWebapp/WEB-INF/classes/ # Update configuration variables based on ENV COPY container_files/bin/setenvironment.sh /usr/local/bin/ RUN chmod 755 /usr/local/bin/setenvironment.sh && /usr/local/bin/se= tenvironment.sh
The following is an example Elastic Container Service (ECS) task definit= ion for running this container. Configuration such as log groups for Cloudw= atch, the port to direct traffic from the Elastic Load Balancer (ELB), the = memory requirements for the container, as well as the ECR location to pull = the container from. Note the "command" value, as this instructs the contain= er as what component to act as. In this case, it is going to run as a Group= er UI. Alternately, the command could be "loader" (to run as a Grouper Load= er), or "ws" (to run as a Grouper Web Services container). This task defini= tion is part of an overall Cloudformation template that builds out the envi= ronment that houses other TIER containers used by the Internet2 Collaborati= on Platform.
{ "taskDefinitionArn": "arn:aws:ecs:us-east-2:************:task-definitio= n/dev-grouper-at:51", "containerDefinitions": [ { "name": "dev-grouper-at", "image": "************.dkr.ecr.us-east-2.amazonaws.com/grouper.= at.internet2.edu:v0.6.4-4.1.6-dev", "cpu": 0, "memory": 2048, "portMappings": [ { "containerPort": 80, "hostPort": 0, "protocol": "tcp" } ], "essential": true, "command": [ "ui" ], "environment": [], "mountPoints": [], "volumesFrom": [], "secrets": [ { "name": "GROUPER_MORPHSTRING_ENCRYPT_KEY", "valueFrom": "arn:aws:secretsmanager:us-east-2:********= ****:secret:R_GROUPER_MORPHSTRING_ENCRYPT_KEY" }, { "name": "GROUPER_DATABASE_PASSWORD", "valueFrom": "arn:aws:secretsmanager:us-east-2:********= ****:secret:R_GROUPER_DATABASE_PASSWORD" }, { "name": "GROUPER_LDAP_PASSWORD", "valueFrom": "arn:aws:secretsmanager:us-east-2:********= ****:secret:R_GROUPER_LDAP_PASSWORD" } ], "logConfiguration": { "logDriver": "awslogs", "options": { "awslogs-group": "iam-dev", "awslogs-region": "us-east-2", "awslogs-stream-prefix": "grouper" } } } ], "family": "dev-grouper-at", "executionRoleArn": "arn:aws:iam::************:role/ecsRoleSecretsManag= erRead", "revision": 51, "volumes": [], "status": "ACTIVE", "requiresAttributes": [ { "name": "com.amazonaws.ecs.capability.logging-driver.awslogs" }, { "name": "ecs.capability.execution-role-awslogs" }, { "name": "com.amazonaws.ecs.capability.ecr-auth" }, { "name": "com.amazonaws.ecs.capability.docker-remote-api.1.19" }, { "name": "ecs.capability.secrets.asm.environment-variables" }, { "name": "ecs.capability.execution-role-ecr-pull" } ], "placementConstraints": [], "compatibilities": [ "EXTERNAL", "EC2" ], "requiresCompatibilities": [ "EC2" ], "memory": "2048", "registeredAt": "2023-05-15T19:15:02.832Z", "tags": [] }
The following is an apache configuration for running the Grouper contain= er in ECS. Since containers have to sit behind load balancers, the = remote IP address is that of the load balancer. However, the load balancer = inserts a header (X-Forwarded-For) that contains the IP address of the actu= al client, so that is logged. Rather than having the httpd listen = on 2 ports, if the request comes in as http (as seen in the X-Forwarde= d-Proto), it will redirect to https.
ServerN= ame https://grouper.at.internet2.edu:443 UseCanonicalName On RemoteIPHeader X-Forwarded-For RewriteEngine On RewriteCond %{X-Forwarded-Proto}i http RewriteRule ^ https://%{HTTP_HOST}:443%{REQUEST_URI} [R=3D302,L,QSA] RewriteEngine on RewriteRule "^/$" "/grouper/" [R] # log the X-Forwarded-For (the real client IP) header if present LogFormat "httpd;access_log;%{ENV}e;%{USERTOKEN}e;%{X-Forwarded-For}i %l %u= %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" proxy SetEnvIf X-Forwarded-For "^.*\..*\..*\..*" forwarded CustomLog "/tmp/logpipe" combined env=3D!forwarded CustomLog "/tmp/logpipe" proxy env=3Dforwarded ErrorLogFormat "httpd;error_log;%{ENV}e;%{USERTOKEN}e;[%{u}t] [%-m:%l] [pid= %P:tid %T] %7F: %E: [client\ %a] %M% ,\ referer\ %{Referer}i"
Note: Since TLS termination typically happens at the ELB, inform Tomcat = that it should create URLs of the form https by specifying
secure=3D"true" in the Connector de= claration.