This post appeared originally in our sysadvent series and has been moved here following the discontinuation of the sysadvent microsite
Our company has embraced our local GitLab installation extensively. At its core, GitLab provides a repository management system based on the Git versioning system. A very practical extension to GitLab is the GitLab CI feature.
In short, the GitLab CI is a set of commands that can be run when a
repository receives a push, when an API is called, or when it is triggered
by someone through the GUI. The commands are specified in a given YAML
formatted configuration file - .gitlab-cy.yml
which contains a set of
specific keywords and arguments.
The software that performs the actual commands - a GitLab Runner - is written in go and can be installed as a standalone script on one or more separate servers. A Runner can be set up as shared or specific, all according to your preference.
Jekyll
As noted in an earlier blog-entry, our techblog and this sysadvent calendar uses the Jekyll software to produce a static site from markdown content. This content, together with custom templates are stored in Git repositories in our private GitLab server.
At this point it should be noted that GitHub Pages not only supports Jekyll, but also lets you configure custom domains. So why not just use GitHub Pages? In our case, we are a corporation and would like these pages to be served from main our corporate address (hello SEO). More importantly, the Jekyll software is easy to set up and use, and as such serves as a good example of the GitLab CI workflow.
GitLab CI
In our case, we would like to do the following steps after one of our employees pushes changes to the Jekyll Git repository:
- Build the static site
- Perform some consistency checks on the result
- Publish the site
We do start out by declaring the 3 stages in the top of our gitlab-ci.yml
file:
---
stages:
- build
- test
- deploy
Build stage
In this stage, we would like to build the website with the Jekyll software. A GitLab Runner can start a new docker container and check out the git repository inside the container for us:
image: containers.redpill-linpro.com/ci-images/ruby:2.3
build site:
stage: build
script:
- bundle install --path=vendor
- bundle exec jekyll build
- mv _site sysadvent
artifacts:
expire_in: 1 month
paths:
- sysadvent
Here, the image
keyword defines which docker container we should run. This
container can be self-hosted as all GitLab repositories also includes
a docker registry.
The build site
keyword defines a stage block, and the “stage” keyword
defines which stage this is.
The script
keyword defines a shell script that the Runner should run in
this stage. In this case, the script installs Jekyll, instructs Jekyll
to build the site, and renames the resulting _site
directory to
something more natural.
Finally the artifacts
keyword tells the Runner that we want to save
the result defined in the paths
keyword. The Runner will return this
result as a compressed volume and store it in the GitLab
repository. The expire_in
keyword sets an expiry date for the
artifacts. This is a useful variable to set as default expiration is never.
Test stage
In this stage, we want to test the result of the previous build:
---
test site:
image: containers.redpill-linpro.com/ci-images/linkchecker:latest
stage: test
script:
- webfsd -p 4000 -f index.html
- http GET http://localhost:4000/sysadvent/varnish-probe | grep OK
- linkchecker http://localhost:4000/sysadvent/
This stage is set up to run in a separate, specific docker
instance specified through the image
keyword. The previous
artifacts
step ensures that the build created in the previous step
is imported into this container and unpacked in a “sysadvent”
directory.
The main takeaway from this stage is the script. Here, we set up a local web server serving the newly unpacked sysadvent site on port 4000 (Jekyll also includes a web server, but the original author took a shortcut by not installing the Jekyll software in the container in this stage). The script then performs two separate, simple consistency checks on the site.
The first consistency check is checking for the varnish-probe
file. This file is used as a health check for our varnish
frontends and failing to deploy this file would result in the production
frontend varnish instances considering this backend as down. The second
check checks link consistency through the linkchecker
tool. If any of the script
lines returns a non-zero result, the Runner and GitLab CI will stop the
CI process at that point and display a corresponding error in the GitLab GUI.
These checks can of course be complemented with more complex testing.
Deploy stage
At this point we have built the site and tested it. Now we need to deploy it:
---
deploy:
stage: deploy
environment: production
script:
- export LC_ALL=C.UTF-8
- export LANG=C.UTF-8
- s3cmd --no-mime-magic --access_key=$ACCESS_KEY --secret_key=$SECRET_KEY --acl-public --delete-removed --delete-after --no-ssl --host=$s3host --host-bucket='%(bucket)s.$s3host' sync sysadvent s3://rl_web.sysadvent.prod
In this case, we are publishing the site to a local s3 compliant site redundant Ceph cluster. In order to do that, we need to authenticate the publishing process. So how do we store secrets safely in what is essentially a public Git repository?
Enter the GitLab concept of Secret Variables. This is located in the project configuration settings, and any key/value pairs entered here are not stored in the actual Git repository.
As shown, these variables can be accessed as environment
variables in any script-blocks in the .gitlab-ci.yml
file.
Runners
In order to get these steps up and running, we need to configure one or more of Runners to do the actual work. Installing and configuring a runner is documented in the GitLab docs.
After a Runner is configured it should be possible to choose it in the
GitLab project configuration menu. In our case, we have used a specific
runner that serves only our needs. You can also set up several Runners with
different configurations and control which Runner does which step
through the tags
keyword. A Runner will then only pick up a job that
has a corresponding tag.
The result
GitLab will show a graphical representation of the CI process under the Pipelines heading:
In the figure, we can see a check-in triggering three stages. Also present are buttons to trigger the pipeline again. Pressing initial status button allows you to drill down into a more detailed view of the pipeline, eventually ending in logs from individual stages:
If an artifact is built in this stage, the artifact is shown on the right hand side.
Update
- 2024-01-29 Fixed link to GitLab runner documentation.
The irony of insecure security software
It can probably be understood from my previous blog post that if it was up to me, I’d avoid products like CrowdStrike - but every now and then I still have to install something like that. It’s not the idea of “security software” per se that I’m against, it’s the actual implementation of many of those products. This post lists up some properties that should be fulfilled for me to happy to install such a product.