My previous post dove into using GitLab's Shared Runners to verify, in an F# SQLProvider data layer, that the code and schema agree. In this post we'll explore getting off the shared runner infrastructure, and using our own Kubernetes cluster for private CI. I happen to have a Digital Ocean Kubernetes cluster (DOK) - so that's what where my k8s cluster will live, however I'll be sticking with normal helm and kubectl commands in this post, so hopefully it'll apply to any k8s installation.
In this post, I'll be mostly focused on the moving from shared runners to dedicated private CI in our own k8s cluster for our SQLProvider F# project - but that doesn't mean you can't use this post to guide your own CI/CD infrastructure setup for your own project, so cheers 🍻!
Getting a Cluster
First thing's first, get a k8s cluster up and running. Using a cloud provider makes this easy-peasy, but there are tons of way to stand up a cluster using various k8s distributions. The key is to make sure you have an up to date kubernetes version, I'll be using v1.16.2, but anything that supports helm v3 and GitLab's Runners should work.
In order to verify your cluster is up, try to connect to with some standard tooling like helm and kubectl - if you don't have those tools follow the links and get installing ⏬!
$ k version
$ helm version
Connecting the Cluster
Your k8s provider, or your custom setup, will have instructions on how to connect kubectl and helm on your local. For Digital Ocean, setting up an API key in your Digitial Ocean account and using the doctl CLI sets up OAuth tokens for kubectl to authenticate with the k8s cluster. Once you have that in place, you can run a quick kubectl cluster-info and kubectl get all --all-namespaces to see if your local CLI tooling can connect to the cluster.
Once that's settled, we can install our target workload on the cluster - GitLab Runners that support services. That'll be the key for us, since our SQLProvider based CI setup requires a postgres service.
Runner Setup
We'll be following these docs for the k8s runner setup insetad of the integrated setup. The integrated setup uses helm v2, which needs to install tiller on the cluster which I (and others) don't recommend due to its security implications. We can still use helm charts, but we have helm v3 because we live in the future 🔮.
First we'll need to add GitLab's helm repo:
$ helm repo add gitlab https://charts.gitlab.io
Now we can mess with some yaml, the docs give us the basic picture, but I had to play with it a bit to get the services part working correctly. Here are the fruits of my labor with explainers in comments:
values.yaml
gitlabUrl: https://gitlab.com/
# Can set with -set runnerRegistrationToken=$CI_TOKEN
# runnerRegistrationToken: ""
# how many jobs should be able to be ran concurrently
concurrent: 3
# how often we check, in seconds, for work
checkInterval: 5
# Role Based Access Control - #veryMuchGoodSecure ?
rbac:
create: true
runners:
# default image for jobs
image: alpine:3.11.3
locked: false
# This set an env var that presumably allows the postgres service to be used
envVars:
- name: DOCKER_ALLOWED_SERVICES
value: "postgres:12-alpine"
Now that we have this yaml in place we can use helm to deploy the runner
# pull this from settings -> CI -> runner token
export CI_TOKEN=runner-registration-token-from-gitlab
# install!
helm install -n YOUR_NAMESPACE \
YOUR_RELEASE_NAME \
-f values.yaml \
gitlab/gitlab-runner \
--set runnerRegistrationToken=$CI_TOKEN
Note!
I found this stellar SO answer that informs us about the differences between docker networking and kubernetes networking as it pertains to the executors in GitLab's runners. If you, for example, had a postgres service accessed via the postgres hostname in a Shared Runner context (docker executor), then you would need to switch to using 127.0.0.1:5432. That change will handle the could not translate host name. This commit is an example refactor of scripts+code that would be necessary to move from shared runners to kubernetes runners.
Done!
Now that you have the runners deployed, you can disable shared runners for your GitLab project and push commits and get blazing fast, private CI for just your projects 🎉! This is an example merge request for our ongoing fsharp project that shows all the work we did in this post.
Now that we have our own private CI, it's time to think about building out a web API layer for our cat app, so we can have something more robust than a CLI app to deploy to. Before that, we'll need dockerize too. Onwards and upwards!
.