My previous post dove into using GitLab's Shared Runners to verify, in an F# SQLProvider data layer, that the code and schema agree. In this post we'll explore getting off the shared runner infrastructure, and using our own Kubernetes cluster for private CI. I happen to have a Digital Ocean Kubernetes cluster (DOK) - so that's what where my k8s cluster will live, however I'll be sticking with normal
kubectl commands in this post, so hopefully it'll apply to any k8s installation.
In this post, I'll be mostly focused on the moving from shared runners to dedicated private CI in our own k8s cluster for our SQLProvider F# project - but that doesn't mean you can't use this post to guide your own CI/CD infrastructure setup for your own project, so cheers 🍻!
Getting a Cluster
First thing's first, get a k8s cluster up and running. Using a cloud provider makes this easy-peasy, but there are tons of way to stand up a cluster using various k8s distributions. The key is to make sure you have an up to date kubernetes version, I'll be using
v1.16.2, but anything that supports helm v3 and GitLab's Runners should work.
$ k version $ helm version
Connecting the Cluster
Your k8s provider, or your custom setup, will have instructions on how to connect
helm on your local. For Digital Ocean, setting up an API key in your Digitial Ocean account and using the
doctl CLI sets up OAuth tokens for
kubectl to authenticate with the k8s cluster. Once you have that in place, you can run a quick
kubectl cluster-info and
kubectl get all --all-namespaces to see if your local CLI tooling can connect to the cluster.
Once that's settled, we can install our target workload on the cluster - GitLab Runners that support services. That'll be the key for us, since our SQLProvider based CI setup requires a postgres service.
We'll be following these docs for the k8s runner setup insetad of the integrated setup. The integrated setup uses helm v2, which needs to install tiller on the cluster which I (and others) don't recommend due to its security implications. We can still use helm charts, but we have helm v3 because we live in the future 🔮.
First we'll need to add GitLab's helm repo:
$ helm repo add gitlab https://charts.gitlab.io
Now we can mess with some yaml, the docs give us the basic picture, but I had to play with it a bit to get the services part working correctly. Here are the fruits of my labor with explainers in comments:
gitlabUrl: https://gitlab.com/ # Can set with -set runnerRegistrationToken=$CI_TOKEN # runnerRegistrationToken: "" # how many jobs should be able to be ran concurrently concurrent: 3 # how often we check, in seconds, for work checkInterval: 5 # Role Based Access Control - #veryMuchGoodSecure ? rbac: create: true runners: # default image for jobs image: alpine:3.11.3 locked: false # This set an env var that presumably allows the postgres service to be used envVars: - name: DOCKER_ALLOWED_SERVICES value: "postgres:12-alpine"
Now that we have this yaml in place we can use helm to deploy the runner
# pull this from settings -> CI -> runner token export CI_TOKEN=runner-registration-token-from-gitlab # install! helm install -n YOUR_NAMESPACE \ YOUR_RELEASE_NAME \ -f values.yaml \ gitlab/gitlab-runner \ --set runnerRegistrationToken=$CI_TOKEN
I found this stellar SO answer that informs us about the differences between docker networking and kubernetes networking as it pertains to the executors in GitLab's runners. If you, for example, had a postgres service accessed via the
postgres hostname in a Shared Runner context (docker executor), then you would need to switch to using
127.0.0.1:5432. That change will handle the
could not translate host name. This commit is an example refactor of scripts+code that would be necessary to move from shared runners to kubernetes runners.
Now that you have the runners deployed, you can disable shared runners for your GitLab project and push commits and get blazing fast, private CI for just your projects 🎉! This is an example merge request for our ongoing fsharp project that shows all the work we did in this post.
Now that we have our own private CI, it's time to think about building out a web API layer for our cat app, so we can have something more robust than a CLI app to deploy to. Before that, we'll need dockerize too. Onwards and upwards!