data.gov.uk Helm Charts

August 26, 2026 ยท View on GitHub

Please note that the DGU applications do not run on a automated CI/CD pipeline, so please read this to ensure that your deployment is successful.

Getting started

See Helm's documentation to get started with Helm.

See the GOV.UK Kubernetes cluster docs for an introduction to the cluster or ask #govuk-platform-engineering in Slack.

Local development uses k3d which runs a Kubernetes cluster locally using Docker.

Prerequisites

  1. Install Helm
brew install helm
  1. Install Docker

Install from here

  1. Install k3d
brew install k3d
  1. Install kubectl
brew install kubectl
  1. Clone the repository
git clone git@github.com:alphagov/govuk-dgu-charts.git

Setting up a local CKAN cluster

Advantage of using k3d is that we don't need to worry about creating an ingress controller. This aligns with how we run the cluster in EKS as that manages creation of the ingress controller for us.

  1. Create the k3d cluster
k3d cluster create local-ckan --api-port 6550 -p "8081:80@loadbalancer" --registry-create local-registry
  1. Switch kubectl to point to your local cluster:
kubectl config use-context k3d-local-ckan
  1. Locate the port the registry is running on:
docker ps -f name=local-registry

The remaining steps will assume our local registry is running on localhost:53492

  1. Build a docker image

If we want to push an image to our local registry we have to tag our image. See here for background on local Docker registries.

So if our local registry is running at localhost:53492 and we want to build ckan 2.10.4 we have to tag our image as follows:

docker build -t localhost:53492/ckan:2.10.4 -f docker/ckan/2.10.4.Dockerfile .

Note: for Find build the dev.Dockerfile version

docker build -t localhost:53492/datagovuk_find:dev -f docker/dev.Dockerfile .
  1. Push the image to our local registry
docker push localhost:53492/ckan:2.10.4
  1. Use the image in our Helm chart

Currently govuk-dgu-charts/charts/ckan/values.yaml points to the test environment:

environment: test

ckan:
  replicaCount: 1
  ...

Update govuk-dgu-charts/ckan/images/test/ckan.yaml with our new image so it looks like this:

repository: local-registry:53492/ckan
tag: 2.10.4
branch: main

Note that local-registry matches the name of the registry we passed in step 1. If we want to use a different image from a different environment then set environment appropriately.

  1. Apply the local dev Helm Chart
kubectl apply -f local-dev.yaml

This sets up some initial configuration in the cluster.

  1. Switch to the datagovuk namespace
kubectl config set-context --current --namespace=datagovuk
  1. Install the application Helm charts
helm install ckan-dev ./charts/ckan/ -n datagovuk
helm install datagovuk-dev ./charts/datagovuk/ -n datagovuk
helm install dgu-shared-dev ./charts/dgu-shared/ -n datagovuk

Now your local CKAN deployment will use the localhost:54392/ckan.2.10.4 image.

  1. On your local machine, update the /etc/hosts file as follows
127.0.0.1	localhost find.eks.test.govuk.digital ckan.eks.test.govuk.digital

datagovuk-find can be accessed at find.eks.test.govuk.digital:8081 ckan can be accessed at ckan.eks.test.govuk.digital:8081

Running scripts on the cluster

Sometimes it is necessary to run an ad hoc scripts on the cluster for things like removing datasets for a publiisher if there are legacy datasets or orphaned datasets, to do this please follow these steps:

  • Update the values-env files script section with the script to run. If you want to try it out in Integration first, then just update values-integration.yaml first and follow steps to test it out before updating other environments. When you are ready to run the script against Staging and Production you will need to create a PR for review and merge it in to deploy onto the EKS cluster.
  • Before the new job runs you may need to remove existing script job kubectl delete job/ckan-ckan-script -n datagovuk, otherwise the new job will fail with a sync error.
  • After the job has completed the logs should be available to view in argo or via the kubectl logs command if needed.

NOTE - unlike other config files the image used in the script-job.yaml is pinned, this is to prevent the job from re-running each time there is a new CKAN tag. So if your script requires a later version of the CKAN build then also update the CKAN tag in the script-job.yaml

Test Helm chart in EKS

  1. update the targetRevision in ckan-application.yaml or datagovuk-application.yaml to be the branch you want to test against.
  2. update the target revision in Argo on dgu-app-of-apps to be the branch you want to test, normally on the Integration cluster.
  3. after testing is complete remember to set the target revision back to main in dgu-app-of-apps.
  4. if you are creating a PR drop the commit which updates the targetRevision in step 1.

Checking the yaml created from the chart

To see what yaml is created from the chart, run this command in the relevant directory -

helm template . --debug --values values.yaml > values-datagovuk.yaml

In this case the command was run in the charts/datagovuk directory, the output will be in the same directory for you to inspect.

Github API token permissions

When creating fine-grained API tokens, ensure that read/write permissions for PRs and Content are allowed (this will allow for PR and commit creation) and that CI user is used to create the PRs.

Ephemeral cluster

An ephemeral cluster is now available to test changes to the CKAN and datagovuk charts and infrastructure without impacting the other stable EKS environments.

To access the ephemeral cluster from your terminal you will need to log in to the govuk-test account

eval $(gds-cli aws govuk-test-admin -e --art 8h)

and follow these steps to switch to the test environment.

To recreate the cluster from scratch if there is an existing ephemeral datagovuk cluster running, destroy the datagovuk cluster by deleting dgu-app-of-apps in the ephemeral Argo CD website and recreate the cluster by running the following command on the terminal under the charts directory (the environment variable passed in should match your ephemeral cluster name):

helm upgrade -n cluster-services datagovuk-argo-bootstrap argo-bootstrap --set environment=eph-aaa113

To test your changes on the ephemeral cluster without merging them into the main branch:

  • update the targetRevision in the ckan-application.yaml and/or datagovuk-application.yaml and push this change up to your branch onto Github
  • if you have deleted the dgu-app-of-apps module then run the helm upgrade command as above
  • update the targetRevision in dgu-app-of-apps on the ephemeral Argo CD website.
  • after you have finished testing please remove the commit which changes the targetRevision before merging your changes into the main branch

Schemas

We have several Custom Resource Definitions (CRDs) installed in our Kubernetes clusters, and referenced by the Helm charts in this repository.

We use kubeconform to validate our Kubernetes manifests against schemas for those resources. This helps us ensure that our Helm charts are correct.

kubeconform runs in a GitHub Action as a pre-merge check and can also be run locally.

You can run the validation tests locally by installing kubeconform and running

mkdir helm-dist
for c in charts/*; do
  helm template "$(basename "$c")" "$c" --output-dir helm-dist
done

kubeconform -schema-location default \
-schema-location "https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json" \
-summary helm-dist

Team

GOV.UK Platform Engineering team looks after this repo. If you're inside GDS, you can find us in #govuk-platform-engineering.