Sealed Secrets with Bitnami Sealed Secrets Controller

November 2, 2023 · View on GitHub

Requeriments

gcloud CLI (only for Google Kubernetes Engine) - https://cloud.google.com/sdk/docs/install
kubectl CLI:

Installation: https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/
Config to access to GKE: https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-access-for-kubectl

Bitnami Sealed Secrets Controller and kubeseal CLI - https://github.com/bitnami-labs/sealed-secrets#linux
A Kubernetes cluster already working in Google Cloud. If you don't have one, all part related to create a secret and sealing are the same.

Once authenticated successfully, fetch the Kubernetes Cluster configuration using this command gcloud container clusters get-credentials CLUSTER_NAME --region=COMPUTE_REGION|--zone=COMPUTE_ZONE --project=PROJECT_ID:

$ gcloud container clusters get-credentials aragon-devops --zone europe-west6-a --project aragon-devops-319312
$ gcloud container clusters get-credentials aragon-staging --zone europe-west6-a --project aragon-staging
$ gcloud container clusters get-credentials aragon-prod --zone europe-west6-a --project aragon-prod

Now, you will be able to view the kubeconfig and the current context and so on:

$ kubectl config view
$ cat ~/.kube/config
$ kubectl config current-context
$ kubectl config get-context
$ kubectl cluster-info

If you execute any kubectl command, this will use the default context of last gcloud container clusters get-credentials <CLUSTER_NAME> command. For example, next command will show all namespaces in the aragon-prod cluster:

$ kubectl get namespaces

Setup a new connection to GKE cluster (optional)

We are going to follow this guide https://cloud.google.com/sdk/docs/configurations and you should have a proper credentials (email address) and the right permissions to install a Kubernetes Controller. In this case I'm going to use as example this email address chilcano@holisticsecurity.io.

$ gcloud config configurations create ara-prod

$ gcloud config configurations list
NAME      IS_ACTIVE  ACCOUNT                       PROJECT      COMPUTE_DEFAULT_ZONE  COMPUTE_DEFAULT_REGION
ara-prod  True
ara-stg   False
default   False      chilcano@holisticsecurity.io  aragon-prod

$ gcloud config set project aragon-prod
Updated property [core/project].

$ gcloud auth login
// or
$ gcloud config set account chilcano@holisticsecurity.io
Updated property [core/account].

$ gcloud config set compute/zone europe-west6-a 
Updated property [compute/zone].

$ gcloud config set compute/region europe-west6
Updated property [compute/region].

// List active config/project
$ gcloud config list

[compute]
region = europe-west6
zone = europe-west6-a
[core]
account = chilcano@holisticsecurity.io
disable_usage_reporting = True
project = aragon-prod

Fetch the credentials for the cluster:

$ gcloud container clusters get-credentials aragon-prod
Fetching cluster endpoint and auth data.
kubeconfig entry generated for aragon-prod.

$ kubectl get namespaces
NAME                     STATUS   AGE
blind-csp                Active   263d
blind-sms-csp            Active   183d
...
vocdoni-external-dns     Active   409d

Swithching, updating to other configuration and fetching kubectl config:

$ gcloud config configurations activate <project>
$ gcloud config set project aragon-staging
$ gcloud config set compute/zone europe-west6-a 
$ gcloud config set compute/region europe-west6

$ gcloud container clusters get-credentials aragon-staging
$ kubectl get namespaces

Note

A quick way to configure your connection to GKE is copying the connection configuration details from GKE. You should have a valid user and be authenticated.

Example: https://www.bitpoke.io/docs/app-for-wordpress/basic-usage/connect-to-cluster-and-access-k8s-pods/

$ gcloud container clusters get-credentials aragon-devops --zone europe-west6-a --project aragon-devops-319312

Once fetched the config, you will be able to run kubectl

$ kubectl get nodes
NAME                                            STATUS   ROLES    AGE   VERSION
gke-aragon-devops-n2-standard-2f168a49-02ih     Ready    <none>   18d   v1.24.9-gke.3200
gke-aragon-devops-n2-standard-2f168a49-qubg     Ready    <none>   18d   v1.24.9-gke.3200
gke-aragon-devops-smaller-nodes-63cd004d-4iaw   Ready    <none>   18d   v1.24.9-gke.3200
gke-aragon-devops-smaller-nodes-63cd004d-zgjm   Ready    <none>   18d   v1.24.9-gke.3200

Finally, check the kubeconfig generated in your local computer. You should view all cluster config you already connected to. As well, you can rename contexts, users, etc.

$ kubectl config view

Connecting with k9s

$ k9s --context aragon_devops_gke
$ k9s --context aragon_stg_gke
$ k9s --context aragon_prod_gke
$ k9s --context aragon_hetzner

4. Install kubeseal

# Get last version and remove first character to have the semver value. i.e. 0.24.2
$ KUBESEAL_VERSION=$(curl -s https://api.github.com/repos/bitnami-labs/sealed-secrets/releases/latest | jq -r -M '.tag_name' | cut -c 2-)
$ wget "https://github.com/bitnami-labs/sealed-secrets/releases/download/v${KUBESEAL_VERSION:?}/kubeseal-${KUBESEAL_VERSION:?}-linux-amd64.tar.gz"
$ tar -xvzf kubeseal-${KUBESEAL_VERSION:?}-linux-amd64.tar.gz kubeseal
$ sudo install -m 755 kubeseal /usr/local/bin/kubeseal

# Get version installed
$ kubeseal --version

kubeseal version: 0.24.2

5. Create a secret in yaml format

We are going to create a secret in yaml format.

bar is the secret value in plain text
YmFy is the secret value encoded in base64
foo is the secret key name
mysecret is the secret name what K8s will use
mynamespace is namespace that will contain the secret
Use -o json if you want generate a json file instead of yaml.

$ echo -n bar | kubectl create secret generic mysecret -n mynamespace --dry-run=client --from-file=foo=/dev/stdin -o yaml > mysecret.yaml

$ cat mysecret.yaml

apiVersion: v1
data:
  foo: YmFy
kind: Secret
metadata:
  creationTimestamp: null
  name: mysecret
  namespace: mynamespace

You are able to create a secret manifest file using your favorite text editor, if so, you could edit the entire yaml file from the editor and update only the secret with the base64 encoded value. In Linux generate you can encode and decode in base64 using these commands:

$ echo -n mysecretinplaintext | base64
bXlzZWNyZXRpbnBsYWludGV4dA==

$ echo bXlzZWNyZXRpbnBsYWludGV4dA== | base64 --decode
mysecretinplaintext

$ echo -n "generating longer super secret encoded in base64 in only one line" | base64 -w 0
Z2VuZXJhdGluZyBsb25nZXIgc3VwZXIgc2VjcmV0IGVuY29kZWQgaW4gYmFzZTY0IGluIG9ubHkgb25lIGxpbmU=

The -n flag means remove break line and -w 0 generate the base64 string in one single line.

Important:

Once generated a secret yaml file, you must to update the file with all annotations under metadata, namespace and the name of the key under data.
Once updated, when sealing with kubeseal the generated sealed secret yaml file only will be able to be deployed successfully in the already specified namespace using the original annotations and key name in data. Then, make sure using the right values while editing the secret, because once sealed, the sealed secret only will be able unsealed if these values have not been changed.

6. Create a yaml sealed-secret

$ kubeseal <mysecret.yaml >mysealedsecret.yaml

error: invalid configuration: no configuration has been provided, try setting KUBERNETES_MASTER environment variable

Once configured the K8s cluster to avoid the above error, run again the command:

$ kubeseal <mysecret.yaml -o yaml >mysealedsecret.yaml

$ cat mysealedsecret.yaml

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  creationTimestamp: null
  name: mysecret
  namespace: default
spec:
  encryptedData:
    foo: AgBb6dF9UUfkMXgesX9yTzgCH8it90Z........gzU=
  template:
    metadata:
      creationTimestamp: null
      name: mysecret
      namespace: default

7. Create a yaml sealed-secret in automated way

I've created a script that seal your K8s secret file using any K8s context available in your kubeconfig file.

The seal_k8s_secrets.sh bash script

#!/bin/bash

if [ $# -eq 2 ]; then
  arg_ctx="\$1"
  arg_secfile="\$2"
  sealed_secfile="${arg_secfile%.*}-sealed.yaml"

  ctx_values=$(kubectl config get-contexts --no-headers | grep $arg_ctx | awk {'print \$1"+"\$2'})
  if [[ "$ctx_values" != "" ]]; then
    if [[ -f "${arg_secfile}" ]]; then
      ctx_values=$(echo $ctx_values | sed "s/[ \t*+]//g")
      IFS="+"
      read -ra ctx_items <<< "$ctx_values"
      printf "> K8s context found: ${ctx_items[0]}\n"
      seal_cmd="kubeseal --context ${ctx_items[0]} --scope cluster-wide --format=yaml"
      eval "${seal_cmd} < ${arg_secfile} > ${sealed_secfile}"
      printf "> K8s sealed secret file created: ${sealed_secfile} \n\n"
      exit 0
    else
      printf "> Error: K8s secret file not found: '${arg_secfile}' \n\n"
      exit 1      
    fi
  else
    printf "> Error: K8s context not found: '${arg_ctx}' \n\n"
    exit 1
  fi 
else
  printf "> Error: Two arguments are needed.\n"
  printf "\tseal_k8s_secrets.sh <devops|staging|prod> <SecretFile>\n\n"
  exit 1
fi

Using the seal_k8s_secrets.sh bash script

We are going to seal this K8s secret file that contains a base64 encoded secret:

$ echo -n "generating longer super secret encoded in base64 in only one line" | base64 -w 0
Z2VuZXJhdGluZyBsb25nZXIgc3VwZXIgc2VjcmV0IGVuY29kZWQgaW4gYmFzZTY0IGluIG9ubHkgb25lIGxpbmU=

The K8s secret file is myk8ssecret.yaml and contains:

apiVersion: v1
data:
  key-longer-secret: Z2VuZXJhdGluZyBsb25nZXIgc3VwZXIgc2VjcmV0IGVuY29kZWQgaW4gYmFzZTY0IGluIG9ubHkgb25lIGxpbmU=
kind: Secret
metadata:
  creationTimestamp: null
  name: mysecret
  namespace: mynamespace

Checking the available K8s contexts configured:

$ kubectl config get-contexts --no-headers | awk {'print \$1" - "\$2'}

aragon_devops_gke - gke_aragon-devops-319312_europe-west6-a_aragon-devops
aragon_prod_gke - gke_aragon-prod_europe-west6-a_aragon-prod
aragon_stg_gke - gke_aragon-staging_europe-west6-a_aragon-staging

Now we are going to seal using the devops K8s cluster:

$ ./seal_k8s_secrets.sh devops myk8ssecret.yaml 

> K8s context found: aragon_devops_gke
> K8s sealed secret file created: myk8ssecret-sealed.yaml

And finally, we have a sealed secret file in myk8ssecret-sealed.yaml:

apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  annotations:
    sealedsecrets.bitnami.com/cluster-wide: "true"
  creationTimestamp: null
  name: mysecret
  namespace: mynamespace
spec:
  encryptedData:
    key-longer-secret: AgB0MBWCL2fVxgug1YpcA5m7Ual7eY/zwmKcsNT2epMCUl8xcwjq2sBx9sovF6MRaFhlKFOBR4URnUdVfCDZNm/l/p1O/nVb8E1Rbs01NhRWmYDCCI2RpKmIpuro2J+78KB97YOoMbuozH1BRbPY9a2R9MJYj3NweEa7q7rEGua4Fci1TGECt2/L+wbLnDiy2pgBJEd0Z7ljdFU3Py2wpcFIxPeqdHfACDZHzpu6zCh0HqiRJ8ejgHNh6Lg9qV4h+qLC2F/vw631ALgc06AiL0w2y225NbqdJhG87isE4Cxt5kbOD7ZRNL3uwi/B5j5rblU+rgTQGp6xD2FDc7HW+b38eNH+7tBu90LetUosLQMbhukuyH+/d1whMEkg0JG4z70ky/mWX96z51cmMezS+dvv3baxNUo8cFLEk1NFnD4dt9fFU/RUHeqMqHZiClrUevSnrldxRVwFFKstJvLFQBsyEpX3l0tY1T9A02LleHZ+2IOD/sV9eChh7HYHvmSAYi1t1qXXmbaJew/OecEW4lJks9ny0O2AAKMwkBk7H/jyD4WjLvBJEztQojMQ48egDn+3DN4IDIDUUld8WWRMUx9+6agx6gJydbV4zC4c4GyIQCecbkrxY97xkDYoTT3NMZZB7fhxWe6quta1sFfO+RuKE+6S46QZYSt6U8hsLPA3qkYlCW9wmxtps6Eo97vfQK+x5OApU4RRDIgb7jcurR6Mn+8qLTzikjwT/j7hJSZeNKdHtpMGsjwld9fNuin3OqKc8ltkqJAa5DEEcTFEvzdxUQ==
  template:
    metadata:
      annotations:
        sealedsecrets.bitnami.com/cluster-wide: "true"
      creationTimestamp: null
      name: mysecret
      namespace: mynamespace

Sealed Secrets with Bitnami Sealed Secrets Controller

Requeriments

Steps

1. Install gcloud

2. Install kubectl

3. Connecting to Kubernetes (GKE)

4. Install kubeseal

5. Create a secret in yaml format

6. Create a yaml sealed-secret

7. Create a yaml sealed-secret in automated way