Kubernetes: Workload Identity

Nov 1, 2021

The usual approach to consume cloud services from Kubernetes is to create a service account with the appropriate permissions, generate a key, store that key in some secure location, create a secret with that key during deployment, and mount it into a pod for the workload to consume. The biggest downsides in this scenario, are twofold:

Where do we store the key securely?
How to implement a key rotation policy?

GKE’s workload identity solves that problem by avoiding the use of a service account key in the first place. Instead, it allows us to bind a GCP service account and a Kubernetes service account which gives us access to the cloud service we want to consume.

Let’s first verify that Workload Identity is enabled:

gcloud container clusters describe my-cluster --format="value(workloadIdentityConfig.workloadPool)" --region=us-central1
gcloud container node-pools describe my-pool --cluster=my-cluster --format="value(config.workloadMetadataConfig.mode)" --region=us-central1

Make sure we have both service accounts:

gcloud iam service-accounts describe my-svc-account@my-project.iam.gserviceaccount.com
kubectl get serviceaccounts/my-k8s-svc-account

The Kubernetes service account should be annotated:

kubectl describe serviceaccounts/my-k8s-svc-account

Annotations:         iam.gke.io/gcp-service-account: my-svc-account@my-project.iam.gserviceaccount.com

A binding between the two service accounts should exist with the workloadIdentityUser role

gcloud iam service-accounts get-iam-policy my-svc-account@my-project.iam.gserviceaccount.com

bindings:
- members:
  - serviceAccount:my-project.svc.id.goog[my-ns/my-k8s-svc-account]
  role: roles/iam.workloadIdentityUser

Once the workload is deployed, you can exec into a pod and confirm that you are automatically authenticated to GCP:

gcloud auth list

One important caveat. The GKE metadata server connection can have a few seconds lag. This can be solved by either retrying in the workload or using an init container.

Related tags:

Site menu