ServiceAccount token hardening, the secure way

#What goes wrong

Every namespace has a ServiceAccount named default, and every pod that does not name another one runs as it. Unless told otherwise, the kubelet mounts a token for that account into the pod at /var/run/secrets/kubernetes.io/serviceaccount/token. Most applications never call the Kubernetes API, so the token is pure risk: anyone who gets code execution or a file read in the pod gets a credential.

The token is also longer-lived than it looks. The mounted volume asks for 3607 seconds. The API server recognizes that request from the admission plugin and, by default, issues a token valid for one year instead, so that old clients that never reload the file keep working. The kubelet still replaces the file before the requested hour is up (once the token is older than 80 percent of it), but each old token stays valid until it expires or the pod is deleted.

A token without RBAC permissions is still an identity. Anything that accepts Kubernetes ServiceAccount tokens (a secrets manager's Kubernetes login, a cloud workload identity federation, an internal service that calls TokenReview) accepts it, unless it checks the audience.

Older clusters add one more kind: token Secrets of type kubernetes.io/service-account-token. They never expire and never rotate.

#What the docs say

If this flag is enabled, admission injected tokens would be extended up to 1 year to prevent unexpected failure during transition, ignoring value of service-account-max-token-expiration.

Source: Kubernetes docs, kube-apiserver reference (--service-account-extend-token-expiration, default true)

To prevent Kubernetes from automatically injecting credentials for a specified ServiceAccount or the default ServiceAccount, set the automountServiceAccountToken field in your Pod specification to false.

Source: Kubernetes docs, Service Accounts

These tokens don't expire and don't rotate.

Source: Kubernetes docs, Service Accounts (Service Account Token Secrets)

The Service Accounts page calls the mounted token short-lived and automatically rotating. It is rotated. It is short-lived only if you turn the extension flag off, which is documented on a different page, in the command-line reference.

#The secure configuration

1. Turn off automount for the default ServiceAccount in every namespace, and for workloads that do not call the API.

# k8s-serviceaccounts.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
  namespace: app
automountServiceAccountToken: false   # pods that use "default" get no token
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: config-reader                 # one account per workload that needs the API
  namespace: app
automountServiceAccountToken: false   # tokens only where a pod asks for one explicitly

2. For a pod that must call the API, mount a projected token with a short expiry.

# k8s-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: config-reader
  namespace: app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: config-reader
  template:
    metadata:
      labels:
        app: config-reader
    spec:
      serviceAccountName: config-reader
      automountServiceAccountToken: false      # no default kube-api-access volume
      containers:
        - name: app
          image: ghcr.io/example/config-reader@sha256:0000000000000000000000000000000000000000000000000000000000000000
          env:
            - name: KUBE_TOKEN_FILE
              value: /var/run/secrets/tokens/kube/token
          volumeMounts:
            - name: kube-token
              mountPath: /var/run/secrets/tokens/kube
              readOnly: true
          securityContext:
            allowPrivilegeEscalation: false
            capabilities:
              drop: ["ALL"]
      securityContext:
        runAsNonRoot: true
        runAsUser: 10001
        seccompProfile:
          type: RuntimeDefault
      volumes:
        - name: kube-token
          projected:
            sources:
              - serviceAccountToken:
                  path: token
                  expirationSeconds: 600          # the minimum; the kubelet refreshes it at 80 percent
                  # No audience: it defaults to the API server's own. Tokens for any
                  # other system get a separate source with that system's audience.
              - configMap:
                  name: kube-root-ca.crt
                  items:
                    - key: ca.crt
                      path: ca.crt

The application must reread the token file; client libraries from the last few years do. Tokens for other systems (a secrets manager, a cloud provider) get their own projected token with that system's audience.

3. Stop the one-year extension on the API server. On Talos:

# talos-sa-tokens.yaml
# talosctl gen config ... --config-patch-control-plane @talos-sa-tokens.yaml
apiVersion: v1alpha1
kind: KubeAPIServerConfig
extraArgs:
  service-account-extend-token-expiration: "false"  # injected tokens live 1 hour, not 1 year
  service-account-max-token-expiration: "24h"       # cap on any TokenRequest

On kubeadm, set the same two flags in the apiServer.extraArgs of the cluster configuration. Before you switch, check the API server's serviceaccount_stale_tokens_total metric: it counts requests made with tokens that would already have expired without the extension. Fix those clients first.

4. Remove legacy token Secrets.

kubectl get secrets -A --field-selector type=kubernetes.io/service-account-token
# Review each one; delete those no system needs:
kubectl -n app delete secret build-robot-token

#Prove it

Run on a lab cluster (Kubernetes 1.34) with the manifests above. The Deployment's placeholder image was replaced by busybox:1.37 running sleep; every other line is as published. The Talos patch was not applied (the lab is not Talos). Real output:

1. A pod on the default account has no token. Before k8s-serviceaccounts.yaml:

kubectl -n app run probe --restart=Never --image=busybox:1.37 -- \
  ls /var/run/secrets/kubernetes.io/serviceaccount

ca.crt
namespace
token

After it:

ls: /var/run/secrets/kubernetes.io/serviceaccount: No such file or directory

2. The projected token lives 600 seconds, for the API server's audience:

kubectl -n app exec deploy/config-reader -- cat /var/run/secrets/tokens/kube/token \
  | jq -R 'split(".")[1] | gsub("-";"+") | gsub("_";"/") | @base64d | fromjson
           | {aud, lifetime_s: (.exp - .iat)}'

{
  "aud": [
    "https://kubernetes.default.svc.cluster.local"
  ],
  "lifetime_s": 600
}

The token sits only at the path you chose; the default mount is gone:

/var/run/secrets/tokens/kube:
ca.crt
token
ls: /var/run/secrets/kubernetes.io/serviceaccount: No such file or directory

On Talos, talosctl gen config sets the audience to the cluster endpoint instead. For a pod that still uses the default kube-api-access volume, the lifetime is 31536000 (365 days) while the legacy extension is on, and 3607 after the Talos patch.

3. No legacy tokens are left:

kubectl get secrets -A --field-selector type=kubernetes.io/service-account-token

No resources found

#Mistakes people make

#Trusting "short-lived" without decoding one

The default token file is rewritten every hour, which looks like rotation. Each old token is still accepted for a year unless the extension flag is off. Decode a token and check exp.

#Setting automount on the pod only

A pod spec wins over its ServiceAccount, but every Deployment, Job and CronJob then has to remember the field. Set it on the ServiceAccount, especially default, and set it on the pod as well for workloads that call the API.

#One token for every system

A token with the API server's default audience is accepted by anything that trusts your cluster's issuer and does not check the audience. Give each other system its own projected token with its own audience, and configure those systems to require it, so a token stolen for one cannot be replayed to another.

#Keeping a legacy token for the CI system

A token Secret created years ago for CI never expires. If it leaks, it works until someone deletes the Secret. Give CI short tokens from kubectl create token --duration 10m or workload identity federation.

#Switching the flag off without looking

Clients that load the token once at start and never reread it fail after an hour when the extension is off. Check the stale-token metric and fix those clients first.

#Checklist

• Every namespace's default ServiceAccount has automountServiceAccountToken: false.
• Each workload that calls the API has its own ServiceAccount with minimal RBAC.
• Those workloads mount a projected token with expirationSeconds: 600; tokens for any system other than the API server carry that system's audience.
• service-account-extend-token-expiration is false on every API server.
• service-account-max-token-expiration caps TokenRequests.
• serviceaccount_stale_tokens_total was checked before the switch.
• No Secrets of type kubernetes.io/service-account-token exist without a documented reason.
• Systems that accept Kubernetes tokens check the audience.