Talos machine configs in git with age, the secure way

#What goes wrong

A Talos machine configuration is not a settings file. The control plane config contains the private keys of the Talos API CA, the Kubernetes CA, the etcd CA and the aggregator CA, the key that signs every service account token, the key that encrypts Secrets in etcd, and the cluster join tokens. The talosconfig next to it is an admin client certificate valid for a year.

GitOps habits put all of it in git. From there it spreads to every clone, every fork, every CI runner cache, and every backup of the forge. Git never forgets: deleting the file in a later commit leaves it in history. Anyone who reads it once can sign their own admin certificate for as long as those CAs live, which is ten years by default.

Encrypting the whole config file is the second trap. You can no longer review a change, because the diff is ciphertext. People start decrypting to edit, and the plaintext copy ends up committed anyway.

#What the docs say

Limit who has access to machine configuration files, as they contain cluster CA keys and API credentials.

Source: Sidero Labs docs, Talos Security Checklist

Discard the generated configs. Do not commit them, instead, regenerate them when needed.

Source: Sidero Labs docs, Reproducible Machine Configuration

The secrets bundle is a file that contains all the cryptographic keys, certificates, and tokens needed to secure your Talos Linux cluster.

Source: Sidero Labs docs, Production Notes

The Talos docs tell you to keep secrets.yaml and regenerate, but never say where secrets.yaml lives or how to protect it. That file is now the one thing an attacker needs, so it gets the encryption, and everything else stays readable.

#The secure configuration

Layout of the repository:

.sops.yaml                 # which files are encrypted, and to whom
talos/secrets.sops.yaml    # talosctl gen secrets, encrypted with sops + age
talos/patches/common.yaml  # plain text: your intent, reviewable in a diff
talos-gen.sh               # regenerates configs into a temp dir
talos-secrets-guard.sh     # pre-commit and CI guard

1. Encrypt the secrets bundle to two age recipients. One key is the operator's. The second is a recovery key whose private half is printed or stored offline, so losing a laptop does not lose the cluster.

# .sops.yaml
creation_rules:
  - path_regex: talos/secrets\.sops\.yaml$
    # Operator key, then the offline recovery key. Public keys only.
    age: >-
      age1sewg3wr483s2ud6p0s9e8vyc8dsn96pkmzwuv4573kz5w805hpwqyf0he0,
      age1cryqz949vdtvd9fvrp6akynkzutqw6dt2kw8wkdunaudpvdlzehsvjdk9v

umask 077
age-keygen -o ~/.config/sops/age/keys.txt        # operator key, never committed
talosctl gen secrets -o - \
  | sops encrypt --filename-override talos/secrets.sops.yaml \
      --input-type yaml --output-type yaml /dev/stdin > talos/secrets.sops.yaml

The plaintext bundle goes straight from talosctl into sops through a pipe. It never touches the disk.

2. Keep patches in plain text. They carry no keys, so they stay reviewable:

# common.yaml
# talos/patches/common.yaml: intent only, no secrets.
apiVersion: v1alpha1
kind: SecurityProfileConfig
workloadIsolation: true

3. Regenerate configs when you need them, into a private temp directory:

# talos-gen.sh
# Regenerates machine configs from the encrypted bundle and the patches.
# Output lives in a private temp directory that is deleted on exit.
set -euo pipefail
umask 077
out="$(mktemp -d)"
trap 'rm -rf "$out"' EXIT
# sops decrypts to a named pipe and passes its path as {}; no plaintext file is written.
sops exec-file talos/secrets.sops.yaml \
  "talosctl gen config demo https://10.10.1.10:6443 --with-secrets {} \
     --talos-version v1.14 --kubernetes-version 1.37.0 \
     --config-patch @talos/patches/common.yaml \
     --output-types controlplane,worker --output-dir $out"
talosctl validate --config "$out/controlplane.yaml" --mode cloud --strict
talosctl validate --config "$out/worker.yaml" --mode cloud --strict
# Apply from here while the directory exists, for example:
# talosctl -n 10.10.1.11 apply-config -f "$out/controlplane.yaml"

Pin --talos-version so a regeneration next year produces the same config (see Talos upgrades that keep your hardening).

4. Refuse plaintext at commit time and in CI:

# talos-secrets-guard.sh
# Usage: bash talos-secrets-guard.sh $(git diff --cached --name-only)
# Fails if key material or an unencrypted Talos file is about to be committed.
set -euo pipefail
status=0
for f in "$@"; do
  [ -f "$f" ] || continue
  # "LS0tLS1CRUdJT" is base64 for "-----BEGI": how Talos stores PEM keys in YAML.
  if grep -qE 'LS0tLS1CRUdJT|-----BEGIN [A-Z ]*PRIVATE KEY-----' "$f"; then
    echo "FAIL: $f contains key material in plain text" >&2
    status=1
  fi
  case "$f" in
    *secrets*.yaml | *talosconfig* | controlplane*.yaml | worker*.yaml)
      if ! grep -q '^sops:' "$f"; then
        echo "FAIL: $f is a Talos secrets file and is not sops-encrypted" >&2
        status=1
      fi
      ;;
  esac
done
exit "$status"

# .gitignore
controlplane.yaml
worker.yaml
talosconfig
secrets.yaml

Run the guard as a pre-commit hook and again in CI on every pushed commit, because hooks are optional on the client side.

#Prove it

Real output from the test in secure-tests/talos-machine-configs-git-age/, run in a throwaway container. Ciphertext, keys and temp paths change on every run.

1. The committed file is ciphertext:

sed -n '1,4p' talos/secrets.sops.yaml | cut -c1-72
grep -c 'ENC\[AES256_GCM' talos/secrets.sops.yaml
grep -c 'LS0tLS1CRUdJT' talos/secrets.sops.yaml

cluster:
    id: ENC[AES256_GCM,data:1diVVfCrcY9z56vrasz+DZ1mxll1yvROUrAvxtdNBCVb
    secret: ENC[AES256_GCM,data:VUj51kNokuqytZTKKH2DXKNf4EndwjJraOtTeh3A
secrets:
15
0

Fifteen encrypted values, and no base64 PEM header left in the file.

2. Without a key, nobody reads it. An empty home directory stands in for a machine that never had the key:

HOME=/tmp/nokey sops decrypt talos/secrets.sops.yaml

Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  age1sewg3wr483s2ud6p0s9e8vyc8dsn96pkmzwuv4573kz5w805hpwqyf0he0: FAILED
    - | failed to load age identities. Did not find keys in
      | locations 'SOPS_AGE_SSH_PRIVATE_KEY_FILE',
      | 'SOPS_AGE_SSH_PRIVATE_KEY_CMD',
      | '/tmp/nokey/.ssh/id_ed25519', '/tmp/nokey/.ssh/id_rsa',
      | 'SOPS_AGE_KEY', 'SOPS_AGE_KEY_FILE', 'SOPS_AGE_KEY_CMD', and
      | '/tmp/nokey/.config/sops/age/keys.txt'.
  
  age1cryqz949vdtvd9fvrp6akynkzutqw6dt2kw8wkdunaudpvdlzehsvjdk9v: FAILED
    - | failed to load age identities. Did not find keys in
      | locations 'SOPS_AGE_SSH_PRIVATE_KEY_FILE',
      | 'SOPS_AGE_SSH_PRIVATE_KEY_CMD',
      | '/tmp/nokey/.ssh/id_ed25519', '/tmp/nokey/.ssh/id_rsa',
      | 'SOPS_AGE_KEY', 'SOPS_AGE_KEY_FILE', 'SOPS_AGE_KEY_CMD', and
      | '/tmp/nokey/.config/sops/age/keys.txt'.

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

3. The recovery key alone is enough:

HOME=/tmp/nokey SOPS_AGE_KEY_FILE=recovery-key.txt \
  sops decrypt talos/secrets.sops.yaml | grep -c 'LS0tLS1CRUdJT'

9

Nine PEM blocks (certificates and keys) come back with only the offline key.

4. The guard stops a plaintext bundle:

talosctl gen secrets -o secrets.yaml
bash talos-secrets-guard.sh secrets.yaml talos/secrets.sops.yaml; echo "exit $?"
bash talos-secrets-guard.sh talos/secrets.sops.yaml talos/patches/common.yaml; echo "exit $?"

FAIL: secrets.yaml contains key material in plain text
FAIL: secrets.yaml is a Talos secrets file and is not sops-encrypted
exit 1
exit 0

5. Configs regenerate from the encrypted bundle and validate:

bash talos-gen.sh
ls /tmp

generating PKI and tokens
Created /tmp/tmp.X3LVjs5Dji/controlplane.yaml
Created /tmp/tmp.X3LVjs5Dji/worker.yaml
/tmp/tmp.X3LVjs5Dji/controlplane.yaml is valid for cloud mode
/tmp/tmp.X3LVjs5Dji/worker.yaml is valid for cloud mode
nokey

talosctl prints "generating PKI and tokens" even with --with-secrets; the PKI came from the decrypted bundle. After the script exits, only the empty nokey directory from step 2 is left in /tmp.

#Mistakes people make

#Encrypting the whole machine config

A fully encrypted controlplane.yaml cannot be reviewed, so people decrypt it to edit, and the decrypted copy is one git add . away from history. Encrypt the secrets bundle; keep intent in plain patches.

#One recipient

If the only age key lives on one laptop, a lost laptop means a cluster you can no longer reconfigure. Add an offline recovery recipient and test that it decrypts, as in step 3.

#Deleting the file and calling it fixed

A secret that reached git is in the history, in every clone and in the forge's backups. Rotate it: new CAs with talosctl rotate-ca, and plan for the etcd CA, the service account key and the Secrets encryption key, which rotate-ca does not replace.

#Writing the decrypted bundle to disk

sops decrypt secrets.sops.yaml > secrets.yaml leaves a plaintext file for editors, backup tools and the next git add. Use sops exec-file, which hands talosctl a named pipe.

#Testing the lock while holding the key

With sops 3.13.3, SOPS_AGE_KEY_FILE=/dev/null sops decrypt still decrypted the file in the test, because sops also reads the default keys file under ~/.config/sops/age/. To check that a machine without the key is refused, test from an empty HOME or another user, as in step 2.

#Trusting the hook alone

Pre-commit hooks run only where someone installed them. Run the same guard in CI on every push and fail the pipeline.

#Checklist

• talos/secrets.sops.yaml is the only file with Talos secrets in git, and it is sops-encrypted.
• .sops.yaml lists at least two age recipients, one of them offline.
• The recovery key was tested and decrypts the bundle.
• Generated controlplane.yaml, worker.yaml and talosconfig are in .gitignore and never committed.
• Configs are regenerated with sops exec-file into a temp directory that is deleted after apply.
• --talos-version is pinned in the generation script.
• The plaintext guard runs as a pre-commit hook and in CI.
• If plaintext secrets ever reached git, the CAs and keys were rotated.