An offline internal CA with cert-manager, the secure way

#What goes wrong

The cert-manager bootstrap example creates a self-signed root inside the cluster and uses it directly as the CA issuer. The root key is now a Kubernetes Secret. It is in etcd, in every etcd backup, and readable by anyone who can read Secrets in the cert-manager namespace.

That root is trusted by every workload and often by staff laptops. With the key, an attacker can sign a certificate for any name, including public ones, and your machines will accept it.

The root has no expiry plan. Ten years later, it expires on a Sunday.

Leaves are issued with a longer life than the CA that signs them. They work until the CA expires, then all fail at once.

#What the docs say

CA issuers are generally for cert-manager demos or for advanced users with experience and tooling for running a PKI.

Source: cert-manager docs, CA issuer

CA issuers will issue leaf certificates which outlive the CA

Source: cert-manager docs, CA issuer: important information

Other constraints - such as name constraints or the CA "max path length" - are not validated at the time of issuance

Source: cert-manager docs, CA issuer: important information

The docs say what cert-manager does not check. They do not say that those same constraints are what limit the damage when the CA key leaks. Clients check them at verification time, as the test below shows.

#The secure configuration

On an offline machine (no network, full-disk encryption, stored in a safe), create the root and a constrained intermediate. The root passphrase is read from a file, never from the command line.

#!/bin/sh
# offline-ca.sh: offline root and a constrained intermediate.
# Only intermediate.key and intermediate-chain.crt leave this machine.
set -eu
cd "${1:-.}"
umask 077
[ -f root.pass ] || { echo "put the root passphrase in root.pass first" >&2; exit 1; }

cat > root.cnf <<'CNF'
[req]
distinguished_name = dn
prompt = no
[dn]
CN = Example Internal Root CA 2026
O = Example
[v3_root]
basicConstraints = critical, CA:true
keyUsage = critical, keyCertSign, cRLSign
subjectKeyIdentifier = hash
[v3_intermediate]
# pathlen:0: the intermediate cannot create another CA.
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, keyCertSign, cRLSign
# Only DNS names under internal.example.com verify. A DNS-only constraint
# does not limit other name types, such as IP addresses (see Mistakes below).
nameConstraints = critical, permitted;DNS:internal.example.com
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always
CNF

# Root: P-384, 10 years, key encrypted with AES-256. It never leaves this machine.
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-384 \
  -aes-256-cbc -pass file:root.pass -out root.key
openssl req -x509 -new -key root.key -passin file:root.pass -config root.cnf \
  -extensions v3_root -sha384 -days 3650 -out root.crt

# Intermediate: one year.
openssl genpkey -algorithm EC -pkeyopt ec_paramgen_curve:P-384 -out intermediate.key
openssl req -new -key intermediate.key -subj "/CN=Example Cluster Issuing CA 2026/O=Example" -out intermediate.csr
openssl x509 -req -in intermediate.csr -CA root.crt -CAkey root.key -passin file:root.pass \
  -extfile root.cnf -extensions v3_intermediate -sha384 -days 365 -set_serial 0x$(openssl rand -hex 16) \
  -out intermediate.crt

# The chain cert-manager needs: intermediate first, then root.
cat intermediate.crt root.crt > intermediate-chain.crt

Load the intermediate into the cluster resource namespace, then shred the local copy of intermediate.key:

kubectl -n cert-manager create secret tls internal-issuing-ca \
  --cert=intermediate-chain.crt --key=intermediate.key

The issuer and a certificate:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: internal-ca
spec:
  ca:
    secretName: internal-issuing-ca    # in the cert-manager namespace
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: app
  namespace: team-a
spec:
  secretName: app-tls
  duration: 2160h            # 90 days
  renewBefore: 720h          # renew with 30 days left
  commonName: app.team-a.internal.example.com
  dnsNames:
    - app.team-a.internal.example.com
  privateKey:
    algorithm: ECDSA
    size: 256
    rotationPolicy: Always   # new key on every renewal (the default since v1.18)
  usages:
    - server auth
    - digital signature
  issuerRef:
    group: cert-manager.io
    kind: ClusterIssuer
    name: internal-ca

Distribute root.crt (never a key) to workloads with trust-manager or your image build. Restrict get on Secrets in the cert-manager namespace to the cert-manager controller.

#Prove it

From secure-tests/offline-internal-ca-cert-manager/, run in an alpine:3.22 container. Every file is private, and the root key is encrypted:

ls -l /ca
head -1 root.key

-rw------- intermediate-chain.crt
-rw------- intermediate.crt
-rw------- intermediate.csr
-rw------- intermediate.key
-rw------- root.cnf
-rw------- root.crt
-rw------- root.key
-rw------- root.pass
-----BEGIN ENCRYPTED PRIVATE KEY-----

The intermediate carries its limits:

openssl x509 -in intermediate.crt -noout -ext basicConstraints,keyUsage,nameConstraints
openssl verify -CAfile root.crt intermediate.crt

X509v3 Basic Constraints: critical
    CA:TRUE, pathlen:0
X509v3 Key Usage: critical
    Certificate Sign, CRL Sign
X509v3 Name Constraints: critical
    Permitted:
      DNS:internal.example.com
intermediate.crt: OK

The test then signs leaves with the intermediate key, the same thing cert-manager does. A name inside the zone verifies:

openssl verify -CAfile root.crt -untrusted intermediate.crt app.crt

app.crt: OK

Now play the attacker who copied the Secret. A certificate for a name outside the zone:

CN=login.bank.example.net
error 47 at 0 depth lookup: permitted subtree violation
error evil.crt: verification failed

A new CA signed by the intermediate, and a leaf from that CA:

CN=Example Cluster Issuing CA 2026, O=Example
error 25 at 2 depth lookup: path length constraint exceeded
error x.crt: verification failed

A 400-day leaf from a 365-day intermediate. It verifies today and fails once the intermediate expires:

openssl verify -CAfile root.crt -untrusted intermediate.crt long.crt
openssl verify -attime <in 380 days> -CAfile root.crt -untrusted intermediate.crt long.crt

leaf notAfter:         Oct 29 20:49:48 2027 GMT
intermediate notAfter: Sep 24 20:49:47 2027 GMT
long.crt: OK
CN=Example Cluster Issuing CA 2026, O=Example
error 10 at 1 depth lookup: certificate has expired
error long.crt: verification failed

The manifests validate against cert-manager v1.21.2's CRDs:

kubeconform -strict -summary -kubernetes-version 1.34.0 \
  -schema-location default -schema-location 'schemas/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' cert-manager.yaml

Summary: 2 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 0, Skipped: 0

#In a cluster

The script's output on a lab cluster with cert-manager v1.21.2, the intermediate loaded as above, and the manifests applied:

$ kubectl get clusterissuer internal-ca -o wide
NAME          READY   STATUS                AGE
internal-ca   True    Signing CA verified   11s
$ kubectl -n team-a get certificate app
NAME   READY   SECRET    AGE
app    True    app-tls   11s
$ kubectl get secret app-tls -n team-a -o jsonpath='{.data.tls\.crt}' | base64 -d | openssl x509 -noout -issuer -enddate -ext subjectAltName
issuer=CN = Example Cluster Issuing CA 2026, O = Example
notAfter=Dec 24 03:10:24 2026 GMT
X509v3 Subject Alternative Name:
    DNS:app.team-a.internal.example.com
$ openssl verify -CAfile root.crt -untrusted intermediate.crt app.crt
app.crt: OK

ca.crt in the Secret is the root (subject=CN = Example Internal Root CA 2026), the last certificate of the chain you loaded.

cert-manager does not check names against the CA's constraints. Two more Certificates through the same issuer, one for a name outside the zone and one for 400 days:

$ kubectl -n team-a get certificate evil long
NAME   READY   SECRET     AGE
evil   True    evil-tls   10s
long   True    long-tls   10s
== evil: notAfter=Dec 24 03:10:55 2026 GMT X509v3 Subject Alternative Name: critical DNS:login.bank.example.net
error 47 at 0 depth lookup: permitted subtree violation
error evil.crt: verification failed
== long: notAfter=Oct 30 03:10:55 2027 GMT
long.crt: OK
intermediate: notAfter=Sep 25 03:10:23 2027 GMT

Both were signed. The name constraint is what makes evil useless, not cert-manager. And long outlives its intermediate by a month, which the offline test above shows turning into a verification failure.

#Mistakes people make

#Expecting cert-manager to refuse a name

The CA issuer signs whatever a Certificate or CertificateRequest asks for. Anyone who can create one in a namespace that uses the ClusterIssuer can get any name signed. The intermediate's name constraint stops it from verifying; to stop the request itself, restrict who may create CertificateRequests for this issuer, or add cert-manager's approver-policy.

#The root as the issuer

If the root key is in a Secret, it is online. Anything it signed can be forged forever. The root signs intermediates, a few times a year, from a machine with no network.

#An intermediate with no constraints

Without a name constraint, a stolen intermediate key signs any name your clients trust. Without pathlen:0, it signs new CAs. cert-manager does not add or check these; your offline script must.

#A DNS constraint and nothing for IP addresses

RFC 5280 says name constraints apply only to the name types they list: "If no name of the type is in the certificate, the certificate is acceptable." A stolen intermediate with only permitted;DNS: can still sign a leaf whose only name is an IP address, and it verifies. If clients connect by IP, add excluded;IP:0.0.0.0/0.0.0.0, excluded;IP:::/:: to nameConstraints, or permit only your internal ranges.

#Name constraints that are not critical

Mark nameConstraints critical so a client that does not understand it rejects the chain instead of ignoring it. RFC 5280 requires it: "Conforming CAs MUST mark this extension as critical."

#Leaves that outlive the intermediate

cert-manager does not compare lifetimes. Keep leaves at 90 days and rotate the intermediate while it has more than 120 days left.

#Rotating the intermediate and waiting for renewals

Updating the CA Secret does not re-issue existing leaves. After a rotation, trigger renewal of every Certificate from this issuer, for example with cmctl renew.

#The root key passphrase on the command line

-passout pass:... puts it in shell history and in ps. Use -pass file: (see never put a secret on the command line).

#Checklist

• Generate the root key on an offline machine and encrypt it.
• Keep the root key and its passphrase in separate places.
• Sign intermediates with pathlen:0 and critical nameConstraints for your internal zones.
• Exclude or limit IP addresses in nameConstraints too, not only DNS names.
• Give the intermediate a one-year lifetime.
• Load only the intermediate key and chain into the cert-manager namespace.
• Limit get secrets in the cert-manager namespace to the controller.
• Set Certificate duration to 90 days or less and rotationPolicy: Always.
• Rotate the intermediate while it has more than 120 days left.
• Renew all leaves after each intermediate rotation.
• Distribute root.crt only; never ship a key with a trust bundle.
• Put the root and intermediate expiry dates in monitoring.