Monitoring DNSSEC signature expiry, the secure way

#What goes wrong

Every DNSSEC signature (RRSIG) has an expiration time. The signer must replace it before that time, and every secondary must receive the new one. If either step stops, the old signatures run out and validating resolvers treat the answers as bogus: SERVFAIL for every name in the zone.

The signer stopping is easy to notice. The quiet failure is a secondary that stopped receiving transfers. It keeps answering with the last copy it has, signatures included, until they expire. Resolvers that happen to ask that server fail; resolvers that ask the others do not. From the outside it looks random.

Most monitoring misses this because it checks through a resolver, or checks "is there an RRSIG", or checks only one server. None of those show a date.

#What the docs say

We also suggest that operators of name servers that supply secondary services develop systems to identify upcoming signature expirations in zones they slave and take appropriate action where such an event is detected.

Source: RFC 6781, Section 4.4.1

We suggest that the SOA expiration timer be approximately one third or a quarter of the signature validity period.

Source: RFC 6781, Section 4.4.1

A period (in seconds) how long at least before a signature expiration the signature will be refreshed, in order to prevent expired RRSIGs on secondary servers or resolvers' caches.

Source: Knot DNS 3.5 reference, rrsig-refresh

When a RRSIG expires on a secondary server, the whole zone is expired (permanent SERVFAIL until resolved).

Source: Knot DNS 3.5 reference, dnssec-validation

The RFC asks for monitoring but gives no tool or thresholds. Knot's docs explain when the signer refreshes, which is exactly what the thresholds should be built from, but Knot has no built-in alert for a secondary elsewhere that stopped updating.

#The secure configuration

The check, check-rrsig-expiry. It asks each server directly, with DO set and no recursion, and exits with the usual monitoring codes:

#!/bin/sh
# check-rrsig-expiry.sh ZONE WARN_SECONDS CRIT_SECONDS SERVER [SERVER...]
# Asks EVERY authoritative server directly (no resolver, no cache) for the SOA
# and DNSKEY RRsets with DO set, and reports the earliest RRSIG expiration.
# Exit codes follow the Nagios/Icinga convention: 0 OK, 1 WARNING, 2 CRITICAL.
zone=$1; warn=$2; crit=$3; shift 3
now=$(date -u +%s); worst=0
for server in "$@"; do
  for type in SOA DNSKEY; do
    exp=$(kdig @"$server" "$zone" "$type" +dnssec +norecurse +noall +answer +timeout=2 +retry=1 \
          | awk -v t="$type" '$4=="RRSIG" && $5==t {print $9}' | sort | head -1)
    if [ -z "$exp" ]; then
      echo "CRITICAL $server $zone $type: no RRSIG in the answer"; worst=2; continue
    fi
    left=$(( $(date -u -d "$(echo "$exp" | sed -E 's/(....)(..)(..)(..)(..)(..)/\1-\2-\3 \4:\5:\6/')" +%s) - now ))
    if   [ "$left" -le 0 ];      then state=CRITICAL; code=2; msg="EXPIRED $(( -left ))s ago"
    elif [ "$left" -lt "$crit" ]; then state=CRITICAL; code=2; msg="expires in ${left}s"
    elif [ "$left" -lt "$warn" ]; then state=WARNING;  code=1; msg="expires in ${left}s"
    else state=OK; code=0; msg="expires in $(( left / 86400 ))d $(( left % 86400 / 3600 ))h"; fi
    echo "$state $server $zone $type RRSIG $exp: $msg"
    [ "$code" -gt "$worst" ] && worst=$code
  done
  serial=$(kdig @"$server" "$zone" SOA +norecurse +short +timeout=2 +retry=1 | awk '{print $3}')
  echo "INFO $server $zone SOA serial ${serial:-none}"
done
exit $worst

Thresholds come from the signer's policy. With the Knot policy from the automatic signing page (rrsig-lifetime: 14d, rrsig-refresh: 7d), a healthy server never shows less than about 7 days left:

# Every authoritative address, including each anycast node's unicast address.
# Warn at 6 days (a refresh is a day late), critical at 3 days.
check-rrsig-expiry example.com 518400 259200 \
  198.51.100.20 198.51.100.21 2001:db8::20 2001:db8::21

Make a stale secondary stop answering before its signatures expire. In the zone, set SOA expire to a quarter of rrsig-lifetime:

; refresh 1h, retry 10m, expire 3.5d (a quarter of 14d), minimum 5m
@  SOA ns1.example.com. hostmaster.example.com. 2026092401 3600 600 302400 300

On Knot secondaries, validate what arrives, so a broken transfer is rejected instead of served:

zone:
  - domain: example.com
    master: primary
    dnssec-validation: on

#Prove it

The test (secure-tests/monitoring-dnssec-signature-expiry/run.sh) runs two Knot servers. 127.0.0.1 is a healthy signer. 127.0.0.2 serves a copy signed 14 days ago with the same keys (made with kzonesign -t), so its signatures expire 150 seconds into the test: a secondary that stopped updating two weeks ago.

The naive check sees nothing wrong:

kdig @127.0.0.1 www.example.com A +dnssec +norecurse +noall +answer | grep -c RRSIG
kdig @127.0.0.2 www.example.com A +dnssec +norecurse +noall +answer | grep -c RRSIG

1
1

The expiry check, with lab thresholds (warn 300 s, critical 120 s):

check-rrsig-expiry example.com 300 120 127.0.0.1 127.0.0.2; echo "exit $?"

OK 127.0.0.1 example.com SOA RRSIG 20261008210512: expires in 13d 23h
OK 127.0.0.1 example.com DNSKEY RRSIG 20261008210512: expires in 13d 23h
INFO 127.0.0.1 example.com SOA serial 2026092402
WARNING 127.0.0.2 example.com SOA RRSIG 20260924210742: expires in 145s
WARNING 127.0.0.2 example.com DNSKEY RRSIG 20260924210742: expires in 145s
INFO 127.0.0.2 example.com SOA serial 2026092401
exit 1

The serials differ too, which points at a transfer problem. On the signer, Knot shows when it will re-sign next:

knotc zone-status example.com

[example.com.] role: master | serial: 2026092402 | re-sign: +6D23h59m53s

160 seconds later:

OK 127.0.0.1 example.com SOA RRSIG 20261008210512: expires in 13d 23h
OK 127.0.0.1 example.com DNSKEY RRSIG 20261008210512: expires in 13d 23h
INFO 127.0.0.1 example.com SOA serial 2026092402
CRITICAL 127.0.0.2 example.com SOA RRSIG 20260924210742: EXPIRED 18s ago
CRITICAL 127.0.0.2 example.com DNSKEY RRSIG 20260924210742: EXPIRED 18s ago
INFO 127.0.0.2 example.com SOA serial 2026092401
exit 2

A validator confirms what the check says:

delv @127.0.0.1 -a anchor.conf +root=example.com www.example.com A
delv @127.0.0.2 -a anchor.conf +root=example.com www.example.com A

; fully validated

;; validating example.com/DNSKEY: verify failed due to bad signature (keyid=48655): RRSIG has expired
;; validating example.com/DNSKEY: no valid signature found (DS)

#Mistakes people make

#Checking through a resolver

A resolver answers from cache and may ask any one of your servers. The check passes or fails at random. Query each authoritative address with +norecurse.

#Checking only that an RRSIG exists

An expired RRSIG is still an RRSIG. The naive check in the test passed for a server whose signatures were two minutes from expiry. Parse the date.

#Checking one name

Knot refreshes signatures in batches, so different names expire at slightly different times. The SOA and DNSKEY signatures are the ones that break the whole zone. Check both, and add a few important names if you like.

#SOA expire longer than the signature validity

The common SOA expire of two weeks equals Knot's default 14-day signature lifetime. A stale secondary then serves bogus data instead of expiring the zone. Set SOA expire to a quarter or a third of rrsig-lifetime.

#Anycast hides the stale node

An anycast address reaches only the node nearest to your monitor. Check each node's unicast address, or run the check from a probe in each region.

#Checklist

• List every authoritative address, including each anycast node's unicast address.
• Run the expiry check against each address with +norecurse every 5 to 15 minutes.
• Check the SOA and DNSKEY RRSIGs, not only a sample record.
• Warn when the time left is below rrsig-refresh minus one day.
• Go critical at 3 days or less.
• Alert when SOA serials differ between servers for longer than one refresh interval.
• Set SOA expire to one quarter to one third of rrsig-lifetime.
• Turn on dnssec-validation: on for secondary zones in Knot.
• Page a human for critical, not only a ticket.