Identity and access

An insider kill switch, the secure way

HR calls at 16:55: access ends now, quietly. You delete the account, feel efficient, and the eight-hour token they logged in with this morning keeps reading the database until well after dinner.

The short answer

Prepare the switch before you need it: a small, pre-issued kill-switch credential and a script per system. In OpenBao, disable the person's entity, which stops every token they hold on the next request, including child tokens, then delete the login. Expire their tailnet devices and IdP sessions the same minute, and verify each one.

Updated Houssam Hammoudi, CTOTested with OpenBao 2.7.0

On this page
  1. What goes wrong
  2. What the docs say
  3. The secure configuration
  4. Prove it
  5. Mistakes people make
  6. Checklist

What goes wrong

The account gets deleted, which stops new logins. Every token, session and device key issued before that keeps working until it expires on its own.

The person saw this coming and made extra tokens: a child token from their own, a PAT in the IdP, a second device on the tailnet. Deleting the account touches none of them.

The switch depends on one person with root. That person is on holiday, or is the insider.

Nobody checks afterwards. The ticket says "access removed" because a command returned success, not because anyone tried the old credentials.

What the docs say

Disabled entities' associated tokens cannot be used, but are not revoked.

Source: OpenBao API docs, Identity: entity

When a parent token is revoked, all of its child tokens -- and all of their leases -- are revoked as well.

Source: OpenBao docs, Tokens

The docs explain each mechanism. They do not say that deleting a userpass (or any auth method) user leaves that user's existing tokens valid. The test shows it does, and shows that disabling the entity closes the gap at once while keeping the tokens for the investigation.

The secure configuration

A kill-switch policy, issued ahead of time to the security on-call as a periodic token (or a login of its own), so no root token is needed:

hcl
# kill-switch: find a person's entity, disable it, delete their login.
path "identity/entity-alias/id"   { capabilities = ["list"] }
path "identity/entity-alias/id/*" { capabilities = ["read"] }
path "identity/entity/id/*"       { capabilities = ["update"] }
path "auth/userpass/users/*"      { capabilities = ["delete"] }

The script. Disable first (instant, reversible, keeps evidence), then delete the login:

bash
#!/bin/sh
# kill-switch.sh USERNAME: cut one person out of OpenBao now.
# 1. disable their entity: every token they hold, and every child token,
#    stops working on the next request;
# 2. then clean up: delete the login, so no new token can be issued.
# Needs BAO_ADDR and a BAO_TOKEN allowed to write identity/entity and
# auth/userpass/users (a small "kill-switch" policy, not root).
# Set BAO to run the CLI another way, for example BAO="docker exec bao bao".
set -eu
u=$1
BAO=${BAO:-bao}
bao() { $BAO "$@"; }
alias_id=$(bao list -format=json identity/entity-alias/id | jq -r '.[]' | while read -r a; do
  bao read -format=json "identity/entity-alias/id/$a" | jq -r --arg u "$u" 'select(.data.name==$u and .data.mount_type=="userpass") | .data.canonical_id'
done | head -1)
[ -n "$alias_id" ] || { echo "no entity for $u"; exit 1; }
bao write -format=json "identity/entity/id/$alias_id" disabled=true >/dev/null
echo "$(date -u +%H:%M:%S) entity $alias_id disabled"
bao delete "auth/userpass/users/$u" >/dev/null
echo "$(date -u +%H:%M:%S) login $u deleted"

For an OIDC login, match on the alias from the OIDC mount instead of userpass. Its name is the value of the claim set in the role's user_claim (often sub, or email). The entity step is the same.

The same minute, in the other systems:

  • Tailnet: expire and delete the person's nodes (headscale nodes expire, then delete), as on the offboarding page.
  • IdP: deactivate the user and end their sessions, and revoke their personal access tokens.
  • Forge and cloud: remove SSH keys and tokens; disable cloud console access.

Then verify each system with the person's old credentials, and write the times into the ticket.

Prove it

From secure-tests/insider-kill-switch/. Mallory logs in with an 8-hour token and makes a child token to keep around:

text
{"display_name":"userpass-mallory","ttl":28799,"entity_id":"57a86c3d..."}

The usual reflex is to delete the login. The token still reads the secret:

bash
bao delete auth/userpass/users/mallory
bao kv get -field=password secret/app/db   # mallory's token, after the delete
text
Success! Data deleted (if it existed) at: auth/userpass/users/mallory
example-db-pw

The kill switch, run with the kill-switch token, not root:

bash
kill-switch.sh mallory
text
22:09:32 entity 57a86c3d-88f2-5e1b-2ed7-5c514b33315d disabled
22:09:32 login mallory deleted

In the same second, mallory's token, her child token and a fresh login all fail:

bash
bao kv get secret/app/db                        # mallory's token
bao kv get secret/app/db                        # mallory's child token
bao login -method=userpass username=mallory
text
Code: 403. Errors:
* permission denied
Code: 403. Errors:
* permission denied
Code: 400. Errors:
* invalid username or password

A colleague on the same policy is unaffected, and the kill-switch token cannot read secrets itself:

text
example-db-pw
Code: 403. Errors:
* preflight capability check returned 403, please ensure client's policies grant access to path "secret/"

Mistakes people make

Deleting the account and stopping there

Existing tokens outlive the account. Disable the entity, which blocks them on the next request.

Revoking before looking

Revocation is final. Disabling blocks the tokens and keeps them, so the investigation can still look them up; revoke after it ends.

A switch that needs root

If only a root token can pull it, the switch waits for a quorum, or for the insider. Pre-issue a kill-switch credential with exactly these paths.

One system at a time

OpenBao first, the tailnet tomorrow, the IdP next week: every gap is a window. Run all scripts in the same minute, from one runbook.

Not trying the old credentials

"Success" from a delete command proves nothing about sessions. Try the old token, the old device and the old login, and record the result.

A kill switch nobody has rehearsed

The first run should not be the real one. Rehearse on a test account every quarter, and time it.

Checklist

  • Write a kill-switch policy for OpenBao and issue it to the security on-call.
  • Keep a kill-switch script per system: OpenBao, tailnet, IdP, forge, cloud.
  • In OpenBao, disable the entity first, then delete the login.
  • Expire and delete the person's tailnet devices in the same minute.
  • Deactivate the IdP user, end sessions and revoke personal access tokens.
  • Verify with the old credentials in every system and record the times.
  • Keep disabled tokens until the investigation ends, then revoke them.
  • Watch the audit log for denied requests from the person's old tokens. The entry carries no entity ID, only HMACs, so match on the HMAC of their token accessors (sys/audit-hash).
  • Rehearse on a test account every quarter.

A kill switch you have to design during the incident is just a very stressful meeting. Build it now, test it on a quiet Tuesday.

H2-CIAE

Learn it on a live range

SSO and lifecycle, in Identity and Access Engineering: a real host in your browser, and every objective checked on the machine.

Start free

The Secure Way

More on identity and access

Self-hosted identity with Zitadel, private access with Headscale, break-glass and offboarding.

All identity and access guides