Book a Security Review

Identity and access

Headscale with OIDC and deny-by-default ACLs, the secure way

You're not crazy, you heard right: a fresh Headscale with no policy file lets every node talk to every other node. Add OIDC without filters and anyone your identity provider knows can join the party too.

The short answer

Configure OIDC with PKCE, verified emails, and allowed domains and groups, so only your people can join. Expire nodes weekly. Load a policy file whose grants name exactly who reaches what, and add tests that assert both what must work and what must stay denied, so a careless edit fails the check.

Updated Houssam Hammoudi, CTOTested with Headscale 0.29.4, Tailscale client (stable image)

On this page
  1. What goes wrong
  2. What the docs say
  3. The secure configuration
  4. Prove it
  5. Mistakes people make
  6. Checklist

What goes wrong

Headscale loads no policy unless you give it one, and no policy means allow all. A laptop that joins the tailnet can reach every server on it.

A policy file that forgets the grants (or acls) section is also allow all. So is a grant of * to * added "for a minute" while debugging.

OIDC is set up with an issuer, a client ID and a secret, and nothing else. Headscale's user filters are off by default, so every account in the identity provider can join: contractors, test accounts, a customer tenant if the IdP is shared.

Nodes never expire in the example configuration. A laptop that left with its owner keeps its access until someone deletes it by hand.

What the docs say

By default, no policy is loaded which means that Headscale allows all traffic between nodes.

Source: Headscale docs, Policy

Filters are disabled by default, users are allowed to join once the authentication with the identity provider succeeds.

Source: Headscale docs, OpenID Connect

Tests let you ensure you don't accidentally revoke important permissions or expose a critical system.

Source: Tailscale docs, Tailnet policy file syntax

Headscale's features page lists policy tests as supported, but its policy page never explains what happens when they fail. In Headscale 0.29 a failing test is only a warning at startup ("policy tests failed at boot; server starting anyway"), and an error in headscale policy check. So the check has to run in CI; the server will not stop a bad policy for you.

The secure configuration

The relevant parts of /etc/headscale/config.yaml:

yaml
server_url: https://hs.example.com           # TLS at a reverse proxy or with tls_cert_path
listen_addr: 127.0.0.1:8080                  # behind the proxy
metrics_listen_addr: 127.0.0.1:9090          # metrics on loopback only
grpc_listen_addr: 127.0.0.1:50443            # remote CLI off; use the unix socket
grpc_allow_insecure: false
unix_socket: /var/run/headscale/headscale.sock
unix_socket_permission: "0770"
node:
  expiry: 7d                                 # the example config says 0 (never)
policy:
  mode: file
  path: /etc/headscale/policy.hujson         # without it, every node reaches every node
derp:
  server:
    enabled: true
    verify_clients: true                     # only this tailnet's nodes may relay
    region_id: 999
    region_code: headscale
    region_name: Headscale Embedded DERP
    stun_listen_addr: 0.0.0.0:3478
    private_key_path: /var/lib/headscale/derp_server_private.key
    automatically_add_embedded_derp_region: true
    ipv4: 192.0.2.10
  urls: []                                   # no third-party relays
logtail:
  enabled: false
oidc:
  only_start_if_oidc_is_available: true
  issuer: https://id.example.com
  client_id: headscale
  client_secret_path: ${CREDENTIALS_DIRECTORY}/oidc_client_secret   # systemd LoadCredential
  scope: ["openid", "profile", "email", "groups"]   # allowed_groups needs the groups claim; the scope name varies by IdP
  email_verified_required: true
  allowed_domains: ["example.com"]           # every filter must pass
  allowed_groups: ["tailnet-users"]
  pkce:
    enabled: true
    method: S256

The policy. Grants name exactly who reaches what; the tests pin it down:

json
// /etc/headscale/policy.hujson
// Deny by default: only what a grant allows is reachable.
// An empty file, or one without "grants"/"acls", allows everything.
{
  "groups": {
    "group:ops": ["[email protected]"],
    "group:dev": ["[email protected]"],
  },
  "tagOwners": {
    "tag:prod-db": ["group:ops"],
    "tag:web":     ["group:ops"],
  },
  "grants": [
    // ops: the database port and SSH on database servers
    {"src": ["group:ops"], "dst": ["tag:prod-db"], "ip": ["tcp:5432", "tcp:22"]},
    // dev: HTTPS on web servers, nothing on databases
    {"src": ["group:dev"], "dst": ["tag:web"], "ip": ["tcp:443"]},
    // web servers may reach the database port, not SSH
    {"src": ["tag:web"], "dst": ["tag:prod-db"], "ip": ["tcp:5432"]},
  ],
  // Checked by "headscale policy check": a change that opens more fails here.
  "tests": [
    {"src": "[email protected]", "accept": ["tag:prod-db:5432", "tag:prod-db:22"], "deny": ["tag:web:443"]},
    {"src": "[email protected]",   "accept": ["tag:web:443"], "deny": ["tag:prod-db:5432", "tag:prod-db:22"]},
    {"src": "tag:web",           "accept": ["tag:prod-db:5432"], "deny": ["tag:prod-db:22"]},
  ],
}

Servers join with tagged, single-use, short-lived keys, passed as a file:

bash
headscale preauthkeys create --tags tag:prod-db --expiration 10m -o json | jq -r .key > /run/ts-authkey
# on the server:
tailscale up --login-server=https://hs.example.com --auth-key=file:/run/ts-authkey && shred -u /run/ts-authkey

Run headscale policy check -f policy.hujson in CI on every change to the policy, then reload with systemctl reload headscale.

Prove it

From secure-tests/headscale-oidc-deny-default-acls/. With only_start_if_oidc_is_available: true and the identity provider down, Headscale does not start:

text
Exited (1)
Error: initializing: creating new headscale: creating OIDC provider from issuer config: Get "http://idp:8000/.well-known/openid-configuration": dial tcp: lookup idp on 127.0.0.11:53: no such host

With it up, Headscale starts. Four real Tailscale clients join: two people and two tagged servers:

bash
headscale nodes list
text
alice-laptop  [email protected]  100.64.0.1
bob-laptop    [email protected]    100.64.0.2
web-1         tagged-devices     tag:web      100.64.0.3
db-1          tagged-devices     tag:prod-db  100.64.0.4

The policy and its tests pass:

bash
headscale policy check -f policy.hujson
text
Policy is valid

Someone adds {"src": ["*"], "dst": ["*"], "ip": ["*"]} "for debugging". The check fails and names every hole it opened:

text
Error: rpc error: code = InvalidArgument desc = test(s) failed:
[email protected] -> tag:web:443: expected DENIED, got ALLOWED
[email protected] -> tag:prod-db:5432: expected DENIED, got ALLOWED
[email protected] -> tag:prod-db:22: expected DENIED, got ALLOWED
tag:web -> tag:prod-db:22: expected DENIED, got ALLOWED

Remove the grants section entirely and the result is the same, because a policy without grants is allow all:

text
Error: rpc error: code = InvalidArgument desc = test(s) failed:
[email protected] -> tag:web:443: expected DENIED, got ALLOWED
[email protected] -> tag:prod-db:5432: expected DENIED, got ALLOWED
[email protected] -> tag:prod-db:22: expected DENIED, got ALLOWED
tag:web -> tag:prod-db:22: expected DENIED, got ALLOWED

Every key used in the test was single-use, and the server keys carried their tags:

bash
headscale preauthkeys list
text
1  reusable=false  used=true  tags=
2  reusable=false  used=true  tags=
3  reusable=false  used=true  tags=tag:web
4  reusable=false  used=true  tags=tag:prod-db

Mistakes people make

Tests that only say "accept"

A test that checks what must work catches a broken policy. It does not catch an open one. Write a deny for every sensitive port.

OIDC without allowed_groups

Anyone who can sign in to the IdP can join. Filter by group, and by domain if the IdP serves more than your staff.

node.expiry: 0

Nodes that never expire outlive the people who own them. Seven days is a good start. Tagged nodes are exempt and never expire, so review them by hand. Offboarding still removes nodes at once (see developer onboarding and offboarding).

Reusable, untagged keys for servers

A reusable key in a script lets anyone who finds it add machines as a person. Use single-use keys with --tags and a short expiration.

Policy references by editable email

Headscale's OIDC docs warn that users may change their email or username in some IdPs and take over an existing name. Only use IdPs where email is verified and not user-editable, and keep email_verified_required: true.

The gRPC port open to the network

grpc_listen_addr is the API the Headscale CLI uses to control the server remotely (with an API key). Keep it on loopback, keep grpc_allow_insecure: false, and use the unix socket locally.

Checklist

  • Set policy.path; never run Headscale without a policy.
  • Write grants for each group and tag; no * sources or destinations.
  • Add tests with both accept and deny entries for sensitive ports.
  • Run headscale policy check in CI on every policy change.
  • Configure OIDC with allowed_groups, allowed_domains, email_verified_required and PKCE S256.
  • Read the client secret from a file (client_secret_path).
  • Set node.expiry to 7 days or less.
  • Enroll servers with single-use, tagged keys that expire in minutes.
  • Keep metrics and gRPC on loopback; set derp.server.verify_clients: true.

A tailnet with no policy is a flat network with better marketing. Write the grants, and write the tests that keep them honest.

H2-CPQE

Learn it on a live range

Private access and tailnets, in Edge and Post-Quantum Networking: a real host in your browser, and every objective checked on the machine.

Start free

The Secure Way

More on identity and access

Self-hosted identity with Zitadel, private access with Headscale, break-glass and offboarding.

All identity and access guides