SSH server hardening, the secure way

#What goes wrong

A fresh SSH server accepts passwords. Every host on the internet gets a steady stream of guesses for root, admin, ubuntu and test. One weak password on one forgotten account is enough.

Rate limiting tools such as fail2ban slow the guessing down. They do not remove the target. If the server never accepts a password, the guesses cannot succeed, no matter how many there are.

The second problem is quieter. OpenSSH uses the first value it reads for most keywords. Debian and Ubuntu read /etc/ssh/sshd_config.d/*.conf before the rest of the main file. A cloud image that ships 50-cloud-init.conf with PasswordAuthentication yes can silently cancel the line you added at the bottom of sshd_config. You edited the file, you restarted, and passwords still work.

#What the docs say

Unless noted otherwise, for each keyword, the first obtained value will be used.

Source: OpenBSD manual, sshd_config(5)

Specifies whether password authentication is allowed. The default is yes.

Source: OpenBSD manual, sshd_config(5), PasswordAuthentication

Note that disabling TCP forwarding does not improve security unless users are also denied shell access, as they can always install their own forwarders.

Source: OpenBSD manual, sshd_config(5), AllowTcpForwarding

The manual states the first-value rule once, near the top. It does not connect it to the Include line that distributions put at the top of the file, which is where most "I disabled passwords but they still work" reports come from. The forwarding note is honest: turning off forwarding is hygiene against accidents, not a wall against a user who already has a shell.

#The secure configuration

Create the login group, then put every setting in one early drop-in file. The 10- prefix makes it sort before the 50-cloud-init.conf that cloud images ship.

# The only group allowed to log in over SSH.
groupadd --system ssh-users
usermod -aG ssh-users alice      # repeat for each admin; each has their own key

# /etc/ssh/sshd_config.d/10-hardening.conf
# Keys only: no passwords, no keyboard-interactive prompts.
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
AuthenticationMethods publickey
# No direct root logins; admins log in as themselves and use sudo.
PermitRootLogin no
# Only members of this group may log in at all.
AllowGroups ssh-users
# Fewer guesses per connection, less time for unauthenticated sessions.
MaxAuthTries 3
LoginGraceTime 20
MaxStartups 10:30:60
# Turn off tunnels and forwarding nobody asked for.
X11Forwarding no
AllowTcpForwarding no
AllowAgentForwarding no
PermitTunnel no
# Drop dead sessions.
ClientAliveInterval 300
ClientAliveCountMax 2
# Log key fingerprints on login.
LogLevel VERBOSE

Apply it safely. Keep your current session open until a new login works.

sshd -t                                   # syntax check; prints nothing when the file is valid
sshd -T | grep -E '^(passwordauthentication|permitrootlogin|allowgroups) '
systemctl reload ssh                      # "sshd" on Fedora, RHEL and Rocky
ssh -o PreferredAuthentications=password [email protected]   # must fail: Permission denied (publickey)

On Fedora, RHEL and Rocky the same drop-in directory exists (/etc/ssh/sshd_config.d/); check that the main file still has its Include line near the top.

#Prove it

The stock Debian 12 settings, before the drop-in:

sshd -T | grep -E "^(permitrootlogin|passwordauthentication|kbdinteractiveauthentication|maxauthtries|x11forwarding|allowtcpforwarding|allowagentforwarding|logingracetime|allowgroups) "

logingracetime 120
maxauthtries 6
permitrootlogin without-password
passwordauthentication yes
kbdinteractiveauthentication no
x11forwarding yes
allowtcpforwarding yes
allowagentforwarding yes

After the drop-in, sshd -t passes and the effective values change:

sshd -t && echo "sshd -t: OK"
sshd -T | grep -E "^(permitrootlogin|passwordauthentication|kbdinteractiveauthentication|authenticationmethods|maxauthtries|x11forwarding|allowtcpforwarding|allowagentforwarding|logingracetime|allowgroups|loglevel) "

sshd -t: OK
logingracetime 20
maxauthtries 3
permitrootlogin no
passwordauthentication no
kbdinteractiveauthentication no
x11forwarding no
allowtcpforwarding no
allowagentforwarding no
loglevel VERBOSE
allowgroups ssh-users
authenticationmethods publickey

sshd -T -C evaluates the configuration for one specific connection, which matters once you add Match blocks:

sshd -T -C user=root,host=attacker.example.net,addr=203.0.113.7 | grep -E "^(permitrootlogin|passwordauthentication) "

permitrootlogin no
passwordauthentication no

The ordering trap. A line appended to the main file loses, because the drop-ins were read first. A drop-in that sorts earlier than yours wins:

printf "PasswordAuthentication yes\n" >> /etc/ssh/sshd_config
sshd -T | grep -E "^passwordauthentication "
printf "PasswordAuthentication yes\n" > /etc/ssh/sshd_config.d/00-cloud-init.conf
sshd -T | grep -E "^passwordauthentication "

passwordauthentication no
passwordauthentication yes

The test script is secure-tests/ssh-server-hardening/run.sh.

#Mistakes people make

#Editing the bottom of sshd_config

On Debian, Ubuntu, Fedora and RHEL the Include line is at the top. Anything already set in a drop-in wins over your line at the bottom. Always confirm with sshd -T, never by reading the file.

#Locking yourself out

Disabling passwords before your key works, or before your user is in ssh-users, ends the session for good on a remote host. Test a second login in a new terminal before you close the first one.

#Moving the port and calling it security

A different port cuts log noise. It does not stop a scanner that checks all ports, and it does not replace keys-only login. Do it for quieter logs if you like, never instead of the settings above.

#Trusting fail2ban as the fix

With passwords off, failed guesses are harmless noise. Rate limiting is still useful against connection floods, which MaxStartups and a firewall cover.

#Shared keys and old keys

One key copied to five admins makes the logs useless for knowing who logged in. Give each person their own key, and remove keys from ~/.ssh/authorized_keys when people leave.

#Checklist

• Each admin has a personal key pair and a working key login.
• /etc/ssh/sshd_config.d/10-hardening.conf exists with the settings above.
• sshd -t prints nothing.
• sshd -T shows passwordauthentication no, kbdinteractiveauthentication no and permitrootlogin no.
• sshd -T shows allowgroups with your login group, and only admins are in it.
• No other file in sshd_config.d/ sorts earlier and sets the same keywords.
• A password login attempt fails with Permission denied (publickey).
• The firewall only allows SSH from the networks that need it.
• authorized_keys files are reviewed when someone leaves.