Post-quantum TLS in nginx and HAProxy with OpenSSL 3.5, the secure way

#What goes wrong

With OpenSSL 3.5, nginx and HAProxy are post-quantum out of the box. The default group list of OpenSSL 3.5 includes X25519MLKEM768, the hybrid that browsers offer, and prefers it. So far, so good.

The trouble starts when you want more. ML-KEM-1024 (NIST level 5) is not in the OpenSSL default list. You add it with ssl_ecdh_curve or curves, and you put it first, as you would with any preference list. Then a client that supports both groups still lands on the level 3 hybrid.

The reason is the TLS 1.3 key share. A client sends a guess, a key share for one or two groups, with its list of supported groups. An OpenSSL server that finds a usable guess in its list takes it, unless you tell it that a stronger group is worth an extra round trip. The colon list does not say that. The slash-separated tuple syntax that OpenSSL 3.5 introduced does.

The other failure is quieter: the binary is linked against an older OpenSSL. nginx or HAProxy then rejects the group names, or on a distribution with an older default, runs classical X25519 and nobody notices.

#What the docs say

The special value auto (1.11.0) instructs nginx to use a list built into the OpenSSL library when using OpenSSL 1.0.2 or higher

Source: nginx docs, ssl_ecdh_curve

It sets the string describing the list of elliptic curves algorithms ("curve suite") that are negotiated during the SSL/TLS handshake with ECDHE.

Source: HAProxy 3.2 configuration manual, curves

Group tuples are used by OpenSSL TLS servers to decide whether to request a stronger keyshare than those predicted by sending a Hello Retry Request (HRR) even if some of the predicted groups are supported.

Source: OpenSSL 3.5 docs, SSL_CTX_set1_groups_list

Neither the nginx nor the HAProxy page mentions ML-KEM or the tuple syntax. Both pass the string straight to OpenSSL, so the behavior that matters is documented only in the OpenSSL manual.

#The secure configuration

Check the library first. Both servers must be linked against OpenSSL 3.5 or newer.

nginx -V 2>&1 | grep -o "OpenSSL [0-9.]*"
haproxy -vv | grep -i "openssl library"

nginx:

server {
    listen 443 ssl;
    server_name www.example.com;

    ssl_certificate     /etc/nginx/tls/www.example.com.crt;
    ssl_certificate_key /etc/nginx/tls/www.example.com.key;

    # TLS 1.3 only: every ML-KEM group is a TLS 1.3 group.
    ssl_protocols TLSv1.3;

    # One group per tuple, strongest first. The "/" makes OpenSSL send a
    # Hello Retry Request for MLKEM1024 when the client supports it but
    # guessed X25519MLKEM768. Browsers (no MLKEM1024) get the hybrid,
    # classical-only clients get X25519.
    ssl_ecdh_curve MLKEM1024/X25519MLKEM768/X25519;
}

HAProxy:

global
    # Default for every "bind ... ssl" line: TLS 1.3 and the same tuples.
    ssl-default-bind-options ssl-min-ver TLSv1.3
    ssl-default-bind-curves MLKEM1024/X25519MLKEM768/X25519

frontend https
    bind :443 ssl crt /etc/haproxy/tls/www.example.com.pem
    # Or per listener:
    # bind :443 ssl crt /etc/haproxy/tls/www.example.com.pem ssl-min-ver TLSv1.3 curves MLKEM1024/X25519MLKEM768/X25519
    default_backend app

If you must serve clients that support neither X25519 nor ML-KEM, add a last tuple for them, for example MLKEM1024/X25519MLKEM768/X25519:P-256.

#Prove it

The test in secure-tests/post-quantum-tls-nginx-haproxy-openssl/run.sh starts nginx and HAProxy in one alpine:3.22 container with three group settings and probes each with four client offers. The first column is the client offer; (client default) is what an OpenSSL 3.5 client sends by default, which matches what browsers send.

./run.sh

OpenSSL: OpenSSL 3.5.8 25 Aug 2026 (Library: OpenSSL 3.5.8 25 Aug 2026)
nginx:   1.28.3
HAProxy: 3.2.23-1feef49

nginx: no ssl_ecdh_curve (default)
  (client default)                         Negotiated TLS1.3 group: X25519MLKEM768
  -groups X25519MLKEM768:MLKEM1024         Negotiated TLS1.3 group: X25519MLKEM768
  -groups MLKEM1024:X25519MLKEM768:X25519  Negotiated TLS1.3 group: X25519MLKEM768
  -groups X25519                           Peer Temp Key: X25519, 253 bits

nginx: ssl_ecdh_curve MLKEM1024:X25519MLKEM768:X25519;
  (client default)                         Negotiated TLS1.3 group: X25519MLKEM768
  -groups X25519MLKEM768:MLKEM1024         Negotiated TLS1.3 group: X25519MLKEM768
  -groups MLKEM1024:X25519MLKEM768:X25519  Negotiated TLS1.3 group: MLKEM1024
  -groups X25519                           Peer Temp Key: X25519, 253 bits

nginx: ssl_ecdh_curve MLKEM1024/X25519MLKEM768/X25519;
  (client default)                         Negotiated TLS1.3 group: X25519MLKEM768
  -groups X25519MLKEM768:MLKEM1024         Negotiated TLS1.3 group: MLKEM1024
  -groups MLKEM1024:X25519MLKEM768:X25519  Negotiated TLS1.3 group: MLKEM1024
  -groups X25519                           Peer Temp Key: X25519, 253 bits

HAProxy: bind ... no curves (default)
  (client default)                         Negotiated TLS1.3 group: X25519MLKEM768
  -groups X25519MLKEM768:MLKEM1024         Negotiated TLS1.3 group: X25519MLKEM768
  -groups MLKEM1024:X25519MLKEM768:X25519  Negotiated TLS1.3 group: X25519MLKEM768
  -groups X25519                           Peer Temp Key: X25519, 253 bits

HAProxy: bind ... curves MLKEM1024:X25519MLKEM768:X25519
  (client default)                         Negotiated TLS1.3 group: X25519MLKEM768
  -groups X25519MLKEM768:MLKEM1024         Negotiated TLS1.3 group: X25519MLKEM768
  -groups MLKEM1024:X25519MLKEM768:X25519  Negotiated TLS1.3 group: MLKEM1024
  -groups X25519                           Peer Temp Key: X25519, 253 bits

HAProxy: bind ... curves MLKEM1024/X25519MLKEM768/X25519
  (client default)                         Negotiated TLS1.3 group: X25519MLKEM768
  -groups X25519MLKEM768:MLKEM1024         Negotiated TLS1.3 group: MLKEM1024
  -groups MLKEM1024:X25519MLKEM768:X25519  Negotiated TLS1.3 group: MLKEM1024
  -groups X25519                           Peer Temp Key: X25519, 253 bits

Read the second row of each block. The client supports MLKEM1024 but sends its key share for X25519MLKEM768 first. With the colon list, the server takes the guess. With the tuple list, the server asks for MLKEM1024 and gets it.

To check your own server, run the commands on the key exchange page against it.

#Mistakes people make

#Trusting the order of a colon list

In OpenSSL 3.5 a colon list is one tuple. Inside a tuple, the server prefers a group the client already sent a key share for. Use / between groups when you want the server to hold out for the stronger one.

#Removing X25519 to "force" post-quantum

Without a classical fallback, every client that lacks ML-KEM fails the handshake. That includes older curl, older Java and many monitoring agents. Keep X25519 at the end unless the endpoint is only for clients you control, as on the CNSA 2.0 page.

#Running a packaged binary on an older OpenSSL

Distribution packages link against the system OpenSSL. On anything older than 3.5 the names MLKEM1024 and X25519MLKEM768 are unknown, and nginx refuses to start or HAProxy rejects the bind line. Check nginx -V and haproxy -vv before you change the config.

#Leaving TLS 1.2 open and calling it post-quantum

ML-KEM groups exist only in TLS 1.3. A client that downgrades to TLS 1.2 gets classical ECDHE. Set ssl_protocols TLSv1.3 or ssl-min-ver TLSv1.3 where your clients allow it.

#Forgetting the backends

HAProxy terminates TLS and opens new connections to the backends. Those use ssl-default-server-curves or curves on the server line, a separate setting from the bind side.

#Checklist

• Confirm the nginx or HAProxy binary is linked against OpenSSL 3.5 or newer.
• Set TLS 1.3 as the minimum version.
• Write the group list as tuples: MLKEM1024/X25519MLKEM768/X25519.
• Keep a classical group at the end for public endpoints.
• Set the same groups for HAProxy backend connections (ssl-default-server-curves).
• Probe with -groups X25519MLKEM768:MLKEM1024 and expect MLKEM1024.
• Probe with the client default and expect X25519MLKEM768.