changeset 9454:d7466b820e29

QUIC: fixed ssl_reject_handshake error handling. This was broken in 7468a10b6 (1.29.0), resulting in a missing diagnostics and SSL error queue not cleared for SSL handshakes rejected by SNI, seen as "ignoring stale global SSL error" alerts, for instance, when doing SSL shutdown of a long standing connection after rejecting another one by SNI. The fix is to move the qc->error check after c->ssl->handshake_rejected is handled first, to make the error queue cleared. Although not practicably visible as needed, this is accompanied by clearing the error queue under the qc->error case as well, to be on the safe side. As an implementation note, due to the way of handling invalid transport parameters for OpenSSL 3.5 and above, which leaves a passed pointer not advanced on error, SSL_get_error() may return either SSL_ERROR_WANT_READ or SSL_ERROR_WANT_WRITE depending on a library. To cope with that, both qc->error and c->ssl->handshake_rejected checks were moved out of "sslerr != SSL_ERROR_WANT_READ". Also, this reconstructs a missing "SSL_do_handshake() failed" diagnostics for the qc->error case, replacing using ngx_ssl_connection_error() with ngx_connection_error(). It is made this way to avoid logging at the crit log level because qc->error set is expected to have an empty error queue. Reported and tested by Vladimir Homutov.
author Sergey Kandaurov <pluknet@nginx.com>
date Wed, 10 Sep 2025 17:25:36 +0400
parents 9c9985987d5b
children ab7ae29384b4
files src/event/quic/ngx_event_quic_ssl.c
diffstat 1 files changed, 16 insertions(+), 11 deletions(-) [+]
line wrap: on
line diff
--- a/src/event/quic/ngx_event_quic_ssl.c	Mon Aug 18 20:28:06 2025 +0100
+++ b/src/event/quic/ngx_event_quic_ssl.c	Wed Sep 10 17:25:36 2025 +0400
@@ -695,30 +695,35 @@
 
     ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL_do_handshake: %d", n);
 
-    if (qc->error) {
-        return NGX_ERROR;
-    }
-
     if (n <= 0) {
         sslerr = SSL_get_error(ssl_conn, n);
 
         ngx_log_debug1(NGX_LOG_DEBUG_EVENT, c->log, 0, "SSL_get_error: %d",
                        sslerr);
 
-        if (sslerr != SSL_ERROR_WANT_READ) {
+        if (c->ssl->handshake_rejected) {
+            ngx_connection_error(c, 0, "handshake rejected");
+            ERR_clear_error();
+            return NGX_ERROR;
+        }
 
-            if (c->ssl->handshake_rejected) {
-                ngx_connection_error(c, 0, "handshake rejected");
-                ERR_clear_error();
+        if (qc->error) {
+            ngx_connection_error(c, 0, "SSL_do_handshake() failed");
+            ERR_clear_error();
+            return NGX_ERROR;
+        }
 
-                return NGX_ERROR;
-            }
-
+        if (sslerr != SSL_ERROR_WANT_READ) {
             ngx_ssl_connection_error(c, sslerr, 0, "SSL_do_handshake() failed");
             return NGX_ERROR;
         }
     }
 
+    if (qc->error) {
+        ngx_connection_error(c, 0, "SSL_do_handshake() failed");
+        return NGX_ERROR;
+    }
+
     if (!SSL_is_init_finished(ssl_conn)) {
         if (ngx_quic_keys_available(qc->keys, NGX_QUIC_ENCRYPTION_EARLY_DATA, 0)
             && qc->client_tp_done)