Repository navigation
net/tls: keep the first handshake error for queued handshake waiters - #331
Open
travisdowns wants to merge 1 commit into
Open
travisdowns wants to merge 1 commit into
travisdowns wants to merge 1 commit into
Conversation
A read and a write on a TLS socket that find the session not yet connected both
call handshake(), which serializes them on the input and output semaphores. If
the first handshake (driven by the winner of the race) failed, the second ran it
again on the failed session. The library then reported a different error, often
a generic one ("System error during handshake" in OpenSSL, "The specified
session has been invalidated" in GnuTLS), and it replaced the recorded error, so
the second waiter and every later operation saw it instead of the real cause,
such as a certificate verification failure.
Instead, check the error after acquiring the semaphore, and if set return the
recorded error instead of retrying the handshake, in both backends.
(cherry picked from commit 46f0a47)
There was a problem hiding this comment.
🟢 Approval recommended
The TLS handshake error-preservation change has no active review issues.
No open issues 🎉
What changed in this PR
This PR preserves the original TLS handshake failure for queued callers, preventing later reads or writes from retrying a failed handshake and reporting a less useful error.
Changes:
- Recheck the recorded error after acquiring both handshake semaphores in the OpenSSL and GnuTLS backends.
- Add regression tests for queued reads after certificate verification failures and invalid TLS responses.
| File | Description |
|---|---|
tests/unit/tls_test.cc |
Adds concurrent flush/read tests that verify handshake errors are preserved. |
src/net/tls_openssl.cc |
Returns the recorded error instead of retrying a failed handshake. |
src/net/tls_gnutls.cc |
Applies the same error check to the GnuTLS backend. |
Review effort: Balanced
Lazin
approved these changes
Nov 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A read and a write on a TLS socket that find the session not yet connected both
call handshake(), which serializes them on the input and output semaphores. If
the first handshake (driven by the winner of the race) failed, the second ran it
again on the failed session. The library then reported a different error, often
a generic one ("System error during handshake" in OpenSSL, "The specified
session has been invalidated" in GnuTLS), and it replaced the recorded error, so
the second waiter and every later operation saw it instead of the real cause,
such as a certificate verification failure.
Instead, check the error after acquiring the semaphore, and if set return the
recorded error instead of retrying the handshake, in both backends.
(cherry picked from commit 46f0a47)
Backport of scylladb#3855, which is still in review upstream.
Additional details
Why it is needed here
A Redpanda change that moves the RPC client's background read loop into an explicit scheduling group makes the read loop start one task later than before, so a request's write now usually starts the TLS handshake and the read loop queues behind it. With a revoked peer certificate, the write gets
certificate revoked, but the read loop getsSystem error during handshake: error:400C0107:lib(128)::operation failand logs it at ERROR level. The certificate revocation integration test, which allows only the certificate errors, fails on every debug-build run. With this fix the read loop reportscertificate revokedand the test passes: 3/3 failures before and 3/3 passes after, on a local debug build with TRACE logging.Branch
This is the first commit on
v27.1.x-pre, created from thev26.3.xtip (dfcbf1c05) now that Redpanda has branched for 26.3, following the release-branching process. Redpandadevalready pinsdfcbf1c05, so bumpingdevto this branch adds only this fix.Tests
Two new cases in
tests/unit/tls_test.cc. Each starts a write and flush on a client whose handshake will fail, then a read that queues behind it:test_read_queued_behind_handshake_gets_verification_error: the client does not trust the server's CAtls::verification_errorSystem error during handshake); passes on GnuTLStest_read_queued_behind_handshake_gets_handshake_error: the peer answers with plain HTTPSystem error during handshakevsWrong SSL Version number; GnuTLSThe specified session has been invalidatedvsAn unexpected TLS packet was receivedOn this branch, both pass on both backends, and the full
tls_testpasses in dev mode with the GnuTLS and OpenSSL backends.