Skip to content

fix(guacamole): queue identity token requests instead of refusing them - #2989

Merged
Brad-Edwards merged 1 commit into
devfrom
fix/guac-identity-queue
Nov 19, 2026
Merged

Brad-Edwards merged 1 commit into
devfrom
fix/guac-identity-queue

Conversation

@Brad-Edwards

Copy link
Copy Markdown
Collaborator

Problem

GoogleGatewayIdentity gets the Google identity token on one thread with a queue of one. A third concurrent caller is refused at once with RejectedExecutionException.

Each authority renewal asks for the token. With 16 renewal workers (#2985), most renewals failed. The leases expired, and Guacamole closed the sessions 90 to 150 seconds after sign-in.

On the live event tenant on 2026-11-19, a 45-second Java Flight Recorder capture showed 529 refused token requests in the renewal threads. Participants were logged out every two to five minutes. The portal authority endpoint answered {"allowed": true} for the same sessions.

Change

  • Set the token request queue capacity to 64, above the renewal worker count.
  • Callers wait for the cached token instead of being refused.
  • Each caller still stops waiting after three seconds.
  • Add a test that 32 concurrent callers all receive the assertion.

Live status

The same extension build runs on the live event tenant since 18:33 UTC.

Verification

mvn verify in shifter/engine/guacamole/authority-extension, with the Dockerfile test list, passes.

GoogleGatewayIdentity ran token requests on one thread with a queue of
one. A third concurrent caller was refused at once with
RejectedExecutionException. Each authority renewal asks for the token, so
with 16 renewal workers most renewals failed. The leases expired and
Guacamole closed sessions 90 to 150 seconds after sign-in. On the live
event tenant this logged participants out every two to five minutes.

Set the queue capacity to 64, above the renewal worker count. Callers
wait for the cached token instead of being refused, and each caller still
stops waiting after three seconds. Add a test that 32 concurrent callers
all receive the assertion.
@Brad-Edwards
Brad-Edwards merged commit 5ed42a4 into dev Nov 19, 2026
96 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant