Skip to content

Rate-limit hardening, reliable SSE reconnect & event-thread decoupling (builds 15–19)#18

Open
bumaas wants to merge 5 commits into
symcon:masterfrom
bumaas:pr/rate-limit-thread-hardening
Open

Rate-limit hardening, reliable SSE reconnect & event-thread decoupling (builds 15–19)#18
bumaas wants to merge 5 commits into
symcon:masterfrom
bumaas:pr/rate-limit-thread-hardening

Conversation

@bumaas

@bumaas bumaas commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

This PR builds on the current master (build 14) and adds the hardening from builds 15–19. It is the delta only; the tree matches our tested fork state. All 40 PHPUnit tests pass.

Motivation

Field logs showed three recurring problems: the daily rate limit becoming self-perpetuating, the event stream not recovering after an idle/abort, and FlowHandler/"Warten auf Skriptresultat fehlgeschlagen" thread pile-ups during bursts/reconnects.

Changes

Cloud (Home Connect Cloud)

  • Stop the event-stream IO during a rate-limit block instead of letting the underlying socket keep hitting /events every ~60 s. With a rolling 24 h quota those retries kept the window full and the block never ended. The IO is re-registered (fresh token) when the block clears.
  • Honest status: a custom rate-limit status (201, incl. German locale) instead of reporting IS_ACTIVE while blocked.
  • Handle invalid_token on the stream: refresh the token and re-register instead of looping on 401.
  • Reliable reconnect after an idle/abort: CheckServerEvents reconnects on a stale keep-alive regardless of the parent status (the old HasActiveParent() gate prevented recovery of a dead parent); reconnect backoff capped at 3 min (long backoff reserved for real rate limits).
  • No reconnect on every successful request: ResetRateLimit() (→ IPS_ApplyChanges on the IO) now only runs when a block was actually pending, so normal operations no longer close/reopen the stream.

Device (Home Connect Device)

  • Lightweight value refresh on reconnect/restart so status/settings/selected program are current again (fixes stale OperationState blocking program selection), without the expensive structure re-init.
  • Throttle full refreshes (min. 30 s per device) and a parent-status transition guard to prevent request storms that exhausted the per-minute and daily quota.
  • Decouple cloud calls from the event thread: SelectedProgram and CONNECTED no longer perform synchronous cloud requests inside ReceiveData (root cause of the thread pile-ups). They defer via RegisterOnceTimerRequestAction.
  • Profile-type guard to avoid the "String profiles cannot be modified" warning.

Configurator

  • Always list already-created device instances, even when the appliance discovery fails (rate limit/offline) — the list is no longer empty.

Tests

PHPUnit suite extended from 30 → 40 tests (regressions for rate-limit stop/status, invalid_token, reconnect + backoff cap, no-reconnect-on-success, throttle/transition guard, value refresh, deferred SelectedProgram/CONNECTED refresh). All green.

bumaas and others added 5 commits July 6, 2026 17:25
Device:
- Leichter Wert-Refresh (refreshDeviceValues) bei Reconnect/Neustart statt
  nur bei needsInitialization -> kein veralteter Status/OperationState mehr.
- 30s-Throttle (refreshThrottled) fuer Init- und Refresh-Pfad gegen den
  Request-Sturm bei CONNECTED/IM_CHANGESTATUS-Bursts.
- Transition-Guard in MessageSink: nur echte Parent-Statuswechsel loesen Refresh aus.
- Profil-Guard: IPS_SetVariableProfileValues nur auf numerischen Profilen
  (behebt "Werte von String-Profilen koennen nicht modifiziert werden").
- Diagnose-Debug (Trigger/Modus, refreshDeviceValues done (N requests)).

Cloud:
- stopEventStream: IO waehrend Rate-Limit-Sperre deaktivieren -> kein
  60s-/events-Hammering, das rollierende 24h-Fenster kann leerlaufen.
- Ehrlicher Status 201 (STATUS_RATE_LIMITED) statt IS_ACTIVE, inkl. dt. Locale.
- invalid_token-Handler: abgelaufener Token im Stream -> sofort Reconnect
  mit frischem Token statt 401-Dauerloop.
- Tageszaehler RequestDayCount in ForwardData (Diagnose des Verbrauchs).

Configurator:
- Bereits angelegte Geraeteinstanzen werden immer gelistet, auch wenn die
  Live-Discovery scheitert (Rate-Limit/offline) - kein Frueh-Return mehr.

Tests: 30 -> 34 gruen (neue Regressionstests fuer Throttle/Transition,
IO-Stopp/Reset, Status 201 und invalid_token-Reconnect); Cloud-Tests
registrieren die Parent-IO-Properties selbst, kein SymconStubs-Patch noetig.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CheckServerEvents verbindet bei veraltetem Keep-Alive jetzt unabhängig vom
Parent-Status neu (HasActiveParent-Tor entfernt) - ein toter Stream wird so
auch bei fehlerhaftem Parent wiederhergestellt. Der Fehler-Backoff ist auf
3 Minuten gedeckelt (statt bis zu 1 Stunde); lange Wartezeiten bleiben echten
Rate-Limits (429) vorbehalten.

Regressionstests: Reconnect trotz inaktivem Parent, Backoff-Cap 180s (36 Tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Der SelectedProgram-NOTIFY loeste bisher einen synchronen Cloud-Call in
ReceiveData auf dem Flow-Handler-Thread aus - bei parallelem Stream-Reconnect
fuehrte das zu "Warten auf Skriptresultat fehlgeschlagen" (Thread-Stau). Der
Refresh laeuft jetzt entkoppelt: RegisterOnceTimer -> RequestAction
('RefreshSelectedProgram') auf eigenem Thread, ReceiveData kehrt sofort zurueck.
Ersetzt die Zwischenloesung (statischer Timer + oeffentliche Methode); getTime()
entfernt.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Der CONNECTED-Event rief in ReceiveData weiterhin synchron
refreshDeviceState -> refreshDeviceValues/InitializeDevice auf dem Flow-Thread
auf. Waehrend eines Stream-Reconnects antwortet der Parent nicht rechtzeitig ->
Flow-Thread blockiert -> "Warten auf Skriptresultat fehlgeschlagen". Der Refresh
laeuft jetzt entkoppelt: RegisterOnceTimer -> RequestAction('RefreshDeviceState')
auf eigenem Thread (analog zum SelectedProgram-Fix aus Build 17).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Jede erfolgreiche Anfrage (getData 200 / putData 204 / deleteData 204) rief
bisher ResetRateLimit -> ForceRegisterServerEvents -> IPS_ApplyChanges($parent)
auf und verband damit die SSE-IO bei jeder Bedienung neu (Verbindung schliesst/
oeffnet, Event-Control-Statusflackern). ResetRateLimit laeuft jetzt nur noch,
wenn tatsaechlich eine Sperre anlag (neuer Helper clearRateLimitAfterSuccess,
Guard auf RateLimitUntil != 0). Danke an Thomas fuer den Hinweis.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant