Skip to content

fix(expose,yaml): gate UI WHO 4/18 injection by firmware & fix MappingProxyType MAC mismatch (#832) - #883

Merged
GreenGrassBlueOcean merged 1 commit into
v2-phase1-architecturefrom
fix/issue-832-expose-gateways-and-mac-mismatch
Nov 10, 2026
Merged

GreenGrassBlueOcean merged 1 commit into
v2-phase1-architecturefrom
fix/issue-832-expose-gateways-and-mac-mismatch

Conversation

@GreenGrassBlueOcean

Copy link
Copy Markdown
Contributor

What is the problem

In #832 (comment), a multi-gateway setup (F454 primary + Legrand 067292 / H4890 secondary on a shared SCS bus) surfaced four bugs and UX/documentation gaps around Expose to Bus and legacy myhome.yaml loading:

  1. False-positive myhome.yaml MAC mismatch warning on multi-gateway setups (legacy_yaml.py & websocket.py):
    • _log_mac_mismatch in custom_components/myhome/legacy_yaml.py checked isinstance(getattr(other, "data", None), dict). In Home Assistant core, a real ConfigEntry.data is an immutable types.MappingProxyType (a collections.abc.Mapping, not a dict).
    • As a result, every sibling ConfigEntry was skipped, other_macs was always [], and any secondary gateway without its own myhome.yaml section logged a false-positive warning claiming the primary gateway's MAC section "match[es] no configured gateway".
    • custom_components/myhome/websocket.py (_extract_gateway_info) had the identical isinstance(config_data, dict) check, discarding config_entry.data fallbacks (name, host, port, firmware, command_worker_count) on real entries.
  2. UI Config Subentry flow allowed creating WHO = 4 and WHO = 18 bindings on gateways that cannot inject them (config_flow_expose.py):
    • ExposeSubentryFlowHandler offered all five WHO subsystems (1, 2, 4, 13, 18) unconditionally, even on gateways whose firmware rejects or drops WHO = 4 / WHO = 18 sensor injection (067292 / H4890, H4684, F454, F459, MH200N, MH202), only to disable the created subentry at runtime with startup log warnings.
    • In ExposeManager._async_handle_bus_message, WHO = 18 DIMENSION 1200 status queries (*#18*W*1200##) and streaming handshakes (*#18*W*#1200#1*T##) were not gated by _injection_block_reason(18).
  3. WHO = 4 #0 probe addressing contradiction on bt_termo gateways (validate.py, expose.py, climate.py):
    • Commit a902370b restricted _WHO4_EXPOSE_PROBE_RE to 3-digit addresses (100..900, 101..899) and forbade 1..99.
    • However, both the live F461 (2.0.11) capture (f461_live_2026-10-18) and 8-gateway QEMU Firmware Oracle verification prove that bt_termo (F461, F460, MyHomeServer1) only translates *#4*WHERE*#0*TTTT## onto the SCS bus (D1 <ZZ> 03 02 F0 <TT> <TT> 00) for zone master addresses 1..99, and NACKs (*#*0##, zero SCS bus bytes) every 3-digit probe address >= 100 (100, 101, 150) and #Z (#50).
  4. Documentation (docs/configuration/expose.md):
    • Recommended where: "100" for WHO = 4 and implied virtual temperature (WHO = 4) and virtual energy (WHO = 18) sensors could feed 067292 / H4890 wall touchscreens across standard gateways.

Related to #832 (and complements #880).


What is the solution

  1. Mapping-safe ConfigEntry.data inspection (legacy_yaml.py, websocket.py):
    • Replace isinstance(..., dict) with isinstance(..., collections.abc.Mapping) in _log_mac_mismatch and _extract_gateway_info, skip sibling entries matching entry.entry_id, and fall back to string other.unique_id when CONF_MAC is absent from other.data.
  2. Profile-aware UI filtering & validation (config_flow_expose.py, expose.py, strings.json, translations/{en,fr,it,nl}.json):
    • Extract effective_expose_profile(gateway, who) and injection_block_reason(profile, who, where=None) in expose.py (accounting for shared-bus WHO delegation).
    • Hide unsupported WHO = 4 / WHO = 18 choices in ExposeSubentryFlowHandler._build_schema (while keeping who_default visible when reconfiguring an existing subentry) and reject unsupported submissions with translated who_injection_unsupported / who4_probe_where_unsupported errors across en, fr, it, and nl.
    • Gate WHO = 18 DIMENSION 1200 queries and streaming writes in ExposeManager._async_handle_bus_message behind self._injection_block_reason(18) is None.
  3. Align WHO = 4 #0 master-probe 1..99 addressing & phantom-climate protection (validate.py, expose.py, climate.py):
    • Allow zone master probe addresses 1..99 in _WHO4_EXPOSE_PROBE_RE alongside 100..900 and 101..899 (continuing to reject 0, 001..099, #Z, and 901..999).
    • Block 3-digit WHO = 4 probe addresses (is_probe(where)) on #0-only gateways (supports_dimension_response_injection not True, e.g. F461) in injection_block_reason.
    • In climate.py (accept), defer WHO = 4 DIMENSION 0-only bus discovery on exposed addresses (is_exposed_on_bus) in pending_actions so unused virtual master probes (e.g. WHERE = "50") never spawn phantom climate entities, while real heating zones using a virtual master probe still discover on their first zone frame (DIMENSION 14, mode event, etc.) and retain the deferred temperature.
  4. Documentation (docs/configuration/expose.md):
    • Update examples from where: "100" to where: "50" and document the exact Firmware Oracle matrix for WHO = 1, 2, 4, 13, and 18 across gateways and 067292 / H4890 wall touchscreens.

State Diagram

flowchart TD
    Input["Expose Binding / UI Subentry<br>(who, where)"] --> Resolve["effective_expose_profile(gateway, who)<br>(resolves shared-bus delegation)"]
    Resolve --> GateWHO{"who in {1, 2, 13}?"}
    GateWHO -->|Yes| AllowAll["Allowed on ALL gateways<br>(F454, MH200N, MHS1, F461, 067292, H4890)"]
    GateWHO -->|No: WHO 4 or 18| CheckFlag{"Profile supports injection?<br>(supports_who4_command_injection /<br>supports_who18_response_injection)"}
    CheckFlag -->|False or None| BlockWHO["Blocked: hidden in UI dropdown,<br>who_injection_unsupported error,<br>single startup warning if YAML"]
    CheckFlag -->|True & WHO 18| AllowW18["Allowed (TCP session report)"]
    CheckFlag -->|True & WHO 4| CheckDimResp{"supports_dimension_response_injection?"}
    CheckDimResp -->|True| Allow3Digit["1..99 & 100..900 allowed<br>(*#4*W*0*TTTT##)"]
    CheckDimResp -->|False: bt_termo #0 mode| CheckMaster{"WHERE in 1..99?"}
    CheckMaster -->|Yes: e.g. 50| AllowMaster["Allowed (*#4*50*#0*0215## -> SCS D1 32 03 02 F0 00 D7 00)<br>climate.py defers Dim 0 until real zone frame"]
    CheckMaster -->|No: 3-digit 100..900| Block3Digit["Blocked: who4_probe_where_unsupported<br>(bt_termo NACKs >= 100)"]
Loading

Hardware Verification

  • Live F461 (v2.0.11) hardware capture (tests/fixtures/traces/f461_live_2026-10-18/myhome_trace_F461_live_2026-10-18.json):
    • *#4*50*#0*0215## (WHERE = 50, zone master probe 1..99) was acknowledged with *#*1## in 155 ms (t = 66.612s..66.768s), whereas *#4*000*0*0215## was rejected with *#*0##.
  • Live multi-gateway user diagnostics (F454 + Legrand 067292) (#832):
    • Confirmed ConfigEntry.data MappingProxyType caused the false-positive legacy_yaml.py:272 MAC mismatch warning on the secondary 067292 entry when myhome.yaml contained only the primary F454 section.

How is it tested

  • tests/test_legacy_yaml.py::test_legacy_yaml_mappingproxy_entries_and_unique_id_fallback_do_not_warn:
    • Verifies real MockConfigEntry instances (isinstance(entry.data, MappingProxyType)), unique_id-only fallback entries, and non-Mapping peer guards log zero false-positive warnings.
  • tests/test_websocket.py::test_extract_gateway_info_edge_cases:
    • Verifies _extract_gateway_info preserves model, host, port, firmware, mac_prefix, and worker_count when config_entry.data is a MappingProxyType.
  • tests/test_expose.py::test_expose_subentry_flow_hides_and_rejects_unsupported_gateway_whos:
    • Verifies H4890 / 067292 profiles filter WHO = 4 and WHO = 18 out of SelectSelector options (["1", "2", "13"]), reject WHO = 4 / 18 submissions with who_injection_unsupported, keep who_default selectable when reconfiguring a legacy subentry, and preserve all 5 options when runtime_data is unloaded.
  • tests/test_expose.py::test_expose_subentry_flow_and_diagnostics & test_unverified_gateway_disables_who4_who18_with_one_warning:
    • Verifies F461Profile() rejects 3-digit WHERE = "100" (who4_probe_where_unsupported) and accepts WHERE = "50", logs the bt_termo 1..99 restriction once for legacy 3-digit bindings, and suppresses WHO = 18 DIMENSION 1200 queries/writes when WHO = 18 is blocked.
  • tests/test_expose.py::test_climate_discovery_defers_exposed_who4_master_probe_until_real_zone_frame:
    • Verifies exposed WHO = 4 master probes (50, 6#4#01) do not create phantom climate entities on DIMENSION 0 frames, while a real zone (5) receiving DIMENSION 0 followed by DIMENSION 14 (*#4*5*14*0210*1##) discovers cleanly and retains the deferred 21.5 °C probe reading.
  • Verification suite:
    • Targeted + full WSL pytest (100.0% statement coverage on config_flow_expose.py, expose.py, legacy_yaml.py, validate.py, websocket.py, and all touched lines in climate.py), plus OWNd==2.0.0b13 pinned wheel verification (166 passed).
    • ruff check custom_components tests scripts: passed (0 violations).
    • scripts/verify_ha_standards.py: passed (all 54/54 Platinum rules).
    • scripts/sync_documentation.py --check: passed (0 broken links/anchors).
    • scripts/typing_ratchet.py (mypy --strict): 0 errors.

Where are the traces

  • tests/fixtures/traces/f461_live_2026-10-18/myhome_trace_F461_live_2026-10-18.json (lines 2941–2986, run4_expose_virtual_bus_verification: *#4*50*#0*0215## -> *#*1## in 155 ms; *#4*000*0*0215## -> *#*0##).

What does the Firmware Oracle / Encyclopedia say

  1. OpenWebNet Encyclopedia (WHO = 4 & WHO = 18):
    • WHO = 4 (ownkb:claim:c003147, ownkb:source:s000138, functional/who-4-temperature-control/addressing.md ownkb:section:d000068:s000002): specifies 1..99 as Master probe of zone 1..99, 0 as All probes, 001..099 as All probes in zone, PZZ (P=1..8, ZZ=01..99) as Individual probe, and #1..#99 as Zone via central unit.
    • WHO = 18 (ownkb:claim:c002222, ownkb:chunk:r000405 & ownkb:chunk:r000421, ownkb:source:s000145, functional/who-18-energy-management/dimensions.md): DIMENSION 113 active power (*#18*WHERE*113## / *#18*WHERE*113*Val##) is specified as a meter report/query; #113 write is not specified, and DIMENSION 1200 (*#18*WHERE*#1200#Type*Time##) controls active-power notification streaming.
  2. 8-Gateway QEMU Firmware Oracle (own-firmware-oracle v3):
    • Replayed across all 8 emulated gateway firmwares (F461/020011 03152383…, F460/020012 ba867963…, MyHomeServer1/028206 aa1cd393…, F459/020105 09db3bd1…, F454/020051 be0304f6…, MH200N/010108 e32d3404…, MH202/010024 bb561eb2…, H4684/020054 09b0cf4e…):
      • *#4*1*#0*0215##, *#4*50*#0*0215##, *#4*99*#0*0215##: ACK + SCS TX ($06D1<ZZ>0302F000D700\r, 0x00D7 = 215 = 21.5 °C) on F461, F460, and MyHomeServer1 (bt_termo); NACK / silent on F454, F459, MH200N, MH202 (bt_H4690), and H4684 (bt_termoreg).
      • *#4*100*#0*0215##, *#4*101*#0*0215##, *#4*150*#0*0215##, *#4*#50*#0*0215##: NACK / silent on all 8 gateways.
      • *#18*51*#113*570##: NACK / silent on all 8 gateways.
      • *#18*51*113*570##: NACK / silent on H4684 (bt_energia); ACK on TCP on F461, F460, MyHomeServer1, F459, F454, MH200N, and MH202, but emits $06D1A10233001D0000\r (D1 A1 02 33 00 1D 00 00) — byte-for-byte identical to the *#18*51*113## active-power read request, discarding 570.

Checklist

  • PR Description: Follows the structured sections above (What is the problem, What is the solution, State Diagram, How is it tested, Where are the traces, What does the Firmware Oracle / Encyclopedia say).
  • Trace-Backed (if protocol/device change): Behavior is verified against real OpenWebNet bus traces or gateway firmware fixtures rather than guessed.
  • V2 Architecture: New or modified entities implement handle_event() and do not manually subscribe via async_dispatcher_connect.
  • Tests & Linters: pytest, ruff, and scripts/verify_ha_standards.py pass.

@GreenGrassBlueOcean GreenGrassBlueOcean mentioned this pull request Nov 9, 2026
3 tasks done
…gProxyType MAC mismatch (#832)

- Fix _log_mac_mismatch in legacy_yaml.py and _extract_gateway_info in websocket.py checking isinstance(..., dict) instead of Mapping, which ignored real Home Assistant ConfigEntry.data (MappingProxyType) and logged false-positive myhome.yaml MAC warnings on secondary gateways (#832).
- Gate WHO 4 and WHO 18 expose options and validation in ExposeSubentryFlowHandler by effective gateway profile (including shared-bus delegation), and guard WHO 18 Dimension 1200 queries/writes when WHO 18 injection is unsupported.
- Allow zone master probe addresses 1..99 in WHO 4 expose validation while blocking 3-digit probe addresses >= 100 on bt_termo #0-only gateways (F461/F460/MHS1 translate 1..99 to SCS D1 <ZZ> 03 02 F0 <TT> <TT> 00 and NACK >= 100), and defer climate zone discovery on exposed WHO 4 Dimension 0 frames until a real zone frame arrives.
- Document 8-gateway Firmware Oracle findings for WHO 1/2/4/13/18 expose compatibility in docs/configuration/expose.md.
@GreenGrassBlueOcean
GreenGrassBlueOcean force-pushed the fix/issue-832-expose-gateways-and-mac-mismatch branch from 7dce035 to 96de2fb Compare November 9, 2026 08:26
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@GreenGrassBlueOcean
GreenGrassBlueOcean merged commit f19adc5 into v2-phase1-architecture Nov 10, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants