forked from bonzo81/netbox-librenms-plugin
-
Notifications
You must be signed in to change notification settings - Fork 0
Feat/oob sync #79
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Feat/oob sync #79
Changes from all commits
Commits
Show all changes
120 commits
Select commit
Hold shift + click to select a range
18030d0
feat(oob-sync): OOB management controller support, detection, and mig…
marcinpsk 9812d0c
feat(oob-sync): set OOB IP via an interface-assigned address; drop ge…
marcinpsk b863825
fix(oob-sync): PR #79 review hardening, OOB-flow corrections, and docs
marcinpsk e8e9feb
fix(oob-sync): address PR #295 upstream review
marcinpsk dd45377
fix(oob-sync): PR #79 review round — nested-dict mapping shapes & guards
marcinpsk 24f853a
fix(oob-sync): guard self host/OOB id conflict and non-positive mappi…
marcinpsk 44c859a
fix(oob-sync): normalize librenms_id at source; tighten weak test ass…
marcinpsk 81bd2f4
fix(oob-sync): reject ambiguous device-type mappings; render confirm …
marcinpsk b638b5e
fix(oob-sync): reject concurrent OOB type change on attach
marcinpsk 661755c
fix(ip-sync): re-resolve interface from current NetBox state at sync …
marcinpsk bdfa9b1
fix(interfaces): clear ports cache at start of refresh so failures sh…
marcinpsk bb12b26
fix(interfaces): cache host-only snapshot as oob_incomplete instead o…
marcinpsk d2e48d3
fix(platform): reuse existing platform and add missing mapping on Cre…
marcinpsk f1ee434
fix(oob-sync): PR #79 CodeRabbit round-13 findings
marcinpsk 496cbee
fix(oob-sync): address CodeRabbit review (platform race reuse + error…
marcinpsk a1bf890
fix(oob-sync): neutral skip-warning wording; generic platform-create …
marcinpsk 0bc548d
test(migrate): assert HX-Refresh absent on the 409 conflict paths
marcinpsk 2528f1c
fix(oob-sync): clear stale serial flags, keep migrated context, guard…
marcinpsk ca048a8
fix(oob-sync): don't leak raw save exception in the merge toast
marcinpsk 7be953c
fix(oob-sync): fail-closed librenms_id matching + import/migration ha…
marcinpsk 0edf949
fix(oob-sync): validate server_key before reflecting it in the sync r…
marcinpsk 512b963
fix(oob-sync): re-check PlatformMapping add permission at the write site
marcinpsk 40983a0
fix(oob-sync): surface skipped-mapping warning in the import modal
marcinpsk 4052762
fix(oob-sync): fail closed on duplicate host-only or OOB-only librenm…
marcinpsk c8846f2
fix(oob-sync): signal ambiguous librenms_id distinctly so callers fai…
marcinpsk 6819ddc
fix(oob-sync): make ambiguous librenms_id a durable import blocker
marcinpsk 292812f
fix(oob-sync): address out-of-diff review findings
marcinpsk f20738a
fix(oob-sync): guard server_key redirects with explicit allowlist (Co…
marcinpsk 580d045
fix(oob-sync): gate server_key redirects with url_has_allowed_host_an…
marcinpsk f939b90
fix(oob-sync): populate existing_librenms_link for VM matches; re-sou…
marcinpsk 213ac1a
fix(oob-sync): drop VM binding when cross-model device lookup is ambi…
marcinpsk 3b31f2e
fix(oob-sync): validate PlatformMapping before save; order-independen…
marcinpsk e42e2ac
fix(oob-sync): re-check serial/IP matches when refreshing a stale-lin…
marcinpsk 90e3744
fix(oob-sync): compose HTMX envelope with format_html; don't double-r…
marcinpsk 0504b0b
test(oob-sync): pin full DB-error sanitization; cover url-barrier rej…
marcinpsk 92f4b80
fix(oob-sync): consistent VM link warning, fail-closed perms, script-…
marcinpsk a24ea93
fix(oob-sync): don't report failure when a committed import action ca…
marcinpsk 9f75b71
fix(oob-sync): gate server_key failure redirect with open-redirect ba…
marcinpsk 1ded05c
fix(oob-sync): verify PlatformMapping target platform; fall back to a…
marcinpsk 9c09855
test(oob-sync): stub lazy librenms_api in object.__new__ platform views
marcinpsk 5141dfe
fix(oob-sync): fail closed in HTML permission path when request is ab…
marcinpsk cbeb1b6
test(oob-sync): assert prefix-aware redirect fallback, not hardcoded '/'
marcinpsk 8ef63e9
fix(oob-sync): harden match-drop cleanup, OOB merge, mapping perms, I…
marcinpsk b16d4a1
fix(oob-sync): validate OOB IP before net_host preflight in _missing_…
marcinpsk 690e9c6
fix(modules): adopt template interfaces even when the port_id bind no…
marcinpsk 602aea9
fix(oob-sync): PR 79 review — interface re-lock type guard, prefix fa…
marcinpsk 287482c
fix(oob-sync): VC sync-device scoping, OOB row read-only, no-interfac…
marcinpsk 99c7b9d
fix(oob-sync): host-pair status ordering + tighten two review tests
marcinpsk e32a8fd
fix(oob-sync): set IPAddressTable.tab so IP-tab pagination stays on t…
marcinpsk 8923b07
fix(oob-sync): host-id precedence in VC sync-device, generic skip rea…
marcinpsk f4bcb62
fix(oob): scope cache.ttl/server_key/form markup; warn on existing pl…
marcinpsk 395197b
fix(oob): harden merge/migrate/import edge cases from review
marcinpsk fbd3d4f
fix(oob): collapse multi-line {# #} template comment to one line
marcinpsk 65d0e60
fix(oob): IP table pagination prefix + dedup bulk-import IDs on parse…
marcinpsk c54fe0f
fix(oob): OOB-IP perm preflight no-op guard, mapping race warning, HT…
marcinpsk f2a1dd4
fix(oob): harden multi-server convert, merge/move, and cable verify
marcinpsk e78ca94
fix(oob): require change Device for move-to-winner endpoints
marcinpsk 7667a3a
fix(oob): clear stale ambiguous-id blocker, drop obsolete migrated ma…
marcinpsk be603d7
fix(oob): fail closed on misconfigured server in rebind/redirect; dro…
marcinpsk 13741c7
fix(modules): report both bind and adoption in interface-update message
marcinpsk cc54a1b
fix(tests): isolate rebind_api_for_server in SyncIPAddressesView unit…
marcinpsk 51c0bad
fix(migrate): preserve sync tab + server_key on non-HTMX move redirects
marcinpsk 47a5052
fix(oob): extract OOB iface-picker toggle to a script block; guard me…
marcinpsk 725969c
fix(oob): reject boolean object_id before the falsy check in IP verify
marcinpsk 46af91e
fix(oob): don't close the validation modal on Escape while a nested d…
marcinpsk ff9c58a
fix(import): fail closed on ambiguous librenms_id, harden id input + …
marcinpsk 785de4f
fix(oob/import): harden migrated-marker, OOB id/module rows, inventor…
marcinpsk 0b6dabd
fix(modules): bulk install matches module-scoped bays for orphan tran…
marcinpsk 33fbf5f
fix(modules): fail closed on malformed inventory elements; preserve b…
marcinpsk ac1853d
fix(modules): generic OOB toast, fail closed on malformed ports paylo…
marcinpsk c34061b
fix(modules/ip): fail closed on more malformed LibreNMS payloads; don…
marcinpsk d952728
fix(modules/migrate/import): server_key in module redirects, lock own…
marcinpsk 6bcd47f
test(migrate): real-DB IP-FK reconcile coverage + fix unique-constrai…
marcinpsk 1aae04a
test(import): convert _refresh_existing_device coverage to real DB
marcinpsk 5e3a4b5
test(migrate): convert marker/winner-resolution coverage to real DB
marcinpsk 9debfcf
test: centralise real-DB object builders in conftest
marcinpsk 7117aea
test(oob): convert OOB interface/IP attach coverage to real DB
marcinpsk 721c429
test(ip-sync): convert interface-resolution coverage to real DB
marcinpsk ca04009
fix(merge): save donor before winner so oob_ip transfer respects the …
marcinpsk d0bd644
test(oob): convert AddAsOOBView.post coverage to real DB
marcinpsk a5b789b
test(modules): real VC member + module for VC-normalization report view
marcinpsk de05bb3
fix(ip-sync): reject malformed IP rows before enrichment + strengthen…
marcinpsk 05c462b
fix(cables): skip non-dict port rows before dereferencing in get_link…
marcinpsk 3088470
fix(vlan-sync): keep CSRF + hidden inputs out of the migrated-mode div
marcinpsk d4457be
docs(interfaces): correct get_context_data docstring for OOB-incomple…
marcinpsk 92344e0
chore: stop tracking Playwright MCP session artifacts
marcinpsk 421f33e
fix(import): require a unique serial/hostname peer for merge suggestion
marcinpsk d9d4285
test(migrate): assert reject paths move/persist nothing before the toast
marcinpsk 917af35
test(device-fields): cover _sync_redirect URL-validation reject branch
marcinpsk 60e848e
fix(interfaces): invalidate ports snapshot before missing-librenms_id…
marcinpsk cf40551
fix(migrated): keep CSRF token in migrated interface tab; escapejs hx…
marcinpsk 0a72cda
test: harden HTMX reject-path, OOB message-restore, and log assertions
marcinpsk 357f586
fix(views): harden cable/port refresh against missing ids, malformed …
marcinpsk 406afd4
fix(import-ui): close nested modal in the no-Bootstrap fallback path
marcinpsk 395138a
test(device-ops): exercise the [:2] serial-peer slice and assert the …
marcinpsk 4eb7efc
fix(oob): preserve vendor-specific OOB type in name-derived detection
marcinpsk a8f7d7e
fix(import): skip host drift checks on OOB matches; make ambiguous id…
marcinpsk 1183fa6
fix(cables): return [] for a successful zero-row refresh, None only o…
marcinpsk 8825586
fix(migrate): clear donor IP FK before saving winner FK on direct tra…
marcinpsk 69e8bc6
fix(cables): warn when host LLDP fetch fails but other rows make refr…
marcinpsk b09bb10
fix(interfaces): keep OOB rows non-syncable so a name collision can't…
marcinpsk 1b5b6dd
fix(modules): preserve computed fallback server_key on module-sync re…
marcinpsk 49fdeba
fix(import-ui): suppress outer-modal Escape when keypress originates …
marcinpsk 30bf01a
test(tables): make OOB-type escaping assertion adversarial
marcinpsk 22eefe2
fix(cables): treat empty OOB-only refresh as success, not host-fetch …
marcinpsk d30fe33
fix(import): clear stale role AND cluster blockers when a row resolve…
marcinpsk 208ea62
fix(import/cables): fail closed on cross-model librenms_id collision …
marcinpsk 6516520
fix(import/ui/tests): validate current merge side uniqueness; harden …
marcinpsk c9d731d
fix(ui): emit CSRF/server_key in migrated cable mode; migrated-mode N…
marcinpsk 0b64db0
fix(import): fail closed on ambiguous librenms_id in import_single_de…
marcinpsk 29349c6
fix(import): fail-closed readiness on dropped refresh match and merge…
marcinpsk 60f1c46
harden external-boundary payloads and server-key/accessor handling (u…
marcinpsk ed1ff1b
fix(ui/tests): strict pair-id in OOB-linked title; modal action contr…
marcinpsk cb96e46
refactor(ip-sync): route VC member expansion through a shared helper
marcinpsk 5aa9fac
fix(import,tables,templates): harden malformed payloads + migrated-mo…
marcinpsk b4bb0e7
fix(import,ip,modules,templates): harden malformed/late-match paths +…
marcinpsk 3b6b1b6
fix(api,ip,sync,import-ui): legacy-mode tolerance, stale-key redirect…
marcinpsk 211a80a
fix(migrated): guard server_key hx-vals on migrated-mode Move button
marcinpsk b8fe61f
fix(migrate): allow MoveInterfaceToWinnerView to actually move interf…
marcinpsk 7f8acd7
refactor(ip): route device IP-FK writes through a guarded helper
marcinpsk File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,79 @@ | ||
| # Out-of-Band (OOB) Management | ||
|
|
||
| Many servers expose a dedicated **out-of-band management controller** — iDRAC, iLO, BMC, IPMI, CIMC, and similar. LibreNMS usually polls that controller as its **own device**, separate from the host it lives in. NetBox models the same relationship differently: the controller is not a separate Device — its address is the host Device's **OOB IP** (`oob_ip`). | ||
|
|
||
| This plugin bridges the two models. During import it detects when an incoming LibreNMS device is really the OOB side of a host you already have, and offers the right action to reconcile them instead of creating a duplicate device. | ||
|
|
||
| ## How the link is stored | ||
|
|
||
| OOB linkage is recorded in the `librenms_id` [custom field](custom_field.md) alongside the host's own LibreNMS ID. The per-server value is promoted from a bare integer to a small object: | ||
|
|
||
| ```json | ||
| { | ||
| "production": { | ||
| "id": 42, | ||
| "oob": { "id": 99, "type": "drac" } | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| - `id` — the LibreNMS device ID of the **host**. | ||
| - `oob.id` — the LibreNMS device ID of the **OOB controller**. | ||
| - `oob.type` — a short label for the controller (`idrac`, `drac`, `ilo`, `bmc`, `ipmi`, `cimc`, …), or the generic `oob` when the specific type can't be determined. | ||
|
|
||
| Only these identity essentials are stored. The controller's IP and firmware version are intentionally **not** persisted here — the IP's source of truth is the host Device's interface-assigned `oob_ip`, and the version lives in LibreNMS and can be read back any time from `oob.id`. | ||
|
|
||
| ## OOB detection during import | ||
|
|
||
| When a searched LibreNMS device looks like an OOB controller (by its OS/hardware strings, e.g. an iDRAC) and matches an existing NetBox device, the validation details show an **OOB Detected** panel instead of a plain import button. From there one of three resolution flows is offered, depending on what already exists. | ||
|
|
||
| ### Add as OOB | ||
|
|
||
| Use when the existing NetBox device is the **host** and the incoming LibreNMS device is its OOB controller. | ||
|
|
||
| The **Add as OOB to *device*** action links the controller's LibreNMS ID into the host's `oob.id` slot. NetBox requires `oob_ip` to be assigned to one of the device's interfaces, so the form includes an **OOB IP interface** picker: | ||
|
|
||
| - A sensible interface is **pre-selected** (matched by name — `idrac`/`ilo`/`bmc`-style). Because the OOB IP is frequently *not* physically on that interface, the selection is **overridable**. | ||
| - Choose **+ Create new interface…** to create one (default name suggested) to hang the OOB IP on. | ||
|
|
||
| The OOB IP is then created (or re-homed) assigned to the chosen interface and set as the device's `oob_ip`. If you make no interface selection, the link is still recorded and the OOB IP is left for you to set later. | ||
|
|
||
| !!! note "Permissions" | ||
| Setting the OOB IP can create an Interface, create an IPAddress, or re-home an existing one. The action requires the matching NetBox `add`/`change` permissions for those models; if you lack them the link is still recorded and the IP step is skipped with a warning. See [Permissions & Access](permissions.md). | ||
|
|
||
| ### Promote to host | ||
|
|
||
| Use when the existing NetBox device is currently linked to the **OOB controller** (its `librenms_id` points at the controller) and the incoming LibreNMS device is the **host** side. | ||
|
|
||
| **Promote to host of *device*** re-points the linkage: the incoming host's LibreNMS ID becomes the device's `id`, and the previously-linked controller ID is demoted into the `oob` slot. No new device is created. A pre-promote modal lets you optionally override the device's **name**, **device type**, and **platform** — all default to **Keep current**, so the original promote behaviour is unchanged unless you explicitly choose **Use new**. | ||
|
|
||
| ### Merge NetBox devices | ||
|
|
||
| Use when **two different NetBox devices** turn out to represent one physical box — typically one created from the LibreNMS hostname and another from the chassis serial, where at least one already carries a LibreNMS link. | ||
|
|
||
| The validation modal lists both candidates (hostname-matched and serial-matched) with their current linkage, and you pick which one to **keep** (the *winner*) and which to absorb (the *donor*). Merging consolidates the donor's LibreNMS link state under the active server key into the winner, clears the donor's active link, and writes a `_migrated_to` marker on the donor pointing at the winner. Interfaces, cables, and primary/OOB IPs are **not** moved automatically — you re-home those incrementally (see below). | ||
|
|
||
| ## Migrating a donor device after a merge | ||
|
|
||
| A donor device (one with a `_migrated_to` marker) shows a banner on its LibreNMS sync page with **Move to winner** actions, so you can move resources over at your own pace: | ||
|
|
||
| - **Move interface to winner** — reassigns an interface (and the cables, IPs, and MACs that hang off it) to the winner. Fails if the winner already has an interface with the same name — rename or remove that one first. | ||
| - **Move IP address to winner** — re-homes an interface-assigned IP to the winner's same-named interface (move the interface first if it doesn't exist on the winner yet). | ||
| - **Transfer primary IPv4 / IPv6 / OOB IP** — points the winner's `primary_ip4` / `primary_ip6` / `oob_ip` foreign key at the donor's value and clears it on the donor. Refuses to overwrite a value already set on the winner — clear it there first. | ||
|
|
||
| Each action runs under a row lock and verifies the `_migrated_to` marker before touching anything. Once the donor has nothing left to migrate you can delete it. | ||
|
|
||
| ## Setting Primary and OOB IPs in general | ||
|
|
||
| Outside the OOB import flows, both `primary_ip` and `oob_ip` are driven from interface-assigned addresses: | ||
|
|
||
| - **Primary IP** is set on the device's **IP Addresses** sync tab: with **Set Primary IP** enabled, a synced IP that matches the LibreNMS management IP and is interface-assigned becomes the device's primary. | ||
| - **OOB IP** is set through the **Add as OOB** flow above. | ||
|
|
||
| This keeps every IP relationship valid against NetBox's requirement that primary/OOB IPs be assigned to one of the device's own interfaces. | ||
|
|
||
| ## See also | ||
|
|
||
| - [Custom Field Setup](custom_field.md) — the `librenms_id` field that stores the linkage. | ||
| - [Validation & Configuration](../librenms_import/validation.md) — where OOB is detected during import. | ||
| - [Permissions & Access](permissions.md) — permissions required for the OOB/IP actions. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,42 @@ | ||
| import re | ||
|
|
||
| # Plugin permissions (from LibreNMSSettings model) | ||
| PERM_VIEW_PLUGIN = "netbox_librenms_plugin.view_librenmssettings" | ||
| PERM_CHANGE_PLUGIN = "netbox_librenms_plugin.change_librenmssettings" | ||
|
|
||
| # LibreNMS VLAN state values | ||
| LIBRENMS_VLAN_STATE_ACTIVE = 1 | ||
|
|
||
| # OOB management controller detection | ||
| # Trailing \d*\b restricts matches to whole tokens (optionally with a numeric suffix like | ||
| # iDRAC9 / drac9) so a prefix collision inside an unrelated word — e.g. "dracut", "ipmitool" | ||
| # — can't misclassify a normal device as an OOB controller. | ||
| OOB_TYPE_PATTERN = re.compile(r"\b(idrac|ilo|ipmi|bmc|drac|cimc|oob)\d*\b", re.IGNORECASE) | ||
| OOB_TYPES = ("idrac", "ilo", "ipmi", "bmc", "drac", "cimc", "oob") | ||
|
|
||
|
|
||
| def normalize_oob_type(os_str: str, hardware_str: str = "") -> str | None: | ||
| """ | ||
| Extract and normalize the OOB controller type from LibreNMS os/hardware strings. | ||
|
|
||
| Returns the canonical lowercase token (one of OOB_TYPES) or None if no match. | ||
|
|
||
| A vendor-specific match (idrac/ilo/ipmi/bmc/drac/cimc) always wins over the | ||
| generic ``oob`` token, even when ``oob`` appears earlier in the text, so e.g. | ||
| ``normalize_oob_type("oob", "iDRAC9")`` resolves to ``"idrac"`` rather than | ||
| being masked by the generic token. | ||
|
|
||
| Examples: | ||
| normalize_oob_type("drac9", "iDRAC9") → "drac" | ||
| normalize_oob_type("oob", "iDRAC9") → "idrac" | ||
| normalize_oob_type("ilo", "") → "ilo" | ||
| normalize_oob_type("ubuntu", "") → None | ||
| """ | ||
| generic = None | ||
| for text in (os_str or "", hardware_str or ""): | ||
| for m in OOB_TYPE_PATTERN.finditer(text): | ||
| token = m.group(1).lower() | ||
| if token != "oob": | ||
| return token # vendor-specific match wins immediately | ||
| generic = generic or "oob" # remember the generic fallback, keep scanning | ||
| return generic |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.