Feat/oob sync

.devcontainer/config/codespaces-configuration.py

@@ -19,6 +19,8 @@
         "127.0.0.1",
         "*",
     ]
+    # Development environment — logging config values is an accepted tradeoff here.
+    # CodeQL alert for this is dismissed intentionally.
     print(f"🔗 Codespaces detected: {codespace_name}")
     print(f"🔒 CSRF Trusted Origins: {CSRF_TRUSTED_ORIGINS}")
     print(f"🌐 Allowed Hosts: {ALLOWED_HOSTS}")

.github/copilot-instructions.md

@@ -5,6 +5,7 @@
 > - [frontend.instructions.md](instructions/frontend.instructions.md) – applies to templates and static files
 > - [background-jobs.instructions.md](instructions/background-jobs.instructions.md) – applies to `jobs.py`, import views, and import utilities
 > - [sync.instructions.md](instructions/sync.instructions.md) – applies to sync views, base views, tables, and sync JS
+> - [release.instructions.md](instructions/release.instructions.md) – applies to changelog, pyproject.toml, and `__init__.py` version bumps
 
 ## Architecture & Key Modules
 - Plugin hooks into NetBox (Django 5) under `netbox_librenms_plugin/`; respect NetBox plugin APIs (`navigation.py`, `urls.py`, `api/`).
@@ -62,13 +63,36 @@
 - `_get_safe_redirect_url(request)` validates referrer URLs to prevent open-redirect attacks.
 
 ### Permission Helpers for Background Jobs
-- Background jobs run outside view context and cannot use view mixins. Use standalone helpers from `import_utils.py` (`check_user_permissions`, `require_permissions`). See `background-jobs.instructions.md` for details.
+- Background jobs run outside view context and cannot use view mixins. Use standalone helpers from `import_utils/permissions.py` (`check_user_permissions`, `require_permissions`). See `background-jobs.instructions.md` for details.
 
 ### API & Navigation Permissions
 - API endpoints use `LibreNMSPluginPermission` class in `api/views.py` (GET=view, others=change).
 - Navigation menu (`navigation.py`) has 3 groups: **Settings** (Plugin Settings, Interface Mappings), **Import** (LibreNMS Import), **Status Check** (Site & Location Sync, Device Status, VM Status). All items use `permissions=[PERM_VIEW_PLUGIN]`.
 - **Background job polling requires superuser** — non-superusers fall back to synchronous mode. See `background-jobs.instructions.md` for details.
 
+## CodeQL & Security Patterns
+
+### Clearing CodeQL `py/reflected-xss` false positives
+When a view builds an `HttpResponse` from Django-template-rendered HTML (decoded via `.content.decode()`), CodeQL traces `request → template-render → HttpResponse` as reflected XSS even though Django templates auto-escape all user values.
+
+**Correct fix:** use `format_html()` to compose the envelope and `mark_safe()` as a **trust assertion** on the inner HTML — CodeQL's Django taint model recognises this pattern and stops tracking the taint:
+
+```python
+from django.utils.html import format_html
+from django.utils.safestring import mark_safe
+
+modal_html = some_view.get(request, pk).content.decode("utf-8")
+oob = format_html('<div id="target" hx-swap-oob="innerHTML">{}</div>', mark_safe(modal_html))
+return HttpResponse(oob, content_type="text/html")
+```
+
+> **Important:** `mark_safe()` is a trust assertion, not a sanitizer — it tells Django "I guarantee this string is already safe HTML." Only use it when `modal_html` comes from a server-rendered Django view (whose templates auto-escape all user values). Never pass untrusted user input to `mark_safe()` — that would introduce real XSS.
+
+**Do NOT** use `# lgtm[py/reflected-xss]` — that is LGTM.com legacy syntax and is **not** honoured by GitHub's modern CodeQL Action.
+
+### URL converters
+Always use `<int:pk>` (not `<str:pk>`) for numeric IDs in URL patterns. Django's `<int:>` converter auto-validates and returns 404 for non-integer values, eliminating the URL-parameter taint source that CodeQL otherwise flags.
+
 ## When in Doubt
 - Check docs in `docs/development/` for structure, view inheritance, mixins, and template conventions before introducing new patterns.
 - Review the existing sync views (e.g., `views/sync/interfaces.py`) as reference implementations for data flow and caching patterns.

.github/instructions/background-jobs.instructions.md

@@ -1,5 +1,5 @@
 ---
-applyTo: "**/jobs.py,**/views/imports/**,**/import_utils.py,**/import_validation_helpers.py"
+applyTo: "**/jobs.py,**/views/imports/**,**/import_utils/**,**/import_validation_helpers.py"
 description: Background job architecture, import workflow, and task management patterns
 ---
 
@@ -69,13 +69,15 @@ Filter fields: `librenms_location`, `librenms_type`, `librenms_os`, `librenms_ho
 - **`DeviceVCDetailsView`** (GET) — renders VC member details via `htmx/device_vc_details.html`.
 - **`DeviceRoleUpdateView`**, **`DeviceClusterUpdateView`**, **`DeviceRackUpdateView`** (POST) — per-device dropdown updates. Apply selection to validation state and return re-rendered row via `render_device_row()`.
 
-## Key Import Utilities (`import_utils.py`)
-- `process_device_filters(filters, ...)` — fetches and validates devices from LibreNMS, returns list.
-- `validate_device_for_import(device, ...)` — core validation function, produces validation state dict.
-- `bulk_import_devices_shared(devices, user, ...)` — shared implementation between sync and background import.
-- `bulk_import_vms(vm_imports, user, ...)` — VM import implementation.
-- `fetch_device_with_cache(device_id, ...)` — retrieves/caches individual device data.
-- Cache key functions: `get_validated_device_cache_key()`, `get_cache_metadata_key()`, `get_active_cached_searches()`, `get_import_device_cache_key()`.
+## Key Import Utilities (`import_utils/` package)
+`import_utils/` is a package; the `__init__.py` re-exports key functions so callers can still use `from import_utils import ...`.
+
+- `filters.py` — `process_device_filters(filters, ...)`, `fetch_device_with_cache(device_id, ...)`.
+- `device_operations.py` — `validate_device_for_import(device, ...)`, `bulk_import_devices_shared(devices, user, ...)`.
+- `vm_operations.py` — `bulk_import_vms(vm_imports, user, ...)`.
+- `cache.py` — `get_validated_device_cache_key()`, `get_cache_metadata_key()`, `get_active_cached_searches()`, `get_import_device_cache_key()`.
+- `permissions.py` — `check_user_permissions(user, permissions)`, `require_permissions(user, permissions, action_description)`.
+- `virtual_chassis.py` — `create_virtual_chassis_with_members()`, `_sync_module_bay_counter()`.
 
 ## Validation Helpers (`import_validation_helpers.py`)
 Centralizes validation state mutation used by the role/cluster/rack update views:

.github/instructions/frontend.instructions.md

@@ -11,10 +11,11 @@ description: Frontend patterns for templates, HTMX, and static assets
 - All HTMX requests and `fetch()` calls must include a CSRF token. The standard pattern is `document.querySelector('[name=csrfmiddlewaretoken]').value` (from a hidden form input). The import JS also uses `getCookie('csrftoken')` as a fallback — prefer the hidden input approach for consistency.
 
 ## Modal Implementation
-- Modals use Tabler (Bootstrap-like) but **without** `bootstrap.Modal` helpers.
+- Modals try Bootstrap 5 native (`bootstrap.Modal`) first, falling back to manual DOM manipulation if unavailable. Both `librenms_sync.js` and `librenms_import.js` follow this pattern via `showModal()`/`hideModal()` helpers.
 - Buttons target the `htmx-modal-content` element and JavaScript in `librenms_import.html` toggles the wrapper.
 - Do not reintroduce `data-bs-toggle` or duplicate modal IDs.
 - The import page uses `ModalManager` class and `filterModalManager` instance—always use this reference in fetch callbacks, not undefined `modalInstance` variables.
+- Dismiss handlers (backdrop click, `data-bs-dismiss` buttons) are bound once per element to prevent stacking on repeated `showModal()` calls.
 
 ## JavaScript Fetch Patterns
 - Always check `response.ok` before processing fetch responses to catch HTTP errors.

.github/instructions/release.instructions.md

@@ -0,0 +1,149 @@
+---
+applyTo: "**/changelog.md,**/pyproject.toml,**/__init__.py"
+---
+
+# Release Workflow
+
+This describes the standard release process for the NetBox LibreNMS Plugin. Follow these steps exactly when asked to create a new release.
+
+## Branch Strategy
+
+> **Both `develop` and `master` have branch protection.** All changes must go through pull requests — never push directly.
+
+### Standard Flow (used for all releases)
+
+1. **Create `release/X.Y.Z` branch** from develop
+2. **Version bump commit** on the release branch
+3. **PR `release/X.Y.Z` → develop** (version bump PR)
+4. **PR `develop` → master** (release PR) — GitHub will auto-include any commits master is missing
+5. **Tag `vX.Y.Z`** on master and create GitHub release
+6. **Post-release:** master may now be ahead of develop (e.g. the merge commit). This resolves naturally — step 1 of the next release starts from the current develop, and the release PR (step 4) will reconcile any divergence.
+
+## Files to Update
+
+Three files must be updated in a single commit with message `Bump version to X.Y.Z and update changelog`:
+
+### `netbox_librenms_plugin/__init__.py`
+```python
+__version__ = "X.Y.Z"
+```
+
+### `pyproject.toml`
+```toml
+version = "X.Y.Z"
+```
+
+### `docs/changelog.md`
+Prepend a new section at the top (after `# Changelog`). Use the current date in `YYYY-MM-DD` format. Categories used (pick only those that apply):
+- `### New Features`
+- `### Improvements`
+- `### Fixes`
+- `### Development`
+- `### Documentation`
+
+Example:
+```markdown
+## X.Y.Z (YYYY-MM-DD)
+
+### Fixes
+* Description of fix (#PR_NUMBER)
+```
+
+## Version Bump PR (release/X.Y.Z → develop)
+
+**Title:** `Bump version to X.Y.Z and update changelog`
+
+**Body template:**
+```markdown
+## Summary
+Bump version to X.Y.Z and update changelog for release.
+
+## Motivation / Problem
+- Maintenance / cleanup
+
+Prepare release X.Y.Z with <brief description of what's included>.
+
+## Scope of Change
+
+- Config / settings
+- Docs only
+
+## How Was This Tested?
+
+- Not tested: version bump and changelog only
+
+## Risk Assessment
+- No impact on existing users
+- No code logic changes
+
+## Backwards Compatibility
+- No breaking changes
+```
+
+## Release PR (develop → master)
+
+**Title:** `Release X.Y.Z`
+
+**Body template:**
+```markdown
+## Summary
+Release X.Y.Z — merge develop into master for PyPI release.
+
+## Motivation / Problem
+- <Bug | Feature | Maintenance / cleanup>
+
+<One-line description of what this release contains, referencing PR numbers.>
+
+## Scope of Change
+
+<List applicable scopes from the PR template>
+
+## Changes
+<Bullet list of changes, each referencing PR numbers>
+- Bump version to X.Y.Z
+
+## How Was This Tested?
+
+<Summarize testing from the included PRs>
+
+## Risk Assessment
+<Brief risk assessment>
+
+## Backwards Compatibility
+- No breaking changes
+```
+
+## GitHub Release Text
+
+**Tag:** `vX.Y.Z` (create on master)
+**Release title:** `vX.Y.Z`
+
+**Body template:**
+```markdown
+## <Bug Fix Release | Feature Release | Maintenance Release>
+
+<One-paragraph summary of the release. Explain what was wrong and what changed.>
+
+### <Fixes | New Features | Improvements>
+
+- <Change description> (#PR_NUMBER)
+
+> ### Upgrade note
+> - <Migration/collectstatic notes, or "No database migrations in this release."> Standard [update process](https://github.com/bonzo81/netbox-librenms-plugin#update) applies.
+
+---
+
+## All Changes
+* <commit title> by @<author> in https://github.com/bonzo81/netbox-librenms-plugin/pull/<PR_NUMBER>
+```
+
+The "All Changes" section lists only the feature/fix PRs included in the release — not version bump or merge PRs.
+
+## Checklist
+
+- [ ] Version bumped in `__init__.py` and `pyproject.toml`
+- [ ] Changelog updated in `docs/changelog.md`
+- [ ] Version bump PR merged to develop
+- [ ] Release PR merged to master
+- [ ] Tag created on master
+- [ ] GitHub release published

.github/instructions/testing.instructions.md

@@ -27,8 +27,10 @@ description: Testing patterns and conventions for the NetBox LibreNMS plugin
 
 ## Test Coverage by Module
 - `librenms_api.py` → `test_librenms_api.py`, `test_librenms_api_helpers.py`
-- `import_utils.py`, `import_validation_helpers.py`, `utils.py` → `test_import_utils.py`, `test_import_validation_helpers.py`, `test_utils.py`
+- `import_utils/` package (`filters.py`, `device_operations.py`, `vm_operations.py`, `cache.py`, `permissions.py`, `virtual_chassis.py`), `import_validation_helpers.py`, `utils.py` → `test_import_utils.py`, `test_import_validation_helpers.py`, `test_utils.py`
 - `jobs.py`, `views/imports/list.py` → `test_background_jobs.py`
+- `import_utils/bulk_import.py` → `test_coverage_bulk_import.py`
+- Utility helpers (`utils.py` coverage tests) → `test_coverage_utils.py`
 - Permission mixins, API permissions, constants → `test_permissions.py`
 - VLAN API, mode detection, comparison, sync → `test_vlan_sync.py`
 - `VlanAssignmentMixin`, VLAN enrichment → `test_interface_vlan_sync.py`

contrib/README.md

@@ -0,0 +1,31 @@
+# Contrib: Example Mapping Files
+
+This directory contains example YAML mapping files for bulk import into the
+NetBox LibreNMS Plugin. Each file can be imported via the plugin's bulk import
+feature in the NetBox UI.
+
+## How to Import
+
+1. Navigate to the mapping page (e.g., **LibreNMS → Device Type Mappings**)
+2. Click the **Import** button (upload icon) in the top right
+3. Select **YAML** format
+4. Paste the contents of the relevant YAML file
+5. Click **Submit**
+
+## Available Mappings
+
+| File | Description |
+|------|-------------|
+| `interface_type_mappings.yaml` | Maps LibreNMS interface types + speeds to NetBox interface types |
+| `device_type_mappings.yaml` | Maps LibreNMS hardware strings to NetBox device types |
+| `module_type_mappings.yaml` | Maps LibreNMS inventory model names to NetBox module types (incl. transceivers) |
+| `module_bay_mappings.yaml` | Maps LibreNMS inventory container names to NetBox module bay names |
+| `normalization_rules.yaml` | Regex-based string normalization applied before module type/bay lookups |
+| `inventory_ignore_rules.yaml` | Suppresses phantom ENTITY-MIB entries (e.g. Cisco IOS-XR IDPROM artefacts) |
+| `platform_mappings.yaml` | Maps LibreNMS platform strings to NetBox device platforms |
+
+## Customisation
+
+These files are **examples** — adjust values to match the device types, module
+types, and interface types defined in your NetBox instance.  The `netbox_*`
+fields must reference objects that already exist in your NetBox.

contrib/carrier_auto_install_rules.yaml

@@ -0,0 +1,31 @@
+# Carrier Auto-Install Rules
+#
+# Suggest a holder/carrier ModuleType to install when LibreNMS reports orphan
+# child modules (e.g. CPM cards, mezzanines, MDAs) that have no matching NetBox
+# bay because their parent carrier was never installed in NetBox.
+#
+# Import via: LibreNMS Plugin → Carrier Auto-Install Rules → Import
+#
+# Fields:
+#   manufacturer:                Optional manufacturer name (exact). Empty = any vendor.
+#   device_type_pattern:         Optional regex on device_type.model. Empty = any model.
+#   librenms_child_class:        Exact entPhysicalClass of the orphan child (e.g. cpmModule).
+#   librenms_child_name_pattern: Regex on entPhysicalName of the orphan child.
+#   netbox_bay_name_pattern:     Regex on candidate empty device-level bay name.
+#   carrier_module_type:         NetBox ModuleType model (slug-style model field) to install.
+#   description:                 Optional description.
+#
+# Patterns use Python re.fullmatch() — they must match the entire string.
+# When the chassis has at least one empty bay matching netbox_bay_name_pattern,
+# an "Install Carrier" button appears on the module sync page (suggest-only,
+# never auto-installed). Multiple matching empty bays produce one button each.
+
+# Nokia 7750 SR-s chassis (e.g. SR-7s, SR-14s) report CPM cards in slots A/B
+# but the physical CMA carrier that holds them is invisible to LibreNMS.
+- manufacturer: Nokia
+  device_type_pattern: '^7750 SR-.*$'
+  librenms_child_class: cpmModule
+  librenms_child_name_pattern: '^Slot [AB]$'
+  netbox_bay_name_pattern: '^CMA$'
+  carrier_module_type: CMA2-7s
+  description: Install CMA2-7s carrier when CPM cards are reported orphaned

contrib/device_type_mappings.yaml

@@ -0,0 +1,73 @@
+# Device Type Mappings
+#
+# Maps LibreNMS hardware strings to NetBox device types.
+# Import via: LibreNMS Plugin > Device Type Mappings > Import
+#
+# Fields:
+#   librenms_hardware    — Hardware string exactly as shown in LibreNMS
+#   netbox_device_type   — NetBox DeviceType (matched by model name or ID)
+#   description          — Optional note
+#
+# The librenms_hardware value is matched case-insensitively.
+# These mappings are checked BEFORE the built-in part_number/model fallback.
+
+# Juniper — LibreNMS reports verbose marketing names
+- librenms_hardware: "Juniper MX480 Internet Backbone Router"
+  netbox_device_type: "MX480"
+  description: "Juniper MX480 chassis"
+
+- librenms_hardware: "Juniper MX960 Internet Backbone Router"
+  netbox_device_type: "MX960"
+  description: "Juniper MX960 chassis"
+
+- librenms_hardware: "Juniper MX304 Edge Router"
+  netbox_device_type: "MX304"
+  description: "Juniper MX304 edge router"
+
+- librenms_hardware: "JNP10008 [PTX10008]"
+  netbox_device_type: "PTX10008"
+  description: "Juniper PTX10008 core router"
+
+- librenms_hardware: "JNP7100-32C [ACX7100-32C]"
+  netbox_device_type: "ACX7100-32C"
+  description: "Juniper ACX7100-32C"
+
+- librenms_hardware: "JNP7024 [ACX7024]"
+  netbox_device_type: "ACX7024"
+  description: "Juniper ACX7024"
+
+- librenms_hardware: "Juniper JNP10008 Internet Backbone Router"
+  netbox_device_type: "PTX10008"
+  description: "Juniper PTX10008 (alternate hardware string)"
+
+- librenms_hardware: "Juniper VRR Internet Backbone Router"
+  netbox_device_type: "VRR"
+  description: "Juniper Virtual Route Reflector"
+
+# Nokia — model string matches directly in most cases
+- librenms_hardware: "7750 SR-7s"
+  netbox_device_type: "7750 SR-7s"
+  description: "Nokia 7750 SR-7s service router"
+
+# Cisco — often matches by part_number but not always
+- librenms_hardware: "WS-C4900M"
+  netbox_device_type: "WS-C4900M"
+  description: "Cisco Catalyst 4900M"
+
+# Cisco IOS XR
+- librenms_hardware: "8201-SYS"
+  netbox_device_type: "8201"
+  description: "Cisco 8201 (hardware string differs from model)"
+
+# UfiSpace — LibreNMS reports SONiC/ONIE platform names
+- librenms_hardware: "x86-64-ufispace-s9610-36d-r0"
+  netbox_device_type: "S9610-36D"
+  description: "UfiSpace S9610-36D"
+
+- librenms_hardware: "x86-64-ufispace-s9610-46dx-r0"
+  netbox_device_type: "S9610-46DX"
+  description: "UfiSpace S9610-46DX"
+
+- librenms_hardware: "x86-64-ufispace-s9700-53dx-r9"
+  netbox_device_type: "S9700-53DX"
+  description: "UfiSpace S9700-53DX"