Pr/librenms-id-multi-server U#245

.github/pull_request_template.md

@@ -36,7 +36,7 @@ Delete items that don’t apply and describe briefly.
 3.
 
 ## Risk Assessment
-- Does this change affect existing users?  
+- Does this change affect existing users?
 - Could this cause unintended imports / updates?
 
 Explain briefly.

docs/development/testing.md

@@ -35,8 +35,18 @@ The test suite covers all major plugin functionality. Tests are organized by the
 | [test_sync_interfaces.py](../../netbox_librenms_plugin/tests/test_sync_interfaces.py) | Interface sync—port matching, attribute updates, MAC handling, librenms_id assignment |
 | [test_virtual_chassis.py](../../netbox_librenms_plugin/tests/test_virtual_chassis.py) | Virtual chassis detection—VC member naming patterns and name generation |
 | [test_sync_view_mismatch.py](../../netbox_librenms_plugin/tests/test_sync_view_mismatch.py) | Sync page context—device type mismatch detection and badge rendering |
+| [test_coverage_device_fields.py](../../netbox_librenms_plugin/tests/test_coverage_device_fields.py) | Device field sync view—field update logic and device field mapping |
+| [test_coverage_list.py](../../netbox_librenms_plugin/tests/test_coverage_list.py) | Import list view—background job decision, job result loading, and GET handler |
+| [test_coverage_api.py](../../netbox_librenms_plugin/tests/test_coverage_api.py) | LibreNMS API client—malformed payload guards, error paths, and edge cases |
+| [test_coverage_sync_view.py](../../netbox_librenms_plugin/tests/test_coverage_sync_view.py) | Sync view base class—context preparation and tab rendering |
+| [test_coverage_filters.py](../../netbox_librenms_plugin/tests/test_coverage_filters.py) | Import filter logic—filter form processing and device count helpers |
+| [test_sync_modules.py](../../netbox_librenms_plugin/tests/test_sync_modules.py) | Module sync—inventory matching, module type resolution, and normalization rules |
+| [test_modules_view.py](../../netbox_librenms_plugin/tests/test_modules_view.py) | Module sync view—context preparation, table rendering, and module bay mapping |
+| [test_tables_modules.py](../../netbox_librenms_plugin/tests/test_tables_modules.py) | Module tables—column rendering, row formatting, and action buttons |
 | [test_permissions.py](../../netbox_librenms_plugin/tests/test_permissions.py) | Permission enforcement—mixin contracts, object-level permissions, and write guards |
+| [test_vm_operations.py](../../netbox_librenms_plugin/tests/test_vm_operations.py) | VM operations—virtual machine sync, interface handling, and VM-specific views |
 | [test_integration_sync.py](../../netbox_librenms_plugin/tests/test_integration_sync.py) | Integration tests—API client against local mock HTTP server |
+| [test_integration_virtual_chassis.py](../../netbox_librenms_plugin/tests/test_integration_virtual_chassis.py) | Integration tests—VC detection, negative cache, multi-server cache isolation |
 | [test_view_wiring.py](../../netbox_librenms_plugin/tests/test_view_wiring.py) | Smoke tests—view class MRO, mixin wiring, permission contracts, and template syntax |
 
 Supporting files:
@@ -77,15 +87,15 @@ pytest netbox_librenms_plugin/tests/test_background_jobs.py -v
 # Multi-server librenms_id tests
 pytest netbox_librenms_plugin/tests/test_librenms_id.py -v
 
-# Sync view tests (devices, interfaces)
-pytest netbox_librenms_plugin/tests/test_sync_devices.py netbox_librenms_plugin/tests/test_sync_interfaces.py -v
+# Sync view tests (devices, interfaces, modules)
+pytest netbox_librenms_plugin/tests/test_sync_devices.py netbox_librenms_plugin/tests/test_sync_interfaces.py netbox_librenms_plugin/tests/test_sync_modules.py -v
+
+# Integration tests (API client against mock HTTP server)
+pytest netbox_librenms_plugin/tests/test_integration_*.py -v
 
 # Sync view mismatch detection and permission enforcement
 pytest netbox_librenms_plugin/tests/test_sync_view_mismatch.py netbox_librenms_plugin/tests/test_permissions.py -v
 
-# Integration tests (API client against mock HTTP server)
-pytest netbox_librenms_plugin/tests/test_integration_sync.py -v
-
 # View wiring and template syntax smoke tests
 pytest netbox_librenms_plugin/tests/test_view_wiring.py -v
 ```
@@ -111,9 +121,10 @@ pytest netbox_librenms_plugin/tests/ -v --lf
 The test suite prioritizes speed and isolation so you can run tests frequently during development:
 
 - **Mock-based**: Unit tests use `MagicMock` instead of real database objects. No Django database setup required.
-- **Fast execution**: The full suite runs in under 0.5 seconds.
+- **Fast execution**: The full suite runs in approximately 15-20 seconds (varies by environment).
 - **Isolated**: Each test is independent with no shared state between tests.
 - **No external network access**: Tests never call external services. Integration tests use a local loopback HTTP server (`mock_librenms_server.py`) to exercise the real API client against realistic HTTP responses without requiring a running LibreNMS instance.
+- **Coverage exclusions**: Test files themselves are excluded from coverage reports (see `[tool.coverage.run]` omit list in `pyproject.toml`).
 
 This approach means tests work identically in your local development environment, in the devcontainer, and in CI pipelines.
 

docs/usage_tips/custom_field.md

@@ -38,7 +38,16 @@ Follow these steps to create the `librenms_id` custom field in NetBox:
     - **Name:** `librenms_id`
     - **Label:** `LibreNMS ID`
     - **Description:** (Optional) Add a description like "LibreNMS Device ID for synchronization".
-    - **Type:** Integer
+    - **Type:** JSON (object) — stores a per-server mapping.
+      - Multi-server example:
+        ```json
+        {"production": 42, "staging": 17}
+        ```
+      - Legacy single-server example (integer) — read-only/deprecated; do not use for new entries:
+        ```
+        42
+        ```
+        > Note: to create new entries manually use the JSON format shown above.
     - **Required:** Leave unchecked (optional).
     - **Default Value:** Leave blank.
 
@@ -61,7 +70,11 @@ You can manually assign a value to the `librenms_id` custom field for a device u
 2. **Set the LibreNMS ID:**
 
     - Scroll to the **Custom Fields** section.
-    - Enter the LibreNMS device ID in the `librenms_id` field.
+    - Enter the `librenms_id` value as a JSON object with your server key(s):
+      ```json
+      {"production": 42}
+      ```
+      For multiple servers: `{"production": 42, "staging": 17}`
 
 3. **Save Changes:**
 

netbox_librenms_plugin/forms.py

@@ -4,6 +4,7 @@
 from dcim.choices import InterfaceTypeChoices
 from dcim.models import Device, DeviceRole, DeviceType, Location, Rack, Site
 from django import forms
+from django.db.models import Case, IntegerField, Value, When
 from django.http import QueryDict
 from django.utils.translation import gettext_lazy as _
 from netbox.forms import (
@@ -60,8 +61,8 @@ def _get_librenms_poller_group_choices():
         api = LibreNMSAPI()
         success, poller_groups = api.get_poller_groups()
 
-        if success and poller_groups:
-            for group in poller_groups:
+        if success:
+            for group in poller_groups or []:
                 group_id = str(group.get("id", ""))
                 group_name = group.get("group_name", "")
                 group_descr = group.get("descr", "")
@@ -545,8 +546,13 @@ def __init__(self, *args, **kwargs):
             ]
             has_filters = any(data.get(field) for field in filter_fields)
 
-            # Apply default only on initial load (no filters, no job_id)
-            if "use_background_job" not in data and not data.get("job_id") and not has_filters:
+            non_option_fields = [
+                f for f in filter_fields if data.get(f) not in (None, "", []) and str(data.get(f, "")).strip()
+            ]
+            has_option_only = bool(data) and not bool(non_option_fields) and not has_filters
+
+            # Apply default only on initial load (no filters, no job_id, no real submission)
+            if "use_background_job" not in data and not data.get("job_id") and not has_filters and not has_option_only:
                 data["use_background_job"] = "on"
             args = (data,) + args[1:]
 
@@ -577,19 +583,36 @@ def _populate_librenms_locations(self):
         """Fetch and populate LibreNMS locations in the dropdown."""
         from django.core.cache import cache
 
+        from netbox_librenms_plugin.import_utils.cache import get_location_choices_cache_key
         from netbox_librenms_plugin.librenms_api import LibreNMSAPI
 
         try:
-            # Use caching to avoid repeated API calls
-            cache_key = "librenms_locations_choices"
+            # Determine server_key cheaply from settings to check cache before instantiating the API
+            try:
+                from netbox_librenms_plugin.models import LibreNMSSettings
+
+                _settings = LibreNMSSettings.objects.first()
+                _server_key = (_settings.selected_server if _settings else None) or "default"
+            except Exception:
+                _server_key = "default"
+
+            cache_key = get_location_choices_cache_key(_server_key)
             cached_choices = cache.get(cache_key)
+            if cached_choices is not None:
+                self.fields["librenms_location"].choices = cached_choices
+                return
 
-            if cached_choices:
+            # Cache miss — instantiate the API client and fetch
+            api = LibreNMSAPI()
+            # Recompute cache_key with the resolved server_key in case it differs from settings
+            cache_key = get_location_choices_cache_key(api.server_key)
+            # Second cache check: the resolved server_key may differ from the settings key
+            cached_choices = cache.get(cache_key)
+            if cached_choices is not None:
                 self.fields["librenms_location"].choices = cached_choices
                 return
 
             # Fetch locations from LibreNMS
-            api = LibreNMSAPI()
             success, locations = api.get_locations()
 
             if success and locations:
@@ -752,8 +775,13 @@ def __init__(self, *args, **kwargs):
         if validation and validation.get("device_type", {}).get("suggestions"):
             suggestions = validation["device_type"]["suggestions"]
             if suggestions:
-                # Include suggested device types first, then all others
+                # Annotate with suggested_order so suggested types sort first
                 suggested_ids = [s["device_type"].id for s in suggestions]
-                self.fields["device_type"].queryset = DeviceType.objects.filter(
-                    id__in=suggested_ids
-                ) | DeviceType.objects.exclude(id__in=suggested_ids)
+                priority = Case(
+                    *[When(id=pk, then=Value(i)) for i, pk in enumerate(suggested_ids)],
+                    default=Value(len(suggested_ids)),
+                    output_field=IntegerField(),
+                )
+                self.fields["device_type"].queryset = DeviceType.objects.annotate(suggested_order=priority).order_by(
+                    "suggested_order", "manufacturer__name", "model"
+                )

netbox_librenms_plugin/import_utils/__init__.py

@@ -23,6 +23,7 @@
     get_active_cached_searches,
     get_cache_metadata_key,
     get_import_device_cache_key,
+    get_import_search_cache_key,
     get_validated_device_cache_key,
 )
 from .device_operations import (  # noqa: F401