docs(compliance): add M-26-14 compliance crosswalk

.github/ISSUES_DRAFTS/01_compliance_logging.md

@@ -0,0 +1,17 @@
+# compliance: Centralized structured logging for all components
+
+### Source
+White House OMB Memorandum M-26-14, Appendix B.2, B.3, and B.5.
+
+### Extracted Requirement
+> *"Logs must include a consistently accurate timestamp. To ensure accuracy, network time must be synchronized to Network Time Protocol (NTP) or equivalent mechanisms to a traceable time source designated within the agency. Agencies are encouraged to use authoritative time sources traceable to the U.S. Naval Observatory or the National Institute of Standards (NIST)... At a minimum, agencies must collect logs that support...: a. Determining the identity used for performing operations... b. Determining source and destination network address... c. Identifying object/resource/data events... d. Identifying actions that affect changes to privilege levels."*
+
+### Suggested Tasks
+1. Ensure all LocalObserve collector/agent components (`otelcol`, `osqueryd`, `falco`) serialize log payloads to standard structured JSON format.
+2. Synchronize all Docker host environments and collector runtime nodes via NTP against authoritative servers.
+3. Validate that standard JSON schemas consistently include crucial tracking fields: UTC timestamp, component identifier, severity/priority, user/session identifier, and network metadata (source/destination IP, port, protocol).
+
+### Acceptance Criteria
+- All system collectors emit fully structured, standardized JSON logs at `INFO+` levels.
+- Verification tests exist in CI to validate the presence of mandatory structural logging fields.
+- Logging architecture and schema documentation are fully updated in the repository.

.github/ISSUES_DRAFTS/02_compliance_retention.md

@@ -0,0 +1,17 @@
+# compliance: Log retention and secure archival policy
+
+### Source
+White House OMB Memorandum M-26-14, Appendix B.1 and Appendix C.
+
+### Extracted Requirement
+> *"Retained logs must be actively searchable for a minimum of 6 months after creation to support continuous event monitoring (CEM). They must be retrievable for a year after creation to support threat-hunting, investigation, response, and forensics (THIRF)..."*
+
+### Suggested Tasks
+1. Configure OpenObserve retention policies to retain hot/active logging datasets for at least 180 days (6 months).
+2. Configure long-term cold storage lifecycles (e.g. S3 glacier, local gzip archives) to securely retain and index logs up to 365 days (12 months).
+3. Draft clear recovery playbooks detailing cold-log "thawing" procedures to restore archived telemetry back to active searchable analysis tiers during threat-hunting operations.
+
+### Acceptance Criteria
+- Storage configuration parameters explicitly map to the 6-month hot searchable and 12-month cold retrieval baseline.
+- Automated tests or staging scripts verify that log data past the 180-day window is successfully rolled to archival storage and not deleted prematurely.
+- Archival extraction and "thawing" procedures are fully documented under `docs/compliance_crosswalk.md`.

.github/ISSUES_DRAFTS/03_compliance_access_controls.md

@@ -0,0 +1,17 @@
+# compliance: Access controls and role-based access for log and network data
+
+### Source
+White House OMB Memorandum M-26-14, Par. 125-131 and Appendix C (Log Management).
+
+### Extracted Requirement
+> *"In the event of a known or suspected compromise of one or more Federal networks, agencies shall provide logs and other relevant data to CISA and the Federal Bureau of Investigation (FBI) upon request... To the greatest extent practicable, agencies shall provide access to logs within the timeframes requested... Logs are encrypted, access is granted just in time, permissions and workloads are regularly monitored and reviewed..."*
+
+### Suggested Tasks
+1. Implement a Just-In-Time (JIT) role-based access control (RBAC) permission structure for the logging pipeline in OpenObserve.
+2. Formulate audit log generation rules to capture all logon events, data access attempts, query histories, and permission changes.
+3. Establish a standard programmatic log packaging/export command that compiles telemetry datasets in formats agreed upon by CISA and the FBI.
+
+### Acceptance Criteria
+- Detailed JIT access roles and permissions are documented in a formal RBAC matrix.
+- Verification tests confirm that any data access or system configuration change triggers a secure, unalterable log audit event.
+- Export utility scripts are validated to produce standard structured JSON artifacts.

.github/ISSUES_DRAFTS/04_compliance_network_capture.md

@@ -0,0 +1,17 @@
+# compliance: Network capture controls, minimization, and retention
+
+### Source
+White House OMB Memorandum M-26-14, Appendix A and Appendix B.5.b.
+
+### Extracted Requirement
+> *"The LRA will include guidance to ensure that logs will not capture or expose data in contravention of law. It will also advise agencies on how to protect the confidentiality and integrity of sensitive log data... At a minimum, agencies must collect logs that support...: b. Determining source and destination network address information, including protocols, ports, and session attributes."*
+
+### Suggested Tasks
+1. Complete the docker-compose integration of `openobserve/goflow2` to capture structured sFlow/NetFlow/IPFIX logs.
+2. Develop high-efficiency collector regex filters to purge incidental payload fields containing potentially sensitive PII or data in contravention of privacy laws.
+3. Configure KMS encryption keys and standard TLS 1.3 tunnels for all network capture log transit paths.
+
+### Acceptance Criteria
+- GoFlow2 integrates smoothly within the compose orchestrator, emitting structured network connection logs.
+- Test logs demonstrate successful filtering and stripping of payload details while retaining required IP, port, protocol, and session attributes.
+- Network log data retention configurations map to the 12-month THIRF requirements.

.github/ISSUES_DRAFTS/05_compliance_audit_trail.md

@@ -0,0 +1,17 @@
+# compliance: Tamper-evident audit trail for logging pipeline and config changes
+
+### Source
+White House OMB Memorandum M-26-14, Appendix C (Log Management Level 3).
+
+### Extracted Requirement
+> *"Logs are encrypted in transit and at rest, and regularly hashed for veracity."*
+
+### Suggested Tasks
+1. Enforce strict repository branch protections on all telemetry configuration files (`otel-collector-config.yaml`, `osqueryd.conf`, `falco_rules.local.yaml`).
+2. Implement automated drift detection scripts in GitHub CI to ensure running container states match committed repository configurations.
+3. Configure automated storage block hashing inside the storage engine, logging calculated SHA-256 integrity hashes to a secondary immutable target for tamper-evidence.
+
+### Acceptance Criteria
+- Configuration files are locked under protected branches with signature requirements.
+- Automated CI steps evaluate configuration integrity and alert on pipeline anomalies or unauthorized modifications.
+- Active logs show regular hashing of data partitions for veracity validation.

.github/ISSUES_DRAFTS/06_compliance_monitoring.md

@@ -0,0 +1,17 @@
+# compliance: Monitoring, alerting, and incident response hooks for logging anomalies
+
+### Source
+White House OMB Memorandum M-26-14, Appendix B.5.k and Appendix C (Collection Operations).
+
+### Extracted Requirement
+> *"Generating appropriate automated alerts for all of the above... Logs generate actionable alerts covering at least 70% of baseline logging requirements, and detections are routinely evaluated and tuned [to enable CEM/THIRF outcomes]."*
+
+### Suggested Tasks
+1. Set up active health probes monitoring `otelcol` buffer sizes and OpenObserve ingest rates.
+2. Define alerts for anomalous logging states, such as a drop of >=50% in log ingest volume or complete loss of connection to endpoints.
+3. Incorporate detailed runbooks in the repository linking compliance alarms to rapid incident response processes.
+
+### Acceptance Criteria
+- Storage ingestion monitors are actively running and visible in dashboard setups.
+- Real-time alerts fire on buffer overflows, storage connection failures, or throughput anomalies.
+- Baseline alert coverage is tuned to encompass >=70% of the custom OSquery/Falco compliance rules.

.github/ISSUES_DRAFTS/07_compliance_encryption.md

@@ -0,0 +1,17 @@
+# compliance: Data protection for stored log and capture artifacts (encryption & KMS)
+
+### Source
+White House OMB Memorandum M-26-14, Appendix C (Log Management Level 2 & 3).
+
+### Extracted Requirement
+> *"Logs are stored and encrypted at rest... Logs are encrypted in transit and at rest, and regularly hashed for veracity."*
+
+### Suggested Tasks
+1. Enforce TLS 1.3 encryption across all internal log routing paths (`otelcol` to OpenObserve).
+2. Configure AES-256 at-rest encryption on the primary OpenObserve storage tier.
+3. Establish a standard policy and key rotation schedule for storage encryption keys using an integrated Key Management Service (KMS).
+
+### Acceptance Criteria
+- Configuration files explicitly reference TLS 1.3 and active KMS-backed at-rest encryption setups.
+- Compliance test files confirm that unauthenticated users cannot read stored data partitions.
+- Security key rotation policy is fully documented in the compliance baseline files.

.github/ISSUES_DRAFTS/08_compliance_docs_tests.md

@@ -0,0 +1,17 @@
+# compliance: Documentation, tests, and CI checks for compliance controls
+
+### Source
+White House OMB Memorandum M-26-14, Par. 105-121 and Appendix C (Maturity Model).
+
+### Extracted Requirement
+> *"Agencies must submit an Agency Logging Plan to OMB and CISA... This plan must describe the operational steps required for the agency to deploy and maintain effective CEM and THIRF objectives... Agencies will measure and report on progress in terms of the percentage of systems that are determined to be operating at each maturity level."*
+
+### Suggested Tasks
+1. Maintain and continuously update `docs/compliance_crosswalk.md` as compliance controls are implemented.
+2. Develop automated Pytest validation files that inspect logging JSON samples and verify compliance field mappings.
+3. Hook these validation scripts directly into GitHub CI pipelines to ensure no future configurations compromise baseline M-26-14 compliance requirements.
+
+### Acceptance Criteria
+- Full compliance crosswalk file matches implemented features and issue updates.
+- Automated validation checks are executed and pass successfully in the CI pipeline.
+- Test logs verify exact structured JSON log field definitions.

.github/PR_BODY.md

@@ -0,0 +1,7 @@
+### Summary
+This Pull Request introduces the foundational **M-26-14 Compliance Crosswalk** and corresponding GitHub Issue Drafts mapping OMB M-26-14 federal logging requirements in the **LocalObserve** pipeline.
+
+### What this PR does
+1. **M-26-14 Crosswalk Document**: Creates `docs/compliance_crosswalk.md` mapping M-26-14 requirements for real-time Continuous Event Monitoring (CEM) and Threat Hunting, Investigation, Response, and Forensics (THIRF) to specific architectural features in LocalObserve, fully linked to active tracking issues.
+2. **Standardized Issue Drafts**: Adds 8 per-requirement GitHub Issue Drafts inside `.github/ISSUES_DRAFTS/` containing exact text extractions, suggested tasks, and verification acceptance criteria.
+3. **Tracking Integration**: Programmatically creates all 8 compliance tracking issues (Issues #19 to #26) on the GitHub repository and links them directly inside the crosswalk table and detailed requirement sections.

docs/compliance_crosswalk.md

@@ -0,0 +1,85 @@
+# M-26-14 Logging Compliance Crosswalk & Audit
+
+This document establishes the official compliance crosswalk for the **LocalObserve** project against the White House Office of Management and Budget (OMB) Memorandum **M-26-14**: *Ensuring Effective and Efficient Agency Logging and Network Visibility to Defend Against Evolving Cyber Threats* (published May 22, 2026).
+
+## Executive Summary
+OMB Memorandum M-26-14 rescinded M-21-31 to implement an adaptive, risk-prioritized framework that balances network observability and incident response with operational feasibility and cost effectiveness. It focuses on two core logging objectives:
+1. **Continuous Event Monitoring (CEM)**: Real-time telemetry ingestion and analysis for immediate security monitoring.
+2. **Threat Hunting, Investigation, Response, and Forensics (THIRF)**: Cold/hot data storage and retrieval capabilities for deep post-incident tracing.
+
+This audit crosswalk maps M-26-14 baseline mandates to specific architectural objectives, issue trackers, and technical parameters in LocalObserve.
+
+---
+
+## Logging Compliance Audit Matrix
+
+| Req ID | M-26-14 Target Reference | LocalObserve Implementation Strategy | Tracking Issue | Status |
+| :--- | :--- | :--- | :--- | :--- |
+| **Req-1** | Appendix B.3, B.5 (Structured Schema & Timestamp Sync) | JSON-structured log formatting across all system collectors, aligned with authoritative, agency-approved NTP synchronization. | [Issue #19](https://github.com/JJediny/LocalObserve/issues/19) | 🟡 Planned |
+| **Req-2** | Appendix B.1 (Retention Policy: Search vs. Retrieval) | 6-month hot searchable data retention in OpenObserve alongside a 12-month cold retrieval policy. | [Issue #20](https://github.com/JJediny/LocalObserve/issues/20) | 🟡 Planned |
+| **Req-3** | Par. 125-131, Appendix C (Access Controls & JIT) | SOC integration, Just-In-Time (JIT) access policies, and audited log exports for CISA/FBI. | [Issue #21](https://github.com/JJediny/LocalObserve/issues/21) | 🟡 Planned |
+| **Req-4** | Appendix A, B.5.b (Network Capture Minimization) | Integrated flow captures via `goflow2` with PII filters to protect sensitive identifiers. | [Issue #22](https://github.com/JJediny/LocalObserve/issues/22) | 🟡 Planned |
+| **Req-5** | Appendix C (Log Veracity & Hashing) | Hashing log batches for tamper-evidence and auditing configurations. | [Issue #23](https://github.com/JJediny/LocalObserve/issues/23) | 🟡 Planned |
+| **Req-6** | Appendix B.5.k, Appendix C (Pipeline Monitoring & Alerts) | Pipeline health monitoring, threshold alerting for data drops, and tuning alerts. | [Issue #24](https://github.com/JJediny/LocalObserve/issues/24) | 🟡 Planned |
+| **Req-7** | Appendix C (Data Protection & KMS) | TLS 1.3 transit encryption, AES-256 at-rest encryption, and integration with KMS. | [Issue #25](https://github.com/JJediny/LocalObserve/issues/25) | 🟡 Planned |
+| **Req-8** | Appendix A, C (Compliance Validation & CI) | Automated CI regression tests verifying compliance controls and fields. | [Issue #26](https://github.com/JJediny/LocalObserve/issues/26) | 🟡 Planned |
+
+---
+
+## Detailed Compliance Requirements
+
+### Requirement 1: Centralized Structured Logging for all Components
+- **M-26-14 Text Reference**: 
+  > *"Log storage may be decentralized; however, logs must be readily available to the top-level agency security operations center (SOC)... Logs must include a consistently accurate timestamp. To ensure accuracy, network time must be synchronized to Network Time Protocol (NTP) or equivalent mechanisms to a traceable time source... traceable to the U.S. Naval Observatory or NIST, where feasible."* (Appendix B.2, B.3, B.5)
+- **LocalObserve Implementation Plan**:
+  Configure the collector (`otelcol`) and agent (`osqueryd`, `falco`) engines to append standard UTC timestamps and serialize outputs to JSON formats. Synchronize container and host systems against configurable, agency-approved NTP servers (for example, internal stratum sources or NIST-provided endpoints, where feasible).
+- **Assigned Tracker**: [Issue #19](https://github.com/JJediny/LocalObserve/issues/19)
+
+### Requirement 2: Log Retention and Secure Archival Policy
+- **M-26-14 Text Reference**: 
+  > *"Retained logs must be actively searchable for a minimum of 6 months after creation to support continuous event monitoring (CEM). They must be retrievable for a year after creation to support threat-hunting, investigation, response, and forensics (THIRF)."* (Appendix B.1)
+- **LocalObserve Implementation Plan**:
+  Enforce a tiered retention cycle in OpenObserve and backend storage: 180 days (6 months) hot searchable storage, and 365 days (1 year) compressed archival.
+- **Assigned Tracker**: [Issue #20](https://github.com/JJediny/LocalObserve/issues/20)
+
+### Requirement 3: Access Controls and RBAC for Log & Network Data
+- **M-26-14 Text Reference**: 
+  > *"In the event of a known or suspected compromise... provide logs and other relevant data to CISA and the Federal Bureau of Investigation (FBI) upon request... Logs are encrypted, access is granted just in time, permissions and workloads are regularly monitored and reviewed..."* (Par. 125-131, Appendix C)
+- **LocalObserve Implementation Plan**:
+  Introduce a Just-in-Time (JIT) access permission matrix in OpenObserve. Implement a standardized log export tool matching CISA/FBI ingestion schemas.
+- **Assigned Tracker**: [Issue #21](https://github.com/JJediny/LocalObserve/issues/21)
+
+### Requirement 4: Network Capture Controls, Minimization, and Retention
+- **M-26-14 Text Reference**: 
+  > *"Determining source and destination network address information, including protocols, ports, and session attributes... LRA will include guidance to ensure that logs will not capture or expose data in contravention of law. It will also advise agencies on how to protect the confidentiality and integrity of sensitive log data."* (Appendix A, Appendix B.5.b)
+- **LocalObserve Implementation Plan**:
+  Utilize `openobserve/goflow2` for structured NetFlow/sFlow logging. Apply regex filters at the collector layer to strip out user payloads (PII) to minimize incidental exposure.
+- **Assigned Tracker**: [Issue #22](https://github.com/JJediny/LocalObserve/issues/22)
+
+### Requirement 5: Tamper-Evident Audit Trail for Logging Pipeline & Config
+- **M-26-14 Text Reference**: 
+  > *"Logs are encrypted in transit and at rest, and regularly hashed for veracity."* (Appendix C - Log Management Level 3)
+- **LocalObserve Implementation Plan**:
+  Store logging pipeline configuration in git with strict branch protections. Compute SHA-256 hashes of rolled storage chunks and log them to a separate tamper-evident audit ledger.
+- **Assigned Tracker**: [Issue #23](https://github.com/JJediny/LocalObserve/issues/23)
+
+### Requirement 6: Monitoring, Alerting, and Incident Response Hooks for Logging Anomalies
+- **M-26-14 Text Reference**: 
+  > *"Generating appropriate automated alerts for all of the above... Logs generate actionable alerts covering at least 70% of baseline logging requirements, and detections are routinely evaluated and tuned."* (Appendix B.5.k, Appendix C)
+- **LocalObserve Implementation Plan**:
+  Deploy health monitoring alerting for `otelcol` and `OpenObserve` storage sinks. Define alerts for logging pipeline anomalies (e.g. dramatic drops in log ingestion rate).
+- **Assigned Tracker**: [Issue #24](https://github.com/JJediny/LocalObserve/issues/24)
+
+### Requirement 7: Data Protection for Stored Log & Capture Artifacts (Encryption & KMS)
+- **M-26-14 Text Reference**: 
+  > *"Logs are stored and encrypted at rest... Logs are encrypted in transit and at rest, and regularly hashed for veracity."* (Appendix C - Log Management Level 2 & 3)
+- **LocalObserve Implementation Plan**:
+  Enforce TLS 1.3 for all internal log ingestion streams. Configure AES-256 at-rest encryption inside OpenObserve backend blocks utilizing KMS integration for periodic key rotation.
+- **Assigned Tracker**: [Issue #25](https://github.com/JJediny/LocalObserve/issues/25)
+
+### Requirement 8: Documentation, Tests, and CI Checks for Compliance Controls
+- **M-26-14 Text Reference**: 
+  > *"Agencies must submit an Agency Logging Plan to OMB and CISA... Agencies will measure and report on progress in terms of the percentage of systems that are determined to be operating at each maturity level."* (Par. 105-121)
+- **LocalObserve Implementation Plan**:
+  Develop automated regression check files verifying structured JSON logging schema conformance and retention configurations in GitHub CI pipelines.
+- **Assigned Tracker**: [Issue #26](https://github.com/JJediny/LocalObserve/issues/26)