Create SECURITY.md for security policy - #2
Open
Krishcalin wants to merge 1 commit into
Open
Conversation
Add a security policy document outlining supported versions and vulnerability reporting.
Krishcalin
added a commit
that referenced
this pull request
Aug 3, 2026
…se 1, Backbone #2) A CIM data model is a named schema (Authentication, Network, Web, DNS, Endpoint, Change, Malware, IDS, Industrial, Email, Vulnerability) plus a membership rule deciding which events belong to it. Detections and `datamodel:` searches bind to a model, not a vendor, so onboarding a source becomes a registry edit rather than a code change. This is the second backbone of docs/SPLUNK_TRANSFORMATION_ROADMAP.md, after LOQL (ec1ed09). New pure package app/cim/: - spec.py frozen contract (CimField/CimTerm/CimClause/CimModel/CimRegistry) - registry.py loads + validates models.yaml, failing loudly at load rather than as broken SQL later - sql.py the ONLY place CIM becomes SQL — per-model views, whitelisted identifiers, quoted labels, escaped literals - match.py the pure-Python membership evaluator: the ONE implementation - models.yaml 11 models, as DATA MEMBERSHIP IS A PLAIN COLUMN, NOT A GENERATED ONE. events.cim_models is text[] written by match.tags_for at ingest, mirroring how search_tsv is written by tsv_text. A STORED generated column is a dead end on PostgreSQL 16 (which docker-compose pins): ADD COLUMN IF NOT EXISTS freezes the generation expression, and ALTER COLUMN ... SET EXPRESSION is 17+, so every models.yaml edit would be inert for new rows as well as old. It is also the only design with ONE evaluator — detection needs membership BEFORE the INSERT (evaluate_event runs per event; the INSERT is per 5000-event chunk), so a generated column would mean two implementations that provably diverge on raw-key casing. A membership edit now takes effect on the next ingest with no DDL; history is corrected by db.backfill_cim, chunked and resumable, from the admin page. THE REGISTRY WAS MEASURED, NOT WRITTEN FROM VENDOR DOCS. Simulated over the 33 shipped samples, the first draft left 28 of 97 events untagged and was wrong where it mattered: all three Windows clauses gated on raw['event_id'], a key no parser writes (the Get-WinEvent key is "Id"), so the highest-value source in the SIEM landed in zero models — silently, because a missing jsonb key is NULL, not an error. Both Palo Alto parsers key log_type on the PAN *subtype*, so no PAN traffic reached Network while PAN IPS alerts were tagged as scanner Vulnerabilities — a false positive and a false negative from one clause. The whole OT/ICS surface matched nothing. 13 raw: field mappings were non-null on zero events. All fixed against measured parser output, and tests/test_cim.py's corpus test now enforces it per source. Also: windows_security.py writes the canonical id back as raw["event_id"] (multi-key COALESCE terms handle the Id/EventID/"Event ID" export shapes regardless). MIGRATION: dedup_hash is derived from raw, so every Windows event changes identity — re-uploading a previously-ingested Windows file inserts duplicates instead of deduping. One-time, pre-1.0. LOQL gains `| datamodel <Name>` and `from datamodel:<Name>`, compiling to the GIN-indexed `cim_models @> ARRAY[%s]` with the tag BOUND, projecting the model's field names. Detection rules gain `datamodels:`; five rules converted. New /datamodels reference page. docs/CIM.md. Tests: 535 -> 984 unit. Hardened by four adversarial review rounds (14 defects, then 11, then a mutation audit that found 5 vacuous tests). Two blockers, both found and fixed: a race publishing the registry-resolution flag before resolving, which cached a permanently dead gate for a datamodel-bound rule; and the SET LOCAL defect fixed in the previous commit. HONEST LIMITS: none of the 68 integration tests has ever executed — the job was green while running nothing until the previous commit closed that. The next CI run is their first, so the GIN index actually serving @>, partition inheritance, the 11 view bodies, backfill chunking and init_cim reconciliation are unproven. Correlation rules are not datamodel-bound (they filter in SQL, never through match_rule). Email and Vulnerability await a source and are labelled as such. app/parsers/sysmon.py has the same event-id writeback gap, deliberately left for a follow-up since fixing it means a second dedup migration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a security policy document outlining supported versions and vulnerability reporting.