Secparse Ecosystem — Security Data Correlation & Aggregation Engine

Secparse is a high-performance, modular ecosystem for parsing, normalizing, and correlating security tool outputs into a unified, actionable attack graph. It transforms raw results from 29+ industry-standard tools into structured intelligence.

---

🏗 Ecosystem Architecture

The ecosystem is composed of four primary crates designed for maximum reuse and clean layering:

Crate	Purpose	Key Responsibilities
secparse-models	Schema Authority	Unified Findings, Asset/Service models, Attack Graph schema.
secparse	Parser Suite	Deterministic parsing of 29+ tools into canonical models.
secparse-enrich	Enrichment Engine	Provider-agnostic metadata augmentation (GeoIP, DNS).
secparse-correlation	Intelligence Engine	Entity resolution, Promotion Layer, and Attack Path reasoning.
secparse-cli	Production Entrypoint	Pipeline orchestration and CLI interface for automation.

---

🚀 Getting Started

1. Installation

Clone the repository and build the workspace:

cargo build --release

2. Basic CLI Usage

Run the end-to-end pipeline with GeoIP enrichment enabled:

./target/release/secparse-cli pipeline --input nmap_output.xml --enrich --geoip ./GeoLite2.mmdb

3. Standalone Enrichment

Enrich a JSON file of findings directly:

./target/release/secparse-cli enrich --input findings.json --geoip ./GeoLite2.mmdb

---

📖 Programmatic Usage

Each crate can be used independently of the CLI for custom integrations.

Parsing Logic (secparse)

Produce canonical findings or graph data from raw strings:

use secparse::{ParserRegistry, ParseOutput};

let registry = ParserRegistry::new();
let input = std::fs::read_to_string("nmap.xml")?;
let output = registry.detect_and_parse(&input)?; // Returns ParseOutput (Findings or Graph)

Correlation & Reasoning (secparse_correlation)

Enrich parsed results and build the attack graph:

use secparse_correlation::{Pipeline, CorrelationConfig, CorrelationInput};

let mut config = CorrelationConfig::default();
config.enable_enrichment = true;
config.geoip_db_path = Some("./GeoLite2.mmdb".to_string());

let mut pipeline = Pipeline::new(config);
let input = CorrelationInput {
    findings: raw_findings,
    graph_nodes: bloodhound_nodes,
    graph_edges: bloodhound_edges,
};

let result = pipeline.run(input)?; // Returns CorrelationResult with Graph and Metrics

Metadata Enrichment (secparse_enrich)

Manually augment findings with external intelligence:

use secparse_enrich::{EnrichmentEngine, builtins::MaxMindProvider};

let mut engine = EnrichmentEngine::new();
engine.register_provider(Box::new(MaxMindProvider::new("./GeoLite2.mmdb".into())?));

let enriched_findings = engine.enrich_batch(raw_findings);

---

🔒 Security & Data Integrity

Secparse follows a Zero Trust Input Handling policy:

• Authoritative Data: Direct graph data (e.g., BloodHound) is ingested with strict schema enforcement.
• Inferred Data: Findings from other tools (e.g., Hydra, SQLMap) are "promoted" into graph nodes/edges using a deterministic rules engine.
• Idempotency: Identical inputs always produce identical UUIDs (v5 namespacing) for stable entity tracking.

---

🛠 Supported Tools (29+)

• Network: Nmap (XML/Text), Naabu, Hydra, Masscan.
• Web: Nuclei, Httpx, WhatWeb, Wafw00f, Katana, GAU, Ffuf, Gobuster, Nikto, Sqlmap, Burp Suite, OWASP ZAP, Dirsearch.
• Secrets: TruffleHog (v3), Gitleaks.
• Cloud: S3Scanner.
• Infrastructure: Ldapsearch, NetExec (CrackMapExec), BloodHound (SharpHound), Enum4linux-ng.
• Recon: Subfinder, Amass, TLSx.
• Vuln Mgmt: Nessus.

---

📊 Output Schema

The pipeline command produces a unified JSON output containing correlated intelligence from all sources.

Core Output Variables

Variable	Description	Key Fields
assets	Unified network and identity entities.	id, ips, hostnames, os, risk_score, sources
findings	Deduplicated findings from all 29+ tools.	id, severity, category, description, evidence
graph	Correlated attack graph and calculated paths.	nodes, edges, paths (scored), metadata

Attack Graph Metadata

The graph.metadata field provides immediate summary metrics for the environment:

• node_count: Total identified entities.
• edge_count: Total discovered relationships.
• max_risk: Highest calculated risk score in the environment.

---

🧪 Testing

We provide two certification scripts to verify ecosystem integrity across platforms:

• Windows: ./test.ps1
• Linux/WSL: ./test.sh

---

⚖ License

Distributed under the MIT License. See LICENSE for more information.