diff --git a/docs/design/README.md b/docs/design/README.md new file mode 100644 index 000000000..1ac6c1287 --- /dev/null +++ b/docs/design/README.md @@ -0,0 +1,234 @@ +# Trustify Architecture Documentation + +This document provides an organized overview of Trustify's architecture from multiple perspectives. Each view serves a different purpose and audience. + +## Quick Start Guide + +**New to Trustify?** Start with the [Domain Model](domain-model.md) to understand business concepts, then review the [C4 Architecture](architecture.md) to see how components work together. + +**Working on database changes?** Go directly to the [Data Model](data-model.md). + +**Designing new features?** Use the [Domain Model](domain-model.md) for concepts and [C4 Architecture](architecture.md) for service boundaries. + +## Architecture Views + +Trustify's architecture is documented through three complementary views following industry-standard modeling approaches: + +- **[Domain Model](domain-model.md)** - Business concepts and relationships + - **Audience:** Product managers, domain experts, developers + - **When to Use:** Understanding terminology, feature planning, API design + +- **[C4 Architecture](architecture.md)** - System decomposition and component interactions + - **Audience:** Architects, developers, DevOps + - **When to Use:** System design, deployment planning, integration work + +- **[Data Model](data-model.md)** - Physical database schema + - **Audience:** DBAs, backend developers + - **When to Use:** Database queries, migrations, schema changes + +### Domain Model (Business View) + +**Standard:** Domain-Driven Design (DDD) concepts +[View Domain Model →](domain-model.md) + +High-level business entities organized into logical domains: + +- **Supply Chain Intelligence**: SBOM, Package, File, License +- **Security Intelligence**: Advisory, Vulnerability, Weakness +- **Product Management**: Organization, Product, ProductVersion +- **Data Pipeline**: Importer, Ingestor, SourceDocument, Storage +- **Analysis & Query**: Analysis Graph (DAG), Query Service + +### C4 Architecture (System View) + +**Standard:** C4 Model (Context, Container, Component, Code) +[View C4 Architecture →](architecture.md) + +Hierarchical view showing system decomposition at multiple zoom levels: + +- **Level 1 - System Context**: External systems and users +- **Level 2 - Container Diagram**: Runtime components and interactions +- **Level 3 - Component Diagrams**: Internal structure of containers + +### Data Model (Implementation View) + +**Standard:** Entity-Relationship Diagram (ERD) +[View Data Model →](data-model.md) + +PostgreSQL database schema with 50+ tables: + +- Primary keys (PK) and foreign keys (FK) explicitly marked +- Cardinality shown with crow's foot notation +- Data types specified for each column +- Organized by domain + +### Relationship Between Views + +```mermaid +graph LR + A[Domain Model
Business Concepts] -->|maps to| B[C4 Architecture
System Components] + B -->|implements| C[Data Model
Database Schema] + A -.->|informs| C + + style A fill:#e1f5e1,stroke:#4caf50 + style B fill:#e1f0ff,stroke:#2196f3 + style C fill:#fff0e1,stroke:#ff9800 +``` + +- **Domain Model → C4 Architecture**: Business entities map to services and components +- **C4 Architecture → Data Model**: Services read/write database tables +- **Domain Model → Data Model**: Conceptual entities normalize into database tables + +--- + +## Key Architectural Concepts + +### Two-Model Architecture + +Trustify maintains two distinct graph representations: + +1. **Logical Model** (Database): Preserves exact SBOM relationships with original directionality +2. **Conceptual Model** (Analysis Graph): Normalized DAG view for efficient traversal queries + +See [ADR-00002: Analysis Graph API](../adrs/00002-analysis-graph.md) for details. + +### Modulith Structure + +Services are organized into modules following a consistent pattern: + +- `endpoints/` - HTTP API handlers +- `model/` - Serializable data models +- `service/` - Business logic layer +- Tests co-located with implementation + +See [modules/README.md](../../modules/README.md) for module conventions. + +### PURL Hierarchy + +Packages are identified using a three-tier PURL structure: + +- **BasePurl**: type, namespace, name +- **VersionedPurl**: adds version to BasePurl +- **QualifiedPurl**: adds qualifiers (platform, arch, etc.) + +This enables efficient deduplication and version range queries. + +--- + +## Subsystem Design Documentation + +These documents provide deep dives into specific subsystems and design patterns: + +### Core Subsystems + +| Document | Topics Covered | +| ------------------------------------ | ----------------------------------------------------------------------- | +| [**Importer System**](importer.md) | Configuration, scheduling, job execution, error handling | +| [**SBOM Storage**](sbom.md) | Node types, graph structure, PURL/CPE resolution, cross-SBOM references | +| [**Product Model**](products.md) | Product hierarchy, version ranges, SBOM associations, status tracking | +| [**Labels System**](labels.md) | Flexible metadata, JSONB implementation, query patterns | +| [**API Patterns**](fetch-service.md) | Head/Summary/Details DTOs, relationship traversal, pagination | + +### Architecture Decision Records (ADRs) + +ADRs document significant architectural decisions with context, rationale, and consequences: + +- [**ADR-00001**: Graph Analytics](../adrs/00001-graph-analytics.md) - Foundation of graph-based analysis +- [**ADR-00002**: Analysis Graph API](../adrs/00002-analysis-graph.md) - Logical vs conceptual model separation +- [**ADR-00003**: External References](../adrs/00003-external-references.md) - Cross-SBOM reference handling +- [**ADR-00004**: Advisory Scores](../adrs/00004-advisory-scores.md) - CVSS score management +- [**ADR-00009**: Conservative PURL Garbage Collection](../adrs/00009-conservative-purl-garbage-collection.md) - Data retention strategy +- [**ADR-00011**: CSAF Remediation](../adrs/00011-csaf-remediation.md) - Advisory remediation handling + +[View all ADRs →](../adrs/) + +ADRs are immutable once accepted. New decisions that supersede old ones reference the original ADR rather than modifying it. + +## Module Structure + +Trustify follows a **modulith architecture** where the codebase is organized into cohesive modules with clear boundaries, deployed as a single application. + +**Module Layout Standard:** + +Each module in `modules/` follows a consistent structure: + +``` +modules/my_module/ +├── src/ +│ ├── endpoints/ # HTTP API handlers (Actix-web routes) +│ ├── model/ # Serializable DTOs for API responses +│ ├── service/ # Business logic layer +│ └── lib.rs # Module entry point +└── Cargo.toml +``` + +**Key Principles:** + +- **Endpoints**: Pure routing and HTTP concerns, thin handlers that delegate to services +- **Models**: Data Transfer Objects (DTOs) with `serde` and `utoipa` annotations for API documentation +- **Services**: Core business logic, accepts database connections, returns domain results +- **Co-located Tests**: Tests live alongside implementation for better maintainability + +**Benefits:** + +- **Clear Boundaries**: Modules encapsulate related functionality +- **Independent Testing**: Each module can be tested in isolation +- **Flexible Deployment**: Could be extracted to separate services if needed +- **Team Ownership**: Teams can own specific modules +- **Reduced Coupling**: Explicit dependencies between modules + +**Current Modules:** + +- `fundamental/` - Core entities (Advisory, SBOM, PURL, Vulnerability, Product) +- `ingestor/` - Document parsing and storage +- `importer/` - Scheduled import management +- `analysis/` - Dependency graph analysis +- `storage/` - Storage backend abstraction +- `graphql/` - GraphQL API alternative +- `ui/` - Web interface +- `user/` - User preferences management + +[Learn more about module conventions →](../../modules/README.md) + +## API Documentation + +Trustify provides comprehensive OpenAPI 3.0 documentation for all REST endpoints, automatically generated from code annotations using `utoipa`. + +[View OpenAPI Specification →](../../openapi.yaml) + +The API follows REST principles with consistent patterns as documented in [API Patterns](fetch-service.md). + +## Contributing to Documentation + +### When to Update Documentation + +- **Domain Model**: When adding new business entities or relationships +- **C4 Architecture**: When adding services, changing interactions, or deployment +- **Data Model**: After migrations that add/modify tables +- **Keep views in sync** to maintain documentation quality + +### Generating Diagram Images + +#### Using Mermaid CLI + +Install from https://github.com/mermaid-js/mermaid-cli or use the pre-built Docker image. + +**Basic command:** + +```bash +mmdc -i domain-model.md -o domain-model.png +``` + +**With configuration:** + +```bash +mmdc -c mermaid.json -i domain-model.md -o domain-model.png +``` + +**For large diagrams (data model):** + +```bash +mmdc -c mermaid.json -i data-model.md -o data-model.png --width 1200 --height 1600 --scale 6 +``` + +**Note:** When providing a markdown file, `mmdc` extracts all Mermaid diagrams and saves them as individual files with incremental suffixes (e.g., `domain-model-1.png`, `domain-model-2.png`). diff --git a/docs/design/architecture.md b/docs/design/architecture.md new file mode 100644 index 000000000..010414708 --- /dev/null +++ b/docs/design/architecture.md @@ -0,0 +1,218 @@ +# Trustify Architecture + +This document presents Trustify's architecture using the C4 model (Context, Container, Component, Code). + +## Level 1: System Context + +Shows how Trustify fits into the broader security and supply chain ecosystem. + +```mermaid +C4Context + title System Context - Trustify SBOM and Security Advisory Platform + + Person(securityAnalyst, "Security Analyst", "Reviews vulnerabilities and manages security advisories") + Person(developer, "Developer", "Queries package vulnerabilities and SBOM data") + Person(complianceOfficer, "Compliance Officer", "Ensures software license and security compliance") + + System(trustify, "Trustify", "SBOM and Security Advisory Management Platform - ingests, stores, and analyzes software bills of materials and security advisories") + + System_Ext(sbomSources, "SBOM Sources", "CycloneDX, SPDX documents from builds, registries") + System_Ext(advisorySources, "Advisory Sources", "CSAF, OSV, CVE feeds from vendors, security organizations") + System_Ext(weaknessSources, "CWE Database", "Common Weakness Enumeration data") + SystemDb_Ext(oidcProvider, "OIDC Provider", "Keycloak, Auth0, etc - authentication and authorization") + System_Ext(storageBackend, "Object Storage", "S3, MinIO - stores original documents") + + Rel(developer, trustify, "Queries vulnerabilities and dependencies", "HTTPS/REST") + Rel(securityAnalyst, trustify, "Uploads and analyzes security data", "HTTPS/REST") + Rel(complianceOfficer, trustify, "Generates compliance reports", "HTTPS/REST") + + Rel(trustify, sbomSources, "Imports SBOMs", "HTTPS") + Rel(trustify, advisorySources, "Imports advisories", "HTTPS") + Rel(trustify, weaknessSources, "Imports CWE data", "HTTPS") + Rel(trustify, oidcProvider, "Authenticates users", "OIDC") + Rel(trustify, storageBackend, "Stores/retrieves documents", "S3 API") + + UpdateRelStyle(developer, trustify, $offsetY="-50") + UpdateRelStyle(securityAnalyst, trustify, $offsetX="-40") + UpdateRelStyle(complianceOfficer, trustify, $offsetY="-50") +``` + +## Level 2: Container Diagram + +Shows the high-level technology and architectural components of Trustify. + +```mermaid +C4Container + title Container Diagram - Trustify Platform + + Person(user, "User", "Security analyst, developer, or compliance officer") + System_Ext(oidc, "OIDC Provider", "Authentication") + System_Ext(externalSources, "External Sources", "SBOM/Advisory feeds") + SystemDb_Ext(objectStorage, "Object Storage", "S3/MinIO") + + Container_Boundary(trustify, "Trustify Platform") { + Container(webUI, "Web UI", "React/TypeScript", "Provides web interface for browsing SBOMs, vulnerabilities, and advisories") + Container(apiServer, "API Server", "Rust/Actix-web", "REST API endpoints for all operations") + Container(importerService, "Importer Service", "Rust/Tokio", "Scheduled fetching of SBOMs and advisories from external sources") + Container(ingestorService, "Ingestor Service", "Rust", "Parses and stores SBOM/advisory documents into database") + Container(analysisService, "Analysis Service", "Rust/Petgraph", "Builds and queries dependency DAG, vulnerability analysis") + ContainerDb(postgres, "PostgreSQL Database", "PostgreSQL 17", "Stores entities: advisories, vulnerabilities, SBOMs, packages, relationships") + Container(graphqlAPI, "GraphQL API", "Rust/async-graphql", "Alternative query interface") + } + + Rel(user, webUI, "Uses", "HTTPS") + Rel(user, apiServer, "Uses", "HTTPS/REST") + Rel(webUI, apiServer, "Calls", "JSON/HTTPS") + Rel(user, oidc, "Authenticates", "OIDC") + Rel(apiServer, oidc, "Validates tokens", "OIDC") + + Rel(importerService, externalSources, "Fetches documents", "HTTPS") + Rel(importerService, ingestorService, "Sends documents") + Rel(ingestorService, objectStorage, "Stores original docs", "S3 API") + Rel(ingestorService, postgres, "Writes parsed data", "SQL") + + Rel(apiServer, ingestorService, "Upload documents") + Rel(apiServer, analysisService, "Query analysis") + Rel(apiServer, postgres, "Reads", "SQL") + Rel(analysisService, postgres, "Reads graph data", "SQL") + Rel(graphqlAPI, postgres, "Reads", "SQL") + + UpdateRelStyle(user, webUI, $offsetY="-40") + UpdateRelStyle(webUI, apiServer, $offsetY="-40") + UpdateRelStyle(importerService, ingestorService, $offsetX="-50") +``` + +## Level 3: Component Diagram - API Server + +Shows the internal components and services within the API Server. + +```mermaid +C4Component + title Component Diagram - API Server (Trustify Core) + + Container(webUI, "Web UI", "React", "User interface") + Container(ingestor, "Ingestor", "Rust", "Document ingestion") + Container(analysis, "Analysis Service", "Rust", "Graph analysis") + ContainerDb(db, "PostgreSQL", "Database", "Persistent storage") + + Container_Boundary(apiServer, "API Server") { + Component(authMiddleware, "Auth Middleware", "Actix-web middleware", "Validates OIDC tokens and enforces authorization") + + Component(advisoryEndpoints, "Advisory Endpoints", "Actix-web handlers", "CRUD for security advisories") + Component(sbomEndpoints, "SBOM Endpoints", "Actix-web handlers", "CRUD for SBOMs") + Component(purlEndpoints, "PURL Endpoints", "Actix-web handlers", "Query packages by PURL") + Component(vulnEndpoints, "Vulnerability Endpoints", "Actix-web handlers", "Query CVEs and vulnerabilities") + Component(productEndpoints, "Product Endpoints", "Actix-web handlers", "Manage products and versions") + Component(analysisEndpoints, "Analysis Endpoints", "Actix-web handlers", "Component dependency traversal") + + Component(advisoryService, "Advisory Service", "Business logic", "Advisory operations and queries") + Component(sbomService, "SBOM Service", "Business logic", "SBOM operations and queries") + Component(purlService, "PURL Service", "Business logic", "Package URL operations") + Component(vulnService, "Vulnerability Service", "Business logic", "Vulnerability lookups") + Component(productService, "Product Service", "Business logic", "Product management") + + Component(queryEngine, "Query Engine", "TrustifyQuery", "Flexible query DSL parser and executor") + Component(graphService, "Graph Service", "SeaORM", "Database access layer") + } + + Rel(webUI, authMiddleware, "All requests", "JSON/HTTPS") + + Rel(authMiddleware, advisoryEndpoints, "Routes to") + Rel(authMiddleware, sbomEndpoints, "Routes to") + Rel(authMiddleware, purlEndpoints, "Routes to") + Rel(authMiddleware, vulnEndpoints, "Routes to") + Rel(authMiddleware, productEndpoints, "Routes to") + Rel(authMiddleware, analysisEndpoints, "Routes to") + + Rel(advisoryEndpoints, advisoryService, "Uses") + Rel(sbomEndpoints, sbomService, "Uses") + Rel(purlEndpoints, purlService, "Uses") + Rel(vulnEndpoints, vulnService, "Uses") + Rel(productEndpoints, productService, "Uses") + Rel(analysisEndpoints, analysis, "Uses") + + Rel(advisoryService, queryEngine, "Uses") + Rel(sbomService, queryEngine, "Uses") + Rel(purlService, queryEngine, "Uses") + Rel(vulnService, queryEngine, "Uses") + + Rel(advisoryService, graphService, "Uses") + Rel(sbomService, graphService, "Uses") + Rel(purlService, graphService, "Uses") + Rel(vulnService, graphService, "Uses") + Rel(productService, graphService, "Uses") + + Rel(advisoryEndpoints, ingestor, "Upload") + Rel(sbomEndpoints, ingestor, "Upload") + + Rel(graphService, db, "Reads/writes", "SQL") + + UpdateRelStyle(authMiddleware, advisoryEndpoints, $offsetX="-100") + UpdateRelStyle(authMiddleware, sbomEndpoints, $offsetX="-80") +``` + +## Level 4: Component Diagram - Ingestor Service + +Shows how document ingestion and parsing works. + +```mermaid +C4Component + title Component Diagram - Ingestor Service + + Container(apiServer, "API Server", "Rust", "Receives uploads") + Container(importer, "Importer Service", "Rust", "Scheduled imports") + Container(storage, "Object Storage", "S3", "Document storage") + ContainerDb(db, "PostgreSQL", "Database", "Parsed entities") + Container(analysis, "Analysis Service", "Rust", "Graph cache") + + Container_Boundary(ingestor, "Ingestor Service") { + Component(formatDetector, "Format Detector", "Pattern matching", "Detects SBOM/advisory format") + + Component(spdxParser, "SPDX Parser", "spdx-rs", "Parses SPDX SBOMs") + Component(cyclonedxParser, "CycloneDX Parser", "cyclonedx-bom", "Parses CycloneDX SBOMs") + Component(csafParser, "CSAF Parser", "csaf crate", "Parses CSAF advisories") + Component(osvParser, "OSV Parser", "JSON parser", "Parses OSV advisories") + Component(cveParser, "CVE Parser", "JSON parser", "Parses CVE records") + + Component(sbomGraph, "SBOM Graph Builder", "SeaORM", "Creates SBOM entities and relationships") + Component(advisoryGraph, "Advisory Graph Builder", "SeaORM", "Creates advisory and vulnerability entities") + Component(purlResolver, "PURL Resolver", "PURL hierarchy", "Resolves BasePurl, VersionedPurl, QualifiedPurl") + Component(cpeResolver, "CPE Resolver", "CPE parsing", "Resolves CPE identifiers") + Component(statusResolver, "Status Resolver", "Version ranges", "Determines affected package versions") + + Component(graphLoader, "Graph Cache Loader", "Async tasks", "Loads SBOMs into analysis cache") + } + + Rel(apiServer, formatDetector, "Sends document bytes") + Rel(importer, formatDetector, "Sends document bytes") + + Rel(formatDetector, spdxParser, "SPDX documents") + Rel(formatDetector, cyclonedxParser, "CycloneDX documents") + Rel(formatDetector, csafParser, "CSAF documents") + Rel(formatDetector, osvParser, "OSV documents") + Rel(formatDetector, cveParser, "CVE documents") + + Rel(spdxParser, sbomGraph, "Parsed SBOM") + Rel(cyclonedxParser, sbomGraph, "Parsed SBOM") + Rel(csafParser, advisoryGraph, "Parsed advisory") + Rel(osvParser, advisoryGraph, "Parsed advisory") + Rel(cveParser, advisoryGraph, "Parsed advisory") + + Rel(sbomGraph, purlResolver, "Package identifiers") + Rel(sbomGraph, cpeResolver, "CPE identifiers") + Rel(advisoryGraph, purlResolver, "Affected packages") + Rel(advisoryGraph, statusResolver, "Version ranges") + + Rel(sbomGraph, db, "Writes entities", "SQL") + Rel(advisoryGraph, db, "Writes entities", "SQL") + Rel(purlResolver, db, "Writes/reads", "SQL") + Rel(cpeResolver, db, "Writes/reads", "SQL") + Rel(statusResolver, db, "Writes", "SQL") + + Rel(formatDetector, storage, "Stores original", "S3 API") + Rel(sbomGraph, graphLoader, "Triggers load") + Rel(graphLoader, analysis, "Loads into cache") + + UpdateRelStyle(formatDetector, spdxParser, $offsetY="-20") + UpdateRelStyle(formatDetector, cyclonedxParser, $offsetY="-10") +``` diff --git a/docs/design/data-model.md b/docs/design/data-model.md new file mode 100644 index 000000000..15cab9623 --- /dev/null +++ b/docs/design/data-model.md @@ -0,0 +1,436 @@ +# Trustify Data Model + +This is the physical database schema showing tables and their relationships. + +```mermaid +--- +title: Trustify Data model +--- +erDiagram + %% Core Documents + source_document { + uuid id PK + varchar sha256 + varchar sha384 + varchar sha512 + bigint size + timestamp ingested + } + + %% Advisory Domain + advisory { + uuid id PK + uuid source_document_id FK + uuid issuer_id FK + varchar identifier + varchar document_id + varchar title + varchar version + timestamp published + timestamp modified + timestamp withdrawn + jsonb labels + bool deprecated + } + + vulnerability { + varchar id PK + varchar title + text[] cwes + timestamp published + timestamp modified + timestamp withdrawn + timestamp reserved + timestamp timestamp + } + + advisory_vulnerability { + uuid advisory_id PK,FK + varchar vulnerability_id PK,FK + varchar title + varchar summary + varchar description + text[] cwes + timestamp discovery_date + timestamp release_date + timestamp reserved_date + } + + vulnerability_description { + uuid id PK + varchar vulnerability_id FK + uuid advisory_id FK + varchar lang + varchar description + timestamp timestamp + } + + weakness { + text id PK + text description + text extended_description + text[] child_of + text[] parent_of + text[] starts_with + text[] can_follow + text[] can_precede + text[] required_by + text[] requires + text[] can_also_be + text[] peer_of + } + + cvss3 { + uuid advisory_id PK,FK + varchar vulnerability_id PK,FK + int minor_version + cvss3_av av + cvss3_ac ac + cvss3_pr pr + cvss3_ui ui + cvss3_s s + cvss3_c c + cvss3_i i + cvss3_a a + double_precision score + cvss3_severity severity + } + + cvss4 { + uuid advisory_id PK,FK + varchar vulnerability_id PK,FK + int minor_version + cvss4_av av + cvss4_ac ac + cvss4_at at + cvss4_pr pr + cvss4_ui ui + cvss4_vc vc + cvss4_vi vi + cvss4_va va + cvss4_sc sc + cvss4_si si + cvss4_sa sa + } + + %% PURL Hierarchy + base_purl { + uuid id PK + varchar type + varchar namespace + varchar name + timestamp timestamp + } + + versioned_purl { + uuid id PK + uuid base_purl_id FK + varchar version + timestamp timestamp + } + + qualified_purl { + uuid id PK + uuid versioned_purl_id FK + jsonb qualifiers + jsonb purl + timestamp timestamp + } + + %% SBOM Domain + sbom { + uuid sbom_id PK + uuid source_document_id FK + varchar node_id + varchar document_id + timestamp published + varchar[] authors + jsonb labels + text[] data_licenses + } + + sbom_node { + uuid sbom_id PK,FK + varchar node_id PK + varchar name + } + + sbom_package { + uuid sbom_id PK,FK + varchar node_id PK,FK + varchar version + } + + sbom_file { + uuid sbom_id PK,FK + varchar node_id PK,FK + } + + sbom_node_checksum { + uuid sbom_id PK,FK + varchar node_id PK,FK + varchar type PK + varchar value + } + + sbom_package_purl_ref { + uuid sbom_id PK,FK + varchar node_id PK,FK + uuid qualified_purl_id FK + } + + sbom_package_cpe_ref { + uuid sbom_id PK,FK + varchar node_id PK,FK + uuid cpe_id FK + } + + sbom_external_node { + uuid sbom_id PK,FK + varchar node_id PK + varchar external_doc_ref + varchar external_node_ref + int external_type + uuid target_sbom_id FK + int discriminator_type + varchar discriminator_value + } + + package_relates_to_package { + uuid sbom_id PK,FK + varchar left_node_id PK,FK + varchar right_node_id PK,FK + int relationship FK + } + + relationship { + int id PK + varchar description + } + + %% CPE + cpe { + uuid id PK + varchar part + varchar vendor + varchar product + varchar version + varchar update + varchar edition + varchar language + varchar sw_edition + varchar target_sw + varchar target_hw + varchar other + } + + %% Product Domain + organization { + uuid id PK + varchar name + varchar cpe_key + varchar website + } + + product { + uuid id PK + uuid vendor_id FK + varchar name + varchar cpe_key + } + + product_version { + uuid id PK + uuid product_id FK + uuid sbom_id FK + varchar version + timestamp timestamp + } + + product_version_range { + uuid id PK + uuid product_id FK + uuid version_range_id FK + varchar cpe_key + } + + %% Status + status { + uuid id PK + varchar slug + varchar name + varchar description + } + + purl_status { + uuid id PK + uuid advisory_id FK + varchar vulnerability_id FK + uuid status_id FK + uuid base_purl_id FK + uuid version_range_id FK + uuid context_cpe_id FK + } + + product_status { + uuid id PK + uuid advisory_id FK + varchar vulnerability_id FK + uuid status_id FK + uuid product_version_range_id FK + uuid context_cpe_id FK + varchar package + } + + %% Version Management + version_scheme { + varchar id PK + varchar name + varchar description + } + + version_range { + uuid id PK + varchar version_scheme_id FK + varchar low_version + bool low_inclusive + varchar high_version + bool high_inclusive + } + + %% License + license { + uuid id PK + varchar text + text[] spdx_licenses + text[] spdx_license_exceptions + } + + purl_license_assertion { + uuid id PK + uuid license_id FK + uuid sbom_id FK + uuid versioned_purl_id FK + } + + cpe_license_assertion { + uuid id PK + uuid license_id FK + uuid sbom_id FK + uuid cpe_id FK + } + + %% Importer + importer { + varchar name PK + uuid revision + int state + timestamp last_change + varchar last_error + timestamp last_success + timestamp last_run + jsonb continuation + jsonb configuration + int progress_current + int progress_total + varchar progress_message + } + + importer_report { + uuid id PK + varchar importer FK + timestamp creation + varchar error + jsonb report + } + + %% User + user_preferences { + varchar user_id PK + varchar key PK + uuid revision + jsonb data + } + + %% Relationships - Advisory Domain + advisory ||--o{ source_document : "stored_in" + advisory ||--o| organization : "issued_by" + advisory ||--o{ advisory_vulnerability : "has" + advisory ||--o{ cvss3 : "has_scores" + advisory ||--o{ cvss4 : "has_scores" + advisory ||--o{ vulnerability_description : "describes" + advisory ||--o{ purl_status : "affects_purl" + advisory ||--o{ product_status : "affects_product" + + vulnerability ||--o{ advisory_vulnerability : "reported_in" + vulnerability ||--o{ vulnerability_description : "has_descriptions" + vulnerability ||--o{ cvss3 : "scored_by" + vulnerability ||--o{ cvss4 : "scored_by" + vulnerability ||--o{ purl_status : "identified_in" + vulnerability ||--o{ product_status : "identified_in" + + weakness ||--o{ vulnerability : "categorizes" + + %% Relationships - PURL Hierarchy + base_purl ||--o{ versioned_purl : "has_versions" + base_purl ||--o{ purl_status : "status_applies_to" + + versioned_purl ||--o{ qualified_purl : "has_qualifiers" + versioned_purl ||--o{ purl_license_assertion : "licensed_as" + + qualified_purl ||--o{ sbom_package_purl_ref : "referenced_in" + + %% Relationships - SBOM + sbom ||--o{ source_document : "stored_in" + sbom ||--o{ sbom_node : "contains" + sbom ||--o{ sbom_package : "has_packages" + sbom ||--o{ sbom_file : "has_files" + sbom ||--o{ purl_license_assertion : "asserts_purl_license" + sbom ||--o{ cpe_license_assertion : "asserts_cpe_license" + sbom ||--o{ product_version : "documents" + + sbom_node ||--o| sbom_package : "is_package" + sbom_node ||--o{ sbom_node_checksum : "has_checksums" + sbom_node ||--o| sbom_file : "is_file" + sbom_node ||--o{ sbom_external_node : "referenced_by" + sbom_node ||--o{ sbom_external_node : "references_external" + + sbom_package ||--o{ sbom_package_purl_ref : "identified_by_purl" + sbom_package ||--o{ sbom_package_cpe_ref : "identified_by_cpe" + sbom_package ||--o{ package_relates_to_package : "defines_relationships" + + relationship ||--o{ package_relates_to_package : "typed_by" + + %% Relationships - CPE + cpe ||--o{ sbom_package_cpe_ref : "referenced_in" + cpe ||--o{ cpe_license_assertion : "licensed_as" + cpe ||--o{ purl_status : "context_for_purl" + cpe ||--o{ product_status : "context_for_product" + + %% Relationships - Product + organization ||--o{ product : "owns" + organization ||--o{ advisory : "issues" + + product ||--o{ product_version : "has_versions" + product ||--o{ product_version_range : "has_version_ranges" + + product_version_range ||--o{ product_status : "status_applies_to" + product_version_range ||--o{ version_range : "uses_range" + + %% Relationships - Status + status ||--o{ purl_status : "status_type" + status ||--o{ product_status : "status_type" + + %% Relationships - Version + version_scheme ||--o{ version_range : "schemes" + + version_range ||--o{ product_version_range : "used_by_product" + version_range ||--o{ purl_status : "constrains_purl" + + %% Relationships - License + license ||--o{ purl_license_assertion : "asserted_for_purl" + license ||--o{ cpe_license_assertion : "asserted_for_cpe" + + %% Relationships - Importer + importer ||--o{ importer_report : "generates" +``` diff --git a/docs/design/domain-model.md b/docs/design/domain-model.md new file mode 100644 index 000000000..08dd98e81 --- /dev/null +++ b/docs/design/domain-model.md @@ -0,0 +1,92 @@ +# Trustify Domain Model + +This is a high-level conceptual view of the core domain objects and their relationships. + +```mermaid +--- +title: Trustify Domain model +--- +flowchart TB + subgraph Supply["Supply Chain Intelligence"] + SBOM[SBOM
---
Software Bill of Materials
inventory of components] + Package[Package
---
PURL: type, namespace, name, version
qualifiers, CPE] + File[File
---
name, path, checksums] + License[License
---
SPDX identifier] + Relationship[Relationship
---
DEPENDS, CONTAINS, etc
package dependencies] + end + + subgraph Security["Security Intelligence"] + Advisory[Advisory
---
CSAF, OSV, etc
security advisory] + Vulnerability[Vulnerability
---
CVE, GHSA, etc
CVSS scores] + Weakness[Weakness
---
CWE classification] + Status[Status
---
fixed, affected, not-affected] + end + + subgraph Products["Product Management"] + Organization[Organization
---
Vendor/Company] + Product[Product
---
name, CPE] + ProductVersion[ProductVersion
---
version, status] + end + + subgraph DataPipeline["Data Pipeline"] + Importer[Importer
---
scheduled fetching
configuration, state] + Ingestor[Ingestor
---
parsing & storage
SBOM/Advisory processing] + SourceDocument[SourceDocument
---
SHA256, content
original documents] + Storage[Storage
---
file storage backend
S3, filesystem] + end + + subgraph Analysis["Analysis & Query"] + AnalysisGraph[Analysis Graph
---
normalized DAG view
component dependencies] + Query[Query Service
---
search & filter
pagination] + end + + %% SBOM relationships + SBOM -->|contains| Package + SBOM -->|contains| File + SBOM -->|references| SBOM + Package -->|relates via| Relationship + Relationship -->|connects| Package + Package -->|has| License + + %% Security relationships + Advisory -->|reports| Vulnerability + Advisory -->|affects with status| Package + Advisory -->|affects with status| Product + Status -->|qualifies| Advisory + Vulnerability -->|categorized by| Weakness + + %% Product relationships + Organization -->|owns| Product + Organization -->|issues| Advisory + Product -->|has versions| ProductVersion + ProductVersion -.->|documented by| SBOM + + %% Data pipeline + Importer -->|schedules| Ingestor + Ingestor -->|parses & stores| SourceDocument + Ingestor -->|creates| SBOM + Ingestor -->|creates| Advisory + SourceDocument -->|stored in| Storage + Storage -->|retrieves for| Ingestor + + %% Analysis + SBOM -->|loaded into| AnalysisGraph + Package -->|normalized in| AnalysisGraph + Relationship -->|normalized in| AnalysisGraph + Query -->|searches| SBOM + Query -->|searches| Advisory + Query -->|searches| Package + Query -->|traverses| AnalysisGraph + + classDef supply fill:#333,stroke:#4caf50,stroke-width:3px + classDef security fill:#333,stroke:#f44336,stroke-width:3px + classDef product fill:#333,stroke:#2196f3,stroke-width:3px + classDef pipeline fill:#333,stroke:#ff9800,stroke-width:3px + classDef analysis fill:#333,stroke:#9c27b0,stroke-width:3px + + class SBOM,Package,File,License,Relationship supply + class Advisory,Vulnerability,Weakness,Status security + class Organization,Product,ProductVersion product + class Importer,Ingestor,SourceDocument,Storage pipeline + class AnalysisGraph,Query analysis +``` diff --git a/docs/design/generate-diagrams.sh b/docs/design/generate-diagrams.sh new file mode 100755 index 000000000..4d0e87bed --- /dev/null +++ b/docs/design/generate-diagrams.sh @@ -0,0 +1,3 @@ +#!/usr/bin/env bash +mmdc -c mermaid.json -i domain-model.md -o domain-model.png +mmdc -c mermaid.json -i data-model.md -o data-model.png --width 1200 --height 1600 --scale 6 diff --git a/docs/design/mermaid.json b/docs/design/mermaid.json new file mode 100644 index 000000000..14935e024 --- /dev/null +++ b/docs/design/mermaid.json @@ -0,0 +1,3 @@ +{ + "theme": "neutral" +}